1 Star 0 Fork 0

homer_1943/VMix

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
贡献代码
同步代码
fenfenfenfan-fenfenfenfan fenfenfenfan init f8d6917 1个月前
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README

VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control

Build Build

Shaojin Wu,1 Fei Ding,1,* Mengqi Huang,1,2 Wei Liu,1 Qian He1
1 ByteDance Inc.   2 University of Science and Technology of China

📖 Introduction

We propose VMix, a plug-and-play aesthetics adapter, to upgrade the quality of generated images while maintaining generality across visual concepts by (1) disentangling the input text prompt into the content description and aesthetic description by the initialization of aesthetic embedding, and (2) integrating aesthetic conditions into the denoising process through value-mixed cross-attention, with the network connected by zero-initialized linear layers. VMix outperforms other state-of-the-art methods and is flexible enough to be applied to community modules (e.g., LoRA, ControlNet, and IPAdapter) for better visual performance without retraining.

🎨 Examples

Qualitative comparison between results with VMix(on the right) and without VMix(on the left)

Aesthetic Fine-grained Control
For more visual results, go checkout our Project Page

🔥Updates

We will open source this project as soon as possible. Thank you for your patience and support! 🌟

  • Release arXiv paper. Check the details here.
  • Release inference code(Coming soon).
  • Release model checkpoints.
  • Release ComfyUI node.

Citation

If VMix is helpful, please help to ⭐ the repo.

If you find this project useful for your research, please consider citing our paper:

@misc{wu2024vmix,
    title={VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control},
    author={Shaojin Wu and Fei Ding and Mengqi Huang and Wei Liu and Qian He},
    year={2024},
    eprint={2412.20800},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

空文件

简介

暂无描述 展开 收起
取消

发行版

暂无发行版

贡献者

全部

近期动态

不能加载更多了
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/homer-1943/VMix.git
git@gitee.com:homer-1943/VMix.git
homer-1943
VMix
VMix
main

搜索帮助