Welcome to the magical world of Diffusion models! DiffSynth-Studio is an open-source Diffusion model engine developed and maintained by the ModelScope Community team. We hope to foster technological innovation through framework construction, aggregate the power of the open-source community, and explore the rich capabilities of generative model technology!
Framework features:
- Model Support: The framework integrates mainstream open-source Diffusion models, covering image generation, video generation, audio generation, and image quality metrics models.
- VRAM Management: Dynamically schedules model parameters across disk, memory, and VRAM, allowing consumer-grade GPUs with low VRAM to run inference with large models.
- Parameter Quantization: Converts model parameters to quantized precisions such as NF4 and INT8, significantly reducing the VRAM requirements of model inference and LoRA training.
- Arbitrary Training: Almost every model that supports inference also supports training, whether base models, LoRAs, or any Adapter models with additional inputs.
- Split Training: Uses a computational graph inference engine to track every variable in the Pipeline, splitting the training process into two stages for efficient training.
References:
- Developer documentation (for humans): 中文版、English version
- Agent Skills (for AI): DiffSynth-Studio Model Integration Skills
See also:
- DiffSynth-WebUI: A lightweight LoRA training tool built on DiffSynth-Studio, enabling LoRA training of models on consumer-grade GPUs.
- ModelScope AIGC Zone (for Chinese users): Productized features powered by DiffSynth-Studio as the core inference and training engine; experience the rich potential of the open-source model ecosystem.
- ModelScope Civision (for global users): Unlock the vast potential of the open-source model ecosystem through productized capabilities powered by DiffSynth-Studio.
DiffSynth-Studio has undergone major version updates, and some old features are no longer maintained. If you need to use old features, please switch to the last historical version before the major version update.
Currently, the development personnel of this project are limited, with most of the work handled by Artiprocher and mi804. Therefore, the progress of new feature development will be relatively slow, and the speed of responding to and resolving issues is limited. We apologize for this and ask developers to understand.
-
August 19, 2026 We have released the model quantization feature. It provides a unified
QuantizeConfigentry point supporting multiple quantization backends including bitsandbytes, torchao, and comfy-kitchen, with capabilities such as online quantization, loading pre-quantized weights, mixed quantization, saving quantized models, and quantization + LoRA training. For details, please refer to the documentation. -
August 17, 2026 MiniMax-Music3 open-sourced, welcome a new member to the audio model family! Support includes text-to-music generation and low VRAM inference. For details, please refer to the documentation and example code.
-
August 7, 2026 We add support for Wan-Animate-2 in the Wan series. Given a reference image and a driving video, it makes the reference character perform the motions in the driving video, generating high-quality character animation, with both standard and distilled variants. For details, please refer to the documentation and example code.
-
August 3, 2026 MiniMax-H3 open-sourced, welcome a new member to the video model family! Support includes text-to-video-audio generation, keyframe-guided generation, reference-driven generation, low VRAM inference, and NF4-quantized inference. For details, please refer to the documentation and example code.
-
July 28, 2026 LingBot-Video open-sourced, welcome a new member to the video model family! This release includes two variants, Dense-1.3B and MoE-30B-A3B (30B total parameters, ~3B active per token), both supporting text-to-video, image-to-video and text-to-image generation, low VRAM inference, and LoRA / full training capabilities. For details, please refer to the documentation and example code. Huge thanks to NancyFyong for contributing the integration of this model!
-
July 21, 2026 We have open-sourced DiffSynth-Studio Model Integration Skills. This is a composable collection of Agent Skills that automates the entire workflow of integrating external diffusion models into DiffSynth-Studio, significantly improving the standardization and efficiency of model integration. Get started with the example!
More
-
June 29, 2026 Boogu-Image open-sourced. Support includes text-to-image generation, image editing, low VRAM inference, and training capabilities. For details, please refer to the documentation and example code.
-
June 24, 2026 Krea-2 is now open-source, and we have provided full support. For more details, please refer to the documentation and example code.
-
June 16, 2026: We have added a new Template model for ACE-Step: vocals2music. For more details, please refer to the documentation and example code.
-
June 15, 2026 We have open-sourced Image-to-LoRA V2, compressing the hours-long training process for image style LoRAs into a single model inference step, thereby exploring a new paradigm for LoRA model training. The technical report has been released. This release includes three models:
- DiffSynth-Studio/ZImage-i2L-v2: Adapted for the Z-Image model
- DiffSynth-Studio/KleinBase4B-i2L-v2: Adapted for the FLUX.2-klein-base-4B model
- DiffSynth-Studio/HidreamO1-i2L-v2: Adapted for the Hidream-O1-Image model
-
June 5, 2026 Ideogram 4 open-sourced. Support includes text-to-image inference. For details, please refer to the documentation and example code.
-
May 21, 2026: Added support for image quality metrics models, including FID, CLIP, Aesthetic, PickScore, ImageReward, HPSv2, and HPSv3. For details, refer to the documentation and example code.
-
May 18, 2026 Added CPU Offload Training support. By moving model weights layer-by-layer between CPU and GPU, it significantly reduces GPU VRAM usage during training, enabling LoRA training of large models even on consumer-grade GPUs, compatible with all models. Simply add
--enable_model_cpu_offloadto your training command to enable (currently supports single-GPU training only). For details, see the documentation. -
May 14, 2026 HiDream-O1-Image open-sourced, welcome a new member to the image model family! Support includes text-to-image generation, image editing, low VRAM inference, and training capabilities. For details, please refer to the documentation and example code.
-
April 28, 2026 We released Diffusion Templates, a plugin framework designed for Diffusion models that significantly lowers the barrier to training controllable generative models. Let's explore this cutting-edge technology together!
- Open-source code: DiffSynth-Studio
- Technical report: arXiv
- Project homepage: GitHub
- Documentation: English Version | Chinese Version
- Online demo: ModelScope
- Model collections: ModelScope | ModelScope International | HuggingFace
- Datasets: ModelScope | ModelScope International | HuggingFace
-
April 27, 2026 We support ACE-Step-1.5! Support includes text-to-music generation, low VRAM inference, and LoRA training capabilities. For details, please refer to the documentation and example code.
-
April 27, 2026: We have reinstated support for the Stable Diffusion v1.5 and SDXL models, providing academic research support exclusively for these two model types.
-
April 14, 2026 JoyAI-Image open-sourced, welcome a new member to the image editing model family! Support includes instruction-guided image editing, low VRAM inference, and training capabilities. For details, please refer to the documentation and example code.
-
March 19, 2026: Added support for openmoss/MOVA-720p and openmoss/MOVA-360p models, including training and inference capabilities. Documentation and example code are now available.
-
March 12, 2026: We have added support for the LTX-2.3 audio-video generation model. The features includes text-to-audio/video, image-to-audio/video, IC-LoRA control, audio-to-video, and audio-video inpainting. We have supported the complete inference and training functionalities. For details, please refer to the documentation and code.
-
March 3, 2026: We released the DiffSynth-Studio/Qwen-Image-Layered-Control-V2 model, which is an updated version of Qwen-Image-Layered-Control. In addition to the originally supported text-guided functionality, it adds brush-controlled layer separation capabilities.
-
March 2, 2026 Added support for Anima. For details, please refer to the documentation. This is an interesting anime-style image generation model. We look forward to its future updates.
-
February 26, 2026 Added full and lora training support for the LTX-2 audio-video generation model. See the documentation for details.
-
February 10, 2026 Added inference support for the LTX-2 audio-video generation model. See the documentation for details. Support for model training will be implemented in the future.
-
February 2, 2026 The first document of the Research Tutorial series is now available, guiding you through training a small 0.1B text-to-image model from scratch. For details, see the documentation and model. We hope DiffSynth-Studio can evolve into a more powerful training framework for Diffusion models.
-
January 27, 2026: Z-Image is released, and our Z-Image-i2L model is released concurrently. You can use it in ModelScope Studios. For details, see the documentation.
-
January 19, 2026: Added support for FLUX.2-klein-4B and FLUX.2-klein-9B models, including training and inference capabilities. Documentation and example code are now available.
-
January 12, 2026: We trained and open-sourced a text-guided image layer separation model (Model Link). Given an input image and a textual description, the model isolates the image layer corresponding to the described content. For more details, please refer to our blog post (Chinese version, English version).
-
December 24, 2025: Based on Qwen-Image-Edit-2511, we trained an In-Context Editing LoRA model (Model Link). This model takes three images as input (Image A, Image B, and Image C), and automatically analyzes the transformation from Image A to Image B, then applies the same transformation to Image C to generate Image D. For more details, please refer to our blog post (Chinese version, English version).
-
December 9, 2025 We release a wild model based on DiffSynth-Studio 2.0: Qwen-Image-i2L (Image-to-LoRA). This model takes an image as input and outputs a LoRA. Although this version still has significant room for improvement in terms of generalization, detail preservation, and other aspects, we are open-sourcing these models to inspire more innovative research. For more details, please refer to our blog.
-
December 4, 2025 DiffSynth-Studio 2.0 released! Many new features online
- Documentation online: Our documentation is still continuously being optimized and updated
- VRAM Management module upgraded, supporting layer-level disk offload, releasing both memory and VRAM simultaneously
- New model support
- Z-Image Turbo: Model, Documentation, Code
- FLUX.2-dev: Model, Documentation, Code
- Training framework upgrade
- Split Training: Supports automatically splitting the training process into two stages: data processing and training (even for training ControlNet or any other model). Computations that do not require gradient backpropagation, such as text encoding and VAE encoding, are performed during the data processing stage, while other computations are handled during the training stage. Faster speed, less VRAM requirement.
- Differential LoRA Training: This is a training technique we used in ArtAug, now available for LoRA training of any model.
- FP8 Training: FP8 can be applied to any non-training model during training, i.e., models with gradients turned off or gradients that only affect LoRA weights.
-
November 4, 2025 Supported the ByteDance/Video-As-Prompt-Wan2.1-14B model, which is trained based on Wan 2.1 and supports generating corresponding actions based on reference videos.
-
October 30, 2025 Supported the meituan-longcat/LongCat-Video model, which supports text-to-video, image-to-video, and video continuation. This model uses the Wan framework for inference and training in this project.
-
October 27, 2025 Supported the krea/krea-realtime-video model, adding another member to the Wan model ecosystem.
-
September 23, 2025 DiffSynth-Studio/Qwen-Image-EliGen-Poster released! This model was jointly developed and open-sourced by us and Taobao Experience Design Team. Built upon Qwen-Image, the model is specifically designed for e-commerce poster scenarios, supporting precise partition layout control. Please refer to our sample code.
-
September 9, 2025 Our training framework supports various training modes. Currently adapted for Qwen-Image, in addition to the standard SFT training mode, Direct Distill is now supported. Please refer to our sample code. This feature is experimental, and we will continue to improve it to support more comprehensive model training functions.
-
August 28, 2025 We support Wan2.2-S2V, an audio-driven cinematic video generation model. See ./examples/wanvideo/.
-
August 21, 2025 DiffSynth-Studio/Qwen-Image-EliGen-V2 released! Compared to the V1 version, the training dataset has been changed to Qwen-Image-Self-Generated-Dataset, so the generated images better conform to Qwen-Image's own image distribution and style. Please refer to our sample code.
-
August 21, 2025 We open-sourced the DiffSynth-Studio/Qwen-Image-In-Context-Control-Union structural control LoRA model, adopting the In Context technical route, supporting multiple categories of structural control conditions, including canny, depth, lineart, softedge, normal, and openpose. Please refer to our sample code.
-
August 20, 2025 We open-sourced the DiffSynth-Studio/Qwen-Image-Edit-Lowres-Fix model, improving the editing effect of Qwen-Image-Edit on low-resolution image inputs. Please refer to our sample code
-
August 19, 2025 Qwen-Image-Edit open-sourced, welcome a new member to the image editing model family!
-
August 18, 2025 We trained and open-sourced the Qwen-Image inpainting ControlNet model DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Inpaint. The model structure adopts a lightweight design. Please refer to our sample code.
-
August 15, 2025 We open-sourced the Qwen-Image-Self-Generated-Dataset dataset. This is an image dataset generated using the Qwen-Image model, containing 160,000
1024 x 1024images. It includes general, English text rendering, and Chinese text rendering subsets. We provide annotations for image descriptions, entities, and structural control images for each image. Developers can use this dataset to train Qwen-Image models' ControlNet and EliGen models. We aim to promote technological development through open-sourcing! -
August 13, 2025 We trained and open-sourced the Qwen-Image ControlNet model DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Depth. The model structure adopts a lightweight design. Please refer to our sample code.
-
August 12, 2025 We trained and open-sourced the Qwen-Image ControlNet model DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny. The model structure adopts a lightweight design. Please refer to our sample code.
-
August 11, 2025 We open-sourced the distilled acceleration model DiffSynth-Studio/Qwen-Image-Distill-LoRA for Qwen-Image, following the same training process as DiffSynth-Studio/Qwen-Image-Distill-Full, but the model structure has been modified to LoRA, thus being better compatible with other open-source ecosystem models.
-
August 7, 2025 We open-sourced the entity control LoRA model DiffSynth-Studio/Qwen-Image-EliGen for Qwen-Image. Qwen-Image-EliGen can achieve entity-level controlled text-to-image generation. Technical details can be found in the paper. Training dataset: EliGenTrainSet.
-
August 5, 2025 We open-sourced the distilled acceleration model DiffSynth-Studio/Qwen-Image-Distill-Full for Qwen-Image, achieving approximately 5x acceleration.
-
August 4, 2025 Qwen-Image open-sourced, welcome a new member to the image generation model family!
-
August 1, 2025 FLUX.1-Krea-dev open-sourced, a text-to-image model focused on aesthetic photography. We provided comprehensive support in a timely manner, including low VRAM layer-by-layer offload, LoRA training, and full training. For more details, please refer to ./examples/flux/.
-
July 28, 2025 Wan 2.2 open-sourced. We provided comprehensive support in a timely manner, including low VRAM layer-by-layer offload, FP8 quantization, sequence parallelism, LoRA training, and full training. For more details, please refer to ./examples/wanvideo/.
-
July 11, 2025 We propose Nexus-Gen, a unified framework that combines the language reasoning capabilities of Large Language Models (LLMs) with the image generation capabilities of diffusion models. This framework supports seamless image understanding, generation, and editing tasks.
- Paper: Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space
- GitHub Repository: https://github.com/modelscope/Nexus-Gen
- Model: ModelScope, HuggingFace
- Training Dataset: ModelScope Dataset
- Online Experience: ModelScope Nexus-Gen Studio
-
June 15, 2025 ModelScope's official evaluation framework EvalScope now supports text-to-image generation evaluation. Please refer to the best practices guide to try it out.
-
March 25, 2025 Our new open-source project DiffSynth-Engine is now open-sourced! Focused on stable model deployment, targeting industry, providing better engineering support, higher computational performance, and more stable features.
-
March 31, 2025 We support InfiniteYou, a face feature preservation method for FLUX. More details can be found in ./examples/InfiniteYou/.
-
March 13, 2025 We support HunyuanVideo-I2V, the image-to-video generation version of Tencent's open-source HunyuanVideo. More details can be found in ./examples/HunyuanVideo/.
-
February 25, 2025 We support Wan-Video, a series of state-of-the-art video synthesis models open-sourced by Alibaba. See ./examples/wanvideo/.
-
February 17, 2025 We support StepVideo! Advanced video synthesis model! See ./examples/stepvideo.
-
December 31, 2024 We propose EliGen, a new framework for entity-level controlled text-to-image generation, supplemented with an inpainting fusion pipeline, extending its capabilities to image inpainting tasks. EliGen can seamlessly integrate existing community models such as IP-Adapter and In-Context LoRA, enhancing their versatility. For more details, see ./examples/EntityControl.
- Paper: EliGen: Entity-Level Controlled Image Generation with Regional Attention
- Model: ModelScope, HuggingFace
- Online Experience: ModelScope EliGen Studio
- Training Dataset: EliGen Train Set
-
December 19, 2024 We implemented advanced VRAM management for HunyuanVideo, enabling video generation with resolutions of 129x720x1280 on 24GB VRAM or 129x512x384 on just 6GB VRAM. More details can be found in ./examples/HunyuanVideo/.
-
December 18, 2024 We propose ArtAug, a method to improve text-to-image models through synthesis-understanding interaction. We trained an ArtAug enhancement module for FLUX.1-dev in LoRA format. This model incorporates the aesthetic understanding of Qwen2-VL-72B into FLUX.1-dev, thereby improving the quality of generated images.
- Paper: https://arxiv.org/abs/2412.12888
- Example: https://github.com/modelscope/DiffSynth-Studio/tree/main/examples/ArtAug
- Model: ModelScope, HuggingFace
- Demo: ModelScope, HuggingFace (coming soon)
-
October 25, 2024 We provide extensive FLUX ControlNet support. This project supports many different ControlNet models and can be freely combined, even if their structures are different. Additionally, ControlNet models are compatible with high-resolution optimization and partition control technologies, enabling very powerful controllable image generation. See
./examples/ControlNet/. -
October 8, 2024 We released extended LoRAs based on CogVideoX-5B and ExVideo. You can download this model from ModelScope or HuggingFace.
-
August 22, 2024 This project now supports CogVideoX-5B. See here. We provide several interesting features for this text-to-video model, including:
- Text-to-video
- Video editing
- Self super-resolution
- Video interpolation
-
August 22, 2024 We implemented an interesting brush feature that supports all text-to-image models. Now you can create stunning images with the assistance of AI using the brush!
- Use it in our WebUI.
-
August 21, 2024 DiffSynth-Studio now supports FLUX.
- Enable CFG and high-resolution inpainting to improve visual quality. See here
- LoRA, ControlNet, and other addon models will be released soon.
-
June 21, 2024 We propose ExVideo, a post-training fine-tuning technique aimed at enhancing the capabilities of video generation models. We extended Stable Video Diffusion to achieve long video generation of up to 128 frames.
- Project Page
- Source code has been released in this repository. See
examples/ExVideo. - Model has been released at HuggingFace and ModelScope.
- Technical report has been released at arXiv.
- You can try ExVideo in this demo!
-
June 13, 2024 DiffSynth Studio has migrated to ModelScope. The development team has also transitioned from "me" to "us". Of course, I will still participate in subsequent development and maintenance work.
-
January 29, 2024 We propose Diffutoon, an excellent cartoon coloring solution.
- Project Page
- Source code has been released in this project.
- Technical report (IJCAI 2024) has been released at arXiv.
-
December 8, 2023 We decided to initiate a new project aimed at unleashing the potential of diffusion models, especially in video synthesis. The development work of this project officially began.
-
November 15, 2023 We propose FastBlend, a powerful video deflickering algorithm.
-
October 1, 2023 We released an early version of the project named FastSDXL. This was an initial attempt to build a diffusion engine.
- Source code has been released at GitHub.
- FastSDXL includes a trainable OLSS scheduler to improve efficiency.
-
August 29, 2023 We propose DiffSynth, a video synthesis framework.
- Project Page.
- Source code has been released at EasyNLP.
- Technical report (ECML PKDD 2024) has been released at arXiv.
Install from source (recommended):
git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
pip install -e .
For more installation methods and instructions for non-NVIDIA GPUs, please refer to the Installation Guide.
Download Source Configuration
Before model inference and training, you can configure the model download source and other options through environment variables.
This project downloads models from ModelScope by default. For users outside China, you can download models from the ModelScope International site with the following configuration:
export MODELSCOPE_ENDPOINT=https://modelscope.aiTo download models from HuggingFace, please modify the environment variables. Note that model IDs may differ across platforms:
export DIFFSYNTH_DOWNLOAD_SOURCE="huggingface"
DiffSynth-Studio redesigns the inference and training pipelines for mainstream Diffusion models (including FLUX, Wan, etc.), enabling efficient memory management and flexible model training.
Quick start: experience popular and the latest models:
| Architecture | Model ID | Inference | Low VRAM Inference | Full Training | Validation After Full Training | LoRA Training | Validation After LoRA Training |
|---|---|---|---|---|---|---|---|
| MiniMax-H3 | MiniMax/MiniMax-H3: FL2VA | code | code | code | code | code | code |
| MiniMax-H3 | DiffSynth-Studio/MiniMax-H3-NF4: FL2VA pruned | code | code | - | - | code | code |
| ACE-Step | ACE-Step/acestep-v15-xl-sft | code | code | code | code | code | code |
| Z-Image | Tongyi-MAI/Z-Image-Turbo | code | code | code | code | code | code |
| Krea-2 | krea/Krea-2-Raw | code | code | code | code | code | code |
| Krea-2 | krea/Krea-2-Turbo | code | code | code | code | code | code |
Model overview:
- Image generation
- Boogu-Image: Documentation, Example code
- Krea-2: Documentation, Example code
- Ideogram 4: Documentation, Example code
- HiDream-O1-Image: Documentation, Example code
- JoyAI-Image: Documentation, Example code
- ERNIE-Image: Documentation, Example code
- FLUX.2: Documentation, Example code
- Z-Image: Documentation, Example code
- Anima: Documentation, Example code
- Qwen-Image: Documentation, Example code
- FLUX.1: Documentation, Example code
- Stable Diffusion XL: Documentation, Example code
- Stable Diffusion: Documentation, Example code
- Video generation
- MiniMax-H3: Documentation, Example code
- LingBot-Video: Documentation, Example code
- LTX-2: Documentation, Example code
- Wan: Documentation, Example code
- Audio generation
- MiniMax-Music3: Documentation, Example code
- ACE-Step: Documentation, Example code
- Evaluation models: Documentation, Example code
We believe that a well-developed open-source code framework can lower the threshold for technical exploration. We have achieved many interesting technologies based on this codebase. Perhaps you also have many wild ideas, and with DiffSynth-Studio, you can quickly realize these ideas.
TreeAdapter: A Model System Built from Structured LoRAs
A model system composed of 10,000+ LoRAs challenges the precise generation of tens of thousands of rare species.
- Paper: TreeAdapter: Hierarchical Taxonomy-Guided Adapter Composition for Fine-Grained Species Image Generation
- Model: ModelScope
Image-to-LoRA: Compressing Model Training into Model Inference
True Meta Learning: feed a dataset of images into one end of the model, and the trained LoRA model comes out the other end.
- Paper: Compressing Image Style Training into a Single Model Forward
- Model:
- DiffSynth-Studio/ZImage-i2L-v2: Adapted for the Z-Image model
- DiffSynth-Studio/KleinBase4B-i2L-v2: Adapted for the FLUX.2-klein-base-4B model
- DiffSynth-Studio/HidreamO1-i2L-v2: Adapted for the Hidream-O1-Image model
| Input example 1 | Output example 1 | Input example 2 | Output example 2 |
|---|---|---|---|
 |
 |
 |
 |
Diffusion-Templates: A Plugin-Based Controllable Generation Framework
One framework that turns every controllable generation capability into a plugin, allowing multiple models to combine and emerge with rich generation capabilities.
- Paper: Diffusion Templates: A Unified Plugin Framework for Controllable Diffusion
- Project homepage: GitHub
- Documentation: English Version | Chinese Version
- Online demo: ModelScope
- Model collections: ModelScope | ModelScope International | HuggingFace
- Datasets: ModelScope | ModelScope International | HuggingFace
| Reference image | Local editing | Style transfer | Sharpness enhancement |
|---|---|---|---|
 |
 |
 |
 |
Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation
Trade inference time for higher quality of generated content.
- Paper: Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation
- Sample Code: /docs/en/Research_Tutorial/inference_time_scaling.md
| FLUX.1-dev | FLUX.1-dev + SES | Qwen-Image | Qwen-Image + SES |
|---|---|---|---|
 |
 |
 |
 |
VIRAL: Visual In-Context Reasoning via Analogy in Diffusion Transformers
Transform image 3 into image 4 based on the change from image 1 to image 2 — the emergent capability of image editing models.
- Paper: VIRAL: Visual In-Context Reasoning via Analogy in Diffusion Transformers
- Sample code: /examples/qwen_image/model_inference/Qwen-Image-Edit-2511-ICEdit.py
- Model: ModelScope
| Example 1 | Example 2 | Query | Output |
|---|---|---|---|
 |
 |
 |
 |
AttriCtrl: Attribute Intensity Control for Image Generation Models
Numerical attributes can also precisely control image generation models.
- Paper: AttriCtrl: Fine-Grained Control of Aesthetic Attribute Intensity in Diffusion Models
- Sample Code: /examples/flux/model_inference/FLUX.1-dev-AttriCtrl.py
- Model: ModelScope
| brightness scale = 0.1 | brightness scale = 0.3 | brightness scale = 0.5 | brightness scale = 0.7 | brightness scale = 0.9 |
|---|---|---|---|---|
 |
 |
 |
 |
 |
AutoLoRA: Automated LoRA Retrieval and Fusion
LoRA is a product that unifies needs and solutions — how can we make better use of these LoRAs?
- Paper: AutoLoRA: Automatic LoRA Retrieval and Fine-Grained Gated Fusion for Text-to-Image Generation
- Sample Code: /examples/flux/model_inference/FLUX.1-dev-LoRA-Fusion.py
- Model: ModelScope
| LoRA 1 | LoRA 2 | LoRA 3 | LoRA 4 | |
|---|---|---|---|---|
| LoRA 1 |  |
 |
 |
 |
| LoRA 2 | |
 |
 |
 |
| LoRA 3 | |
|
 |
 |
| LoRA 4 | |
|
|
 |
Nexus-Gen: Unified Architecture for Image Understanding, Generation, and Editing
What happens when a single model combines image understanding, generation, and editing capabilities?
- Detailed Page: https://github.com/modelscope/Nexus-Gen
- Paper: Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space
- Model: ModelScope, HuggingFace
- Dataset: ModelScope Dataset
- Online Experience: ModelScope Nexus-Gen Studio
ArtAug: Aesthetic Enhancement for Image Generation Models
A single LoRA that significantly enhances detail and aesthetics.
- Detailed Page: ./examples/ArtAug/
- Paper: ArtAug: Enhancing Text-to-Image Generation through Synthesis-Understanding Interaction
- Model: ModelScope, HuggingFace
- Online Experience: ModelScope AIGC Tab
| FLUX.1-dev | FLUX.1-dev + ArtAug LoRA |
|---|---|
 |
 |
EliGen: Precise Image Partition Control
How can region-based layers control the position of content in an image?
- Paper: EliGen: Entity-Level Controlled Image Generation with Regional Attention
- Sample Code: /examples/flux/model_inference/FLUX.1-dev-EliGen.py
- Model: ModelScope, HuggingFace
- Online Experience: ModelScope EliGen Studio
- Dataset: EliGen Train Set
| Entity Control Region | Generated Image |
|---|---|
 |
 |
ExVideo: Extended Training for Video Generation Models
If a video generation model can only generate 25 frames, how can we make it generate longer videos?
- Project Page: Project Page
- Paper: ExVideo: Extending Video Diffusion Models via Parameter-Efficient Post-Tuning
- Sample Code: Please refer to the older version
- Model: ModelScope, HuggingFace
github_title.mp4
Diffutoon: High-Resolution Anime-Style Video Rendering
I don't care what you say, I just love anime!
- Project Page: Project Page
- Paper: Diffutoon: High-Resolution Editable Toon Shading via Diffusion Models
- Sample Code: Please refer to the older version
The Diffutoon examples are from the pre-2.0 codebase and are not included in the current main branch. Use the older source tree above with the corresponding older version of DiffSynth-Studio.
Diffutoon.mp4
DiffSynth: The Original Version of This Project
In the era before video generation models, how could image generation models be used to process videos?
- Project Page: Project Page
- Paper: DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis
- Sample Code: Please refer to the older version
winter_stone.mp4
| Discord：https://discord.gg/Mm9suEeUDc |
|---|
 |