CHATS is a next-generation framework that unifies human preference alignment with classifier-free guidance by modeling both preferred and dispreferred distributions and using a proxy-prompt-based sampling strategy for superior text–image alignment, fidelity, and aesthetic consistency. See the images generated below for examples.
-
Human-Aligned Fine-Tuning with CFG Integration
We integrate human preference alignment with classifier-free guidance sampling into a unified framework.
-
Proxy-Prompt Sampling
Leverage useful signals from both preferred and dispreferred distributions at test time.
-
Data Efficiency
State-of-the-art results across benchmarks with minimal fine-tuning effort on a small, high-quality dataset.
-
Plug-and-Play
Compatible with any diffusion backbone and existing guidance methods.
1git clone https://github.com/AIDC-AI/CHATS.git
2cd CHATS
3pip install -r requirements.txt
4
We provide pretrained CHATS checkpoints on SDXL for easy download and evaluation:
1import torch
2from pipeline import ChatsSDXLPipeline
3
4# Load CHATS-SDXL pipeline
5pipe = ChatsSDXLPipeline.from_pretrained(
6 "AIDC-AI/CHATS",
7 torch_dtype=torch.bfloat16
8)
9pipe.to("cuda")
10
11# Generate images
12images = pipe(
13 prompt=["A serene mountain lake at sunset"],
14 num_inference_steps=50,
15 guidance_scale=5,
16 seed=0
17)
18
19# Save outputs
20for i, img in enumerate(images):
21 img.save(f"output_{i}.png")
1accelerate launch --config_file=config/ac_ds_8gpu_zero0.yaml train.py \
2 --pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0 \
3 --pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix \
4 --resolution=1024 \
5 --dataloader_num_workers 16 \
6 --train_batch_size=1 \
7 --gradient_accumulation_steps=16 \
8 --max_train_steps=6000 \
9 --learning_rate=3e-09 --scale_lr --lr_scheduler=constant_with_warmup --lr_warmup_steps=100 \
10 --mixed_precision=bf16 \
11 --allow_tf32 \
12 --checkpointing_steps=100 \
13 --output_dir=output \
14 --resume_from_checkpoint latest \
15 --use_adafactor \
16 --gradient_checkpointing \
17 --dataset_name=data-is-better-together/open-image-preferences-v1-binarized \
1@inproceedings{fu2025chats,
2title={CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation},
3author={Fu, Minghao and Wang, Guo-Hua and Cao, Liangfu and Chen, Qing-Guo and Xu, Zhao and Luo, Weihua and Zhang, Kaifu},
4booktitle={International Conference on Machine Learning (ICML)},
5year={2025}
6}
The project is released under Apache License 2.0 (
http://www.apache.org/licenses/LICENSE-2.0, SPDX-License-identifier: Apache-2.0).
We used compliance checking algorithms during the training process, to ensure the compliance of the trained model to the best of our ability. Due to complex data and the diversity of language model usage scenarios, we cannot guarantee that the model is completely free of copyright issues or improper content. If you believe anything infringes on your rights or generates improper content, please contact us, and we will promptly address the matter.
