1import torch
2from diffusers import StableDiffusionPipeline, LCMScheduler
34# Load base SD1.5 model5pipe = StableDiffusionPipeline.from_pretrained(6"runwayml/stable-diffusion-v1-5",7 torch_dtype=torch.float16
8)9pipe.to("cuda")1011# Load this LCM-LoRA checkpoint12pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1600")1314# IMPORTANT: Use LCM scheduler15pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)1617# Generate with just 4-6 steps!18prompt ="a portrait of a cat wearing a detective hat, film noir style"19image = pipe(20 prompt=prompt,21 num_inference_steps=6,22 guidance_scale=1.023).images[0]2425image.save("output.png")
Recommended Settings
python
1num_inference_steps =6# Optimal for this checkpoint2guidance_scale =1.0# Required for LCM
Training Details
Parameter
Value
Checkpoint
1600
Base Model
runwayml/stable-diffusion-v1-5
Training Steps
1600
Dataset
Mercity/laion-subset
LoRA Rank
96
LoRA Alpha
96
Resolution
512×512
Batch Size
64
Learning Rate
1e-4
Optimizer
AdamW
Sample Outputs
The comparison grid above shows outputs from this checkpoint at 2, 4, and 6 inference steps, compared to standard SD1.5 at 50 steps.
Prompts included:
Futuristic cyberpunk city with neon lights and rain reflections
Portrait of a cat wearing a detective hat, film noir style
Cozy coffee shop interior with warm lighting and plants
Ancient Japanese temple in misty mountain landscape at sunrise
Majestic lion on rock overlooking African savannah at sunset
Magical forest with glowing blue mushrooms and fireflies
Vintage red steam locomotive crossing stone viaduct over canyon
View individual samples
All sample images for this checkpoint are available in the samples/ directory.
Out-of-Distribution (OOD) Validation Images
To test generalization beyond the training distribution, we generated images for 5 OOD prompts that are deliberately different from training prompts:
🐠 Underwater Scene
"underwater coral reef with colorful fish and sea anemones, crystal clear water, natural sunlight filtering through"
Tests: Water effects, marine life, underwater lighting (not in training)
🚀 Space/Astronomy
"astronaut floating in space with earth in background, stars and galaxies, cinematic lighting, 4k"
Tests: Zero gravity, cosmic environment, space rendering (not in training)
🍰 Food Photography
"gourmet chocolate cake with berries on elegant plate, professional food photography, soft studio lighting"
Tests: Food textures, studio lighting, product photography (not in training)
👴 Human Portrait
"close-up portrait of elderly man with weathered face and kind eyes, dramatic side lighting, black and white"
Tests: Human facial features, skin texture, B&W conversion (training had cat portrait, not human closeup)
🎨 Abstract Art
"abstract watercolor painting with flowing colors, pink and blue gradient, artistic ethereal style"
Tests: Non-representational art, color blending (training was all representational)
Why OOD Validation? These prompts test whether the model truly learned general concepts rather than just memorizing training prompts. Good OOD performance indicates robust generalization.
All validation images can be found in the validation/ directory. See validation/prompts.txt for the complete list of prompts used.
Performance
Speed Comparison
Method
Steps
Time (A100)
Time (RTX 3090)
SD1.5 Default
50
~15s
~25s
SD1.5 Fast
25
~8s
~13s
LCM-LoRA (this)
6
~2s
~3s
LCM-LoRA (this)
4
~1.5s
~2s
Quality Progression
2 steps: Fast, captures main composition
4 steps: Good balance, suitable for most cases
6 steps: Best quality (recommended)
8 steps: Slightly better, diminishing returns
Series Information
Training Progression
This checkpoint is part of a training series showing LCM-LoRA evolution:
1# Download all checkpoints for comparison2huggingface-cli download Mercity/lcm-lora-sd1.5-400
3huggingface-cli download Mercity/lcm-lora-sd1.5-800
4huggingface-cli download Mercity/lcm-lora-sd1.5-1200
5huggingface-cli download Mercity/lcm-lora-sd1.5-1600
Usage Tips
For Best Results
Always use LCMScheduler - Required for LCM
Set guidance_scale=1.0 - CFG doesn't work with LCM
Use 4-8 steps - Optimal range is 6 steps
Same prompts as SD1.5 - No special prompting needed
Checkpoint Selection
Testing/comparison? Try different checkpoints to find your preference
Different characteristics: Each checkpoint has unique qualities
Training progression: See how the model evolves with more training
Limitations
Trained on 512×512 resolution (best results at this size)
Requires LCMScheduler - other schedulers won't work
guidance_scale must be 1.0 (CFG incompatible with LCM)
Each checkpoint has slightly different characteristics
Citation
If you use this model in your research, please cite:
bibtex
1@article{luo2023latent,
2 title={Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference},
3 author={Luo, Simian and Tan, Yiqin and Huang, Longbo and Li, Jian and Zhao, Hang},
4 journal={arXiv preprint arXiv:2310.04378},
5 year={2023}
6}
78@article{hu2021lora,
9 title={LoRA: Low-Rank Adaptation of Large Language Models},
10 author={Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
11 journal={arXiv preprint arXiv:2106.09685},
12 year={2021}
13}
License
This model is released under the same license as Stable Diffusion v1.5: