Metric	Value	Note
Speed	~1.3 sec/image	6 steps, T4 GPU, float16
Speedup	3.1x faster	vs v1.5 (20 steps)
Quality	Equivalent	Same aesthetic quality
Steps	6 (LCM)	vs 20 standard

🎨 Model Overview

This LCM-accelerated LoRA specializes in anime character generation:

✅ High-quality anime faces and character designs
✅ Fast inference (LCM 6-step generation)
✅ Compatible with Stable Diffusion v1.5
✅ Robust prompts (Gao et al. 2306.13103)

🎓 Research Foundation

This model is based on:

LCM-LoRA (Luo et al. 2311.05556)
- Latent Consistency Models for acceleration
- Skip-step training with Augmented PF-ODE
RobustPromptGenerator (Gao et al. 2306.13103)
- Multi-redundant prompt generation
- Typography robustness
ControlNet Integration (Zhang et al. 2302.05543)
- Sketch-to-image pipeline support
- Lineart mode for anime sketches

💻 Usage

Installation

pip install diffusers transformers torch peft

Basic Generation

python

from diffusers import StableDiffusionPipeline
from peft import PeftModel
import torch

# Load base model
base_model_id = "runwayml/stable-diffusion-v1-5"
adapter_id = "Shion1124/anime-character-lcm-lora"

pipe = StableDiffusionPipeline.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LCM-LoRA adapter
pipe.unet = PeftModel.from_pretrained(
    pipe.unet,
    adapter_id,
    adapter_name="lcm"
)

# Enable LCMScheduler for fast inference
from diffusers import LCMScheduler
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

# Generate
prompt = "anime girl, beautiful face, masterpiece, best quality"
image = pipe(
    prompt=prompt,
    num_inference_steps=6,     # LCM: 4-8 steps recommended
    guidance_scale=1.5,        # LCM optimal value
    height=512,
    width=512
).images[0]

image.save("output.png")

With RobustPromptGenerator

python

# (See: https://github.com/Shion1124/anime-character-generator)
from prompt_optimizer_v2 import RobustPromptGenerator

optimizer = RobustPromptGenerator(use_google_api=True)
robust_prompt = optimizer.optimize_prompt(
    request="happy anime girl with long hair",
    mode="lcm_controlnet"
)

image = pipe(
    prompt=robust_prompt['prompt_variants'][0],
    num_inference_steps=6,
    guidance_scale=1.5
).images[0]

With ControlNet (Sketch-to-Image)

python

from diffusers import (
    StableDiffusionControlNetPipeline,
    ControlNetModel, LCMScheduler
)

controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/sd-controlnet-lineart"
)

pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LCM + anime LoRA
pipe.load_lora_weights(adapter_id)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

# Generate from sketch
image = pipe(
    prompt="anime girl, masterpiece",
    image=sketch_image,
    num_inference_steps=6,
    guidance_scale=1.5,
    controlnet_conditioning_scale=0.8
).images[0]

📚 Training Details

Base Model

Model: Stable Diffusion v1.5
Framework: Diffusers library

LoRA Configuration

Rank (r): 64
Alpha: 32
LCM Integration: LCM-LoRA (latent-consistency/lcm-lora-sdv1-5)

Acceleration

Method: LCM (Latent Consistency Models)
Skip-step Training: Multi-step consistency
Augmented PF-ODE: Guidance encoding

Speed Optimization

Inference Steps: 6 (vs 20 standard)
Device: GPU (T4 or better)
Data Type: float16 (for memory efficiency)

📖 Architecture

Input Prompt (Text)
    ↓
[Layer 1: CLIP Text Encoder]
    ↓
[Layer 2: SD v1.5 UNet + Anime LoRA]
    ↓
[Layer 3: LCM-LoRA Scheduler]
    ↓
Output Image (512×512)

Total Time: ~1.3 seconds per image

⚖️ License & Attribution

License

LoRA Model: MIT License
Base Model: Stable Diffusion v1.5 (OpenRAIL M License)

Citation

If you use this model in research, please cite:

bibtex

@article{luo2023lcm,
  title={LCM-LoRA: A Universal Stable-Diffusion Acceleration Module},
  author={Luo, Simian and Sun, Yiqin and Kang, Longxiang and ...},
  journal={arXiv preprint arXiv:2311.05556},
  year={2023}
}

@article{gao2024robustness,
  title={Evaluating Robustness of Text-to-Image Models},
  author={Gao, Yiming and Chen, ...},
  journal={arXiv preprint arXiv:2306.13103},
  year={2024}
}

Acknowledgments

Base Model: Hugging Face Stable Diffusion v1.5
Acceleration: Latent Consistency Models (Luo et al.)
Robustness: Text-to-Image Robustness (Gao et al.)
ControlNet: Sketch Control (Zhang et al.)

📝 Notes

This is a LoRA adapter and requires the base model
Works with any anime-friendly prompt
Can be combined with other LoRAs using set_adapters()
GPU-based inference recommended for best performance

🐛 Issues & Feedback

If you encounter issues:

Check base model install: pip install diffusers
Verify GPU memory: 6GB+ recommended
Try adjusting num_inference_steps (4-8)

Created: 2026-03-05 13:44:11 Model Type: LoRA Adapter (Anime Character Generation) Framework: Diffusers + PEFT Base: Stable Diffusion v1.5