DFloat11/

stable-diffusion-3.5-large-DF11

DFloat11 Compressed Model: stabilityai/stable-diffusion-3.5-large

💡 Key Benefits

• ✅ Bit-for-bit identical outputs to the original BFloat16 model
• 📉 ~30% reduction in model size (from 16GB → 11.3GB)
• 🧠 Lower memory requirements: now runs on 16GB GPUs
• ⚡ Minimal performance overhead: barely any slower than the full model

🔍 How It Works

• Fully GPU-based: no CPU decompression or host-device data transfer.
• DFloat11 is much faster than CPU-offloading approaches, enabling practical deployment in memory-constrained environments.
• The compression is fully lossless, guaranteeing that the model’s outputs are bit-for-bit identical to those of the original model.

🔧 How to Use

1. Install or upgrade the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):
   bash

   1pip install -U dfloat11[cuda12] 2# or if you have CUDA version 11: 3# pip install -U dfloat11[cuda11]
2. Install or upgrade the diffusers package.
   pip install -U diffusers
3. To use the DFloat11 model, run the following example code in Python:
   python

   1import torch 2from diffusers import StableDiffusion3Pipeline 3from dfloat11 import DFloat11Model 4 5pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3.5-large", torch_dtype=torch.bfloat16) 6pipe.enable_model_cpu_offload() 7 8DFloat11Model.from_pretrained('DFloat11/stable-diffusion-3.5-large-DF11', device='cpu', bfloat16_model=pipe.transformer) 9 10image = pipe( 11 "A capybara holding a sign that reads Hello World", 12 num_inference_steps=28, 13 guidance_scale=3.5, 14).images[0] 15image.save("capybara.png")

📄 Learn More

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11