DFloat11/

FLUX.1-dev-DF11

DFloat11 Compressed Model: black-forest-labs/FLUX.1-dev

🔍 How It Works

• No CPU decompression or host-device data transfer --- all operations are handled entirely on the GPU.
• DFloat11 is much faster than CPU-offloading approaches, enabling practical deployment in memory-constrained environments.
• The compression is fully lossless, guaranteeing that the model’s outputs are bit-for-bit identical to those of the original model.

🔧 How to Use

1. Install or upgrade the DFloat11 package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):
   bash

   1pip install dfloat11[cuda12] 2# or if you have CUDA version 11: 3# pip install dfloat11[cuda11]
2. Install or upgrade the diffusers package:
   pip install -U diffusers
3. Save the following code as a Python file flux1.py:
   python

   1import torch 2from diffusers import FluxPipeline, FluxTransformer2DModel 3from dfloat11 import DFloat11Model 4from transformers.modeling_utils import no_init_weights 5 6with no_init_weights(): 7 transformer = FluxTransformer2DModel.from_config( 8 FluxTransformer2DModel.load_config( 9 "black-forest-labs/FLUX.1-dev", subfolder="transformer" 10 ) 11 ).to(torch.bfloat16) 12 13pipe = FluxPipeline.from_pretrained( 14 "black-forest-labs/FLUX.1-dev", 15 transformer=transformer, 16 torch_dtype=torch.bfloat16 17) 18 19DFloat11Model.from_pretrained( 20 'DFloat11/FLUX.1-dev-DF11', 21 device='cpu', 22 bfloat16_model=pipe.transformer, 23) 24 25pipe.enable_model_cpu_offload() 26 27prompt = "A scenic landscape with mountains, a river, and a clear sky." 28image = pipe( 29 prompt, 30 width=1024, 31 height=1024, 32 guidance_scale=3.5, 33 num_inference_steps=50, 34 max_sequence_length=512, 35 generator=torch.Generator(device="cuda").manual_seed(0) 36).images[0] 37 38image.save("image.png")
4. Run python flux1.py in your terminal.

📄 Learn More

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11