DFloat11/

FLUX.1-Fill-dev-DF11

DFloat11 Compressed Model: black-forest-labs/FLUX.1-Fill-dev

🔍 How It Works

• No CPU decompression or host-device data transfer: all operations are handled entirely on the GPU.
• DFloat11 is much faster than CPU-offloading approaches, enabling practical deployment in memory-constrained environments.
• The compression is fully lossless, guaranteeing that the model’s outputs are bit-for-bit identical to those of the original model.

🔧 How to Use

1. Install or upgrade the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):
   bash

   1pip install -U dfloat11[cuda12] 2# or if you have CUDA version 11: 3# pip install -U dfloat11[cuda11]
2. Install or upgrade the diffusers package.
   pip install -U diffusers
3. To use the DFloat11 model, run the following example code in Python:
   python

   1import torch 2from diffusers import FluxFillPipeline 3from diffusers.utils import load_image 4from dfloat11 import DFloat11Model 5 6image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png") 7mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png") 8 9pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16) 10pipe.enable_model_cpu_offload() 11 12DFloat11Model.from_pretrained('DFloat11/FLUX.1-Fill-dev-DF11', device='cpu', bfloat16_model=pipe.transformer) 13 14image = pipe( 15 prompt="a white paper cup", 16 image=image, 17 mask_image=mask, 18 height=1632, 19 width=1232, 20 guidance_scale=30, 21 num_inference_steps=50, 22 max_sequence_length=512, 23 generator=torch.Generator("cpu").manual_seed(0) 24).images[0] 25image.save(f"flux-fill-dev.png")

📄 Learn More

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11