DFloat11/

FLUX.1-Depth-dev-DF11

DFloat11 Compressed Model: black-forest-labs/FLUX.1-Depth-dev

🔍 How It Works

• No CPU decompression or host-device data transfer: all operations are handled entirely on the GPU.
• DFloat11 is much faster than CPU-offloading approaches, enabling practical deployment in memory-constrained environments.
• The compression is fully lossless, guaranteeing that the model’s outputs are bit-for-bit identical to those of the original model.

🔧 How to Use

1. Install or upgrade the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):
   bash

   1pip install -U dfloat11[cuda12] 2# or if you have CUDA version 11: 3# pip install -U dfloat11[cuda11]
2. Install or upgrade the diffusers and image_gen_aux packages.
   bash

   1pip install -U diffusers 2pip install git+https://github.com/asomoza/image_gen_aux.git
3. To use the DFloat11 model, run the following example code in Python:
   python

   1import torch 2from diffusers import FluxControlPipeline 3from diffusers.utils import load_image 4from image_gen_aux import DepthPreprocessor 5from dfloat11 import DFloat11Model 6 7pipe = FluxControlPipeline.from_pretrained("black-forest-labs/FLUX.1-Depth-dev", torch_dtype=torch.bfloat16) 8 9DFloat11Model.from_pretrained('DFloat11/FLUX.1-Depth-dev-DF11', device='cpu', bfloat16_model=pipe.transformer) 10 11prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts." 12control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png") 13 14processor = DepthPreprocessor.from_pretrained("LiheYoung/depth-anything-large-hf") 15control_image = processor(control_image)[0].convert("RGB") 16 17image = pipe( 18 prompt=prompt, 19 control_image=control_image, 20 height=1024, 21 width=1024, 22 num_inference_steps=30, 23 guidance_scale=10.0, 24 generator=torch.Generator().manual_seed(42), 25).images[0] 26image.save("output.png")

📄 Learn More

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11