You can deploy this model and start earning money today!

DFloat11/
DeepSeek-R1-0528-Qwen3-8B-DF11

Name: DeepSeek-R1-0528-Qwen3-8B-DF11
Availability: InStock

70% size, 100% accuracy

2504.11651

deepseek-ai/DeepSeek-R1-0528-Qwen3-8B

quantized

Views

No views yet

DFloat11 Compressed Model: `deepseek-ai/DeepSeek-R1-0528-Qwen3-8B`

This model uses DFloat11 lossless compression. It's 32% smaller than the original BFloat16 model, yet produces bit-identical outputs and runs efficiently on GPUs.

📊 Performance Comparison

Metric	DeepSeek-R1-0528-Qwen3-8B (BFloat16)	DeepSeek-R1-0528-Qwen3-8B (DFloat11)
Model Size	16.38 GB	11.16 GB
Peak GPU Memory (1024 tokens generation)	16.53 GB	12.56 GB
Generation Time (on an A100 GPU)	47 seconds	75 seconds

🔍 How It Works

We apply Huffman coding to the exponent bits of BFloat16 model weights, which are highly compressible. We leverage hardware-aware algorithmic designs to enable highly efficient, on-the-fly weight decompression directly on the GPU. Find out more in our research paper.

🔧 How to Use

Install the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):

bash
1pip install -U dfloat11[cuda12] 2# or if you have CUDA version 11: 3# pip install -U dfloat11[cuda11]
To use the DFloat11 model, run the following example code in Python:

python
1import time 2import torch 3from transformers import AutoModelForCausalLM, AutoTokenizer 4from dfloat11 import DFloat11Model 5 6model_name = "DFloat11/DeepSeek-R1-0528-Qwen3-8B-DF11" 7 8tokenizer = AutoTokenizer.from_pretrained(model_name) 9model = DFloat11Model.from_pretrained(model_name, device_map="auto") 10 11prompt = "Give me an introduction to large language model." 12messages = [ 13 {"role": "user", "content": prompt} 14] 15text = tokenizer.apply_chat_template( 16 messages, 17 tokenize=False, 18 add_generation_prompt=True, 19) 20model_inputs = tokenizer([text], return_tensors="pt").to(model.device) 21 22torch.cuda.reset_peak_memory_stats() 23 24torch.cuda.synchronize() 25start_time = time.time() 26generated_ids = model.generate( 27 **model_inputs, 28 max_new_tokens=1024, 29) 30torch.cuda.synchronize() 31end_time = time.time() 32 33output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 34 35content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n") 36 37print(f"Latency: {end_time - start_time:.2f} seconds") 38print(f"GPU Peak Memory Usage: {torch.cuda.max_memory_allocated() / 1e9:.2f} GB") 39print(f'Prompt: {prompt}') 40print(f'Response: {content}')

📄 Learn More

Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
GitHub: https://github.com/LeanModels/DFloat11
HuggingFace: https://huggingface.co/DFloat11

DFloat11/DeepSeek-R1-0528-Qwen3-8B-DF11

DFloat11 Compressed Model: deepseek-ai/DeepSeek-R1-0528-Qwen3-8B