INT8 quantized version of tencent/HunyuanImage-3.0-Instruct using bitsandbytes. Reduces model size from ~160GB (BF16) to ~81GB while maintaining quality.

Model Details

Architecture: ~130B parameter Mixture-of-Experts (MoE) with 64 experts, top-8 routing
Quantization: INT8 via bitsandbytes Linear8bitLt on transformer linear layers
Original precision: BF16 → INT8 (VAE, vision model, and embeddings remain in full precision)
Variant: Instruct (chat-style, 50 diffusion steps, with classifier-free guidance)

Quality Notes

INT8 quantization preserves the model's strengths remarkably well — generated images feature correct anatomy, proper finger counts, and strong resistance to extra limbs and other common AI artifacts. The Instruct variant produces the highest quality results but requires patience (~30 minutes per generation including diffusion on a 96GB GPU).

Usage

With the generation scripts

The easiest way to use this model is with the companion generation scripts:

bash

git clone https://github.com/jamesw767/hunyuan-image-int8.git
cd hunyuan-image-int8
pip install -r requirements.txt

# Download this model
huggingface-cli download jamesw767/HunyuanImage-3-Instruct-INT8 \
    --local-dir ./HunyuanImage-3-Instruct-INT8

# Generate
python generate.py \
    --model-path ./HunyuanImage-3-Instruct-INT8 \
    --prompt "A red fox sitting in autumn leaves, realistic photography"

Direct loading with transformers

python

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

model_path = "jamesw767/HunyuanImage-3-Instruct-INT8"

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

Note: Direct loading requires the exception-based memory management trick to handle VAE decode — see the generation scripts repo for the full pipeline.

How It Was Made

The INT8 weights were created using save_quantized.py from the generation scripts:

Load the BF16 model with BitsAndBytesConfig(load_in_8bit=True)
Extract the quantized state dict, resolving meta tensors from accelerate's CPU offload hooks
Save as sharded safetensors (5GB per shard, 17 shards total)

Modules excluded from INT8 quantization (kept in original precision): vae, vision_model, vision_aligner, patch_embed, final_layer, time_embed, time_embed_2, timestep_emb, guidance_emb, timestep_r_emb, lm_head, model.wte, model.ln_f

GPU Requirements

96GB VRAM recommended: RTX PRO 6000 Blackwell, A100 80GB+, H100
48GB+ VRAM: May work with aggressive CPU offloading via --gpu-budget / --cpu-budget
System RAM: 64GB+ recommended (offloaded layers use CPU memory)

During diffusion, KV cache and MoE activations expand to ~80GB regardless of model weight placement. The generation scripts use an exception-based stack unwinding trick to free this memory before VAE decode.

Differences from Base/Distil Variants

	Base	Instruct	Instruct-Distil
Steps	20	50	8
CFG	No	Yes	No
Chat format	No	Yes	Yes
Speed (96GB GPU)	~4 min	~13 min	~90s

The Instruct variant produces the highest quality results but is the slowest. Use Instruct-Distil for fast iteration and Instruct for final renders.

Other INT8 Models

jamesw767/HunyuanImage-3-Base-INT8 — Base T2I, 20 steps
jamesw767/HunyuanImage-3-Instruct-Distil-INT8 — Distilled, 8 steps, fastest

License

This model is a derivative of tencent/HunyuanImage-3.0-Instruct, released under the Tencent Hunyuan Community License.

Important: This license does not apply in the European Union, United Kingdom, or South Korea.

Tencent Hunyuan is licensed under the Tencent Hunyuan Community License Agreement, Copyright (c) 2025 Tencent. All Rights Reserved. The trademark rights of "Tencent Hunyuan" are owned by Tencent or its affiliate.

Credits

Original model by Tencent Hunyuan
INT8 quantization and generation scripts by jamesw767