Meta's Llama 3.3 70B instruction-tuned model, optimized for multilingual dialogue. Outperforms many open-source and closed chat models on common benchmarks. Supports English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Quantization Details

Parameter	Value
Format	EXL3
Bits per weight	5.0 bpw
Head bits	6
Tool	ExLlamaV3 latest (dev)
Calibration	Default ExLlamaV3 calibration dataset (250 rows, 2048 columns)
Date quantized	2026-05-05

Quality

Great balance of size and quality

EXL3 uses trellis/codebook quantization which provides better quality than EXL2 at equivalent bitrates.

Hardware Requirements

VRAM: ~46 GB — 2x RTX 3090/4090
Context: Up to 128K tokens supported

Usage

Requires ExLlamaV3 or TabbyAPI with EXL3 support.

hf download anthonyw448/Llama-3.3-70B-Instruct-EXL3-5.0bpw-h6 --local-dir ./Llama-3.3-70B-Instruct-EXL3-5.0bpw-h6

Other Bitrates From This Author

Bitrate	VRAM	Quality
5.0bpw h6 (this repo)	~46 GB — 2x RTX 3090/4090	Great balance of size and quality

Original Model

See meta-llama/Llama-3.3-70B-Instruct for full model details, license, and usage.

Quantized 2026-05-05 by anthonyw448