Quantization tool: NVIDIA TensorRT Model Optimizer
Weight quantization: NVFP4
KV cache quantization: FP8
Calibration size: 1024 samples
Calibration sequence length: 4096
Calibration batch size: 1
Calibration source mix:
- nohurry/Opus-4.6-Reasoning-3000x-filtered: 596 samples
- Jackrong/Qwen3.5-reasoning-700x: 178 samples
- TeichAI/claude-4.5-opus-high-reasoning-250x: 250 samples

The calibration set was converted into a single JSONL file with the source model's chat template applied before PTQ, so the activation distribution is closer to the reasoning format used by this distilled checkpoint.

Runtime Notes

Intended runtime target: SGLang with ModelOpt-compatible HF checkpoint loading
Quantization format: ModelOpt HF export, not compressed-tensors
A few unsupported or intentionally skipped modules may remain excluded by ModelOpt during export; see hf_quant_config.json for the final exclusion list

Files

model.safetensors: quantized weights
hf_quant_config.json: final quantization metadata
tokenizer and processor files inherited from the source checkpoint

rhoninseiei/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4

Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4

Quantization Summary

Runtime Notes

Files

rhoninseiei/
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4