Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled_PTQ

Overview

• No calibration dataset used
• One-shot quantization pipeline
• Fast and scalable

Quantization Details

• Scheme: W8A16
• Weights: INT8 (per-channel symmetric)
• Activations: FP16 / BF16
• Method: llmcompressor.oneshot
• Targets: Linear layers only

• lm_head
• visual modules
• linear attention
• mtp layers

KLD Results

Key Implementation Notes

• Uses AutoModelForImageTextToText to preserve VLM structure
• No calibration dataset (true PTQ)
• Includes key remapping for Transformers v5 compatibility

Usage (vLLM)

Notes

• Requires compressed-tensors runtime
• Not compatible with vanilla transformers loading
• Optimized for reasoning tasks

Credits

• Jackrong (fine-tune)
• TheHouseOfTheDude (quantization)