[!NOTE]
This is a numerical
W8A16 quantization of
TeichAI/Qwen3.8-27B-Fable-Distill, a Fable-5 light tune (distill) of
Qwen/Qwen3.8-27B. All model credit belongs to
TeichAI (the Fable-5 tune) and
Qwen (the base model); refer to the
upstream model card for architecture, capabilities, and usage guidance. This repository changes numerics only.
[!TIP]
This build targets vLLM on Ampere-class GPUs where native FP8 tensor-core execution is unavailable. It retains the model's vision tower, thinking controls, native context, and the byte-preserved BF16 MTP head. The Fable-5 tune is a language-model weight change; the vision tower, recurrent GDN gates, lm_head, and MTP head fall outside the W8A16 target set and are carried through at source precision.
Relationship to the sibling quantization
This checkpoint is the Fable-5-distilled counterpart of
lued/Qwen3.8-27B-INT8-W8A16-MTP. The repositories share:
- the identical W8A16
compressed-tensors recipe (400 packed GEMMs, same BF16 preservation set, group-128 symmetric data-free RTN);
- the identical native BF16 MTP head (15 tensors, byte-identical to Qwen's original MTP — same sha256 as the sibling release);
- the identical tokenizer, chat template, processor configs, and bounded-shard layout;
- the same total checkpoint footprint (29.44 GiB).
The only deviation is the base language-model weights: this checkpoint's LM weights are TeichAI's Fable-5 light-tune weights instead of stock Qwen3.8-27B.
Fable-5 distillation (inherited from the base)
Per the upstream card, the source is a light tune of Qwen/Qwen3.8-27B on the Fable-5 datasets (armand0e/claude-fable-5-claude-code, armand0e/Fable-5-Chat) plus a large corpus of private Fable 5 data, trained with Unsloth and Hugging Face TRL. Upstream-reported benchmark results versus the base: ARC Challenge 0.637 (0.591), ARC Challenge (Easy) 0.832 (0.782), BoolQ 0.911 (0.896). These are the upstream card's claims, not measurements made here. The W8A16 step touches only the 400 GEMMs; the recurrent GDN gates (96), vision tower (333 tensors), lm_head, and MTP head are preserved at source precision.
Quantization fidelity
Measured on this exact checkpoint against its own BF16 teacher (the TeichAI
Fable-Distill source) on 2026-08-18 with tools/measure_kld.py (teacher-forced,
full-vocabulary FP32 log-softmax, FP64 accumulation, nats/token; PyTorch
2.11.0+cu128, Transformers 5.10.1, compressed-tensors 0.17.1). This is the
v0.2 revision: every preserved tensor is a bit-exact copy of the source
(input_layernorm, post_attention_layernorm, q_norm/k_norm, vision
tower, embeddings, lm_head, GDN gates); only the 400 projection GEMMs are
quantized.
| Candidate | Mean KLD vs BF16 | Top-1 agreement | Weight files |
|---|
| This W8 / BF16 A16 | 0.000700 | 98.29% | 29.44 GiB |
The KLD result measures checkpoint-weight drift over 467 teacher-forced positions (fixed 8-prompt short suite). It is not a functional quality score; tool use, coding, multimodal quality, and long-context recall require separate behavioral evaluation. A greedy KV-cache smoke generation (finite logits, non-degenerate output) passes on every release.
Short-prompt suite
| Prompt | Category | Tokens | Positions | Mean KLD | Top-1 agreement |
|---|
factual | factual | 13 | 12 | 0.000291 | 1.0000 |
code-fib | code | 87 | 86 | 0.000512 | 0.9884 |
physics-uncertainty | physics | 86 | 85 | 0.000668 | 0.9765 |
math-train | math | 58 | 57 | 0.000920 | 0.9825 |
sql-top5 | technical | 60 | 59 | 0.000823 | 1.0000 |
narrative-clock | narrative | 68 | 67 | 0.000857 | 0.9701 |
instruction-stack | instruction | 43 | 42 | 0.000774 | 0.9762 |
history-industrial | history | 60 | 59 | 0.000538 | 0.9831 |
| Token-weighted mean | | | 467 | 0.000700 | 98.29% |
kld_result.json in this repository is the single source of truth for these numbers.
Checkpoint profile
| Property | Value |
|---|
| Quantization | Data-free symmetric RTN W8A16, group size 128 |
| Structural audit | STRUCTURAL AUDIT PASS — packed set, ignore list, index contract, value round-trip (every packed tensor dequantizes within the INT8 bound), and preserved bit-identity (all 784 non-quantized tensors byte-equal to source) |
| MTP audit | MTP AUDIT PASS (15 tensors dtype-, shape-, and bit-identical to the source) |
| Quantized GEMMs | 400: 192 MLP, 64 full-attention, 144 dense GDN projections |
| Preserved precision | BF16 vision tower, lm_head, MTP (15 tensors, byte-identical to Qwen's original MTP), recurrent GDN gates (96) |
| Shard layout | 7 main shards (5.34 GB max) + model_mtp.safetensors (849,400,392 bytes) |
| Runtime | vLLM; this repository is not a GGUF checkpoint |
MTP preservation
The MTP head is the exact BF16 MTP from the original Qwen3.8-27B — never re-quantized:
- 15 tensors in
model_mtp.safetensors (849,400,392 bytes).
- sha256
1d8268aa85ace093a561e3e7b63b9d390dac1cd55a90cd55b5ec509c3c9da9fe — identical to the MTP bytes in the Qwen3.8-27B source and every family release. Verify with sha256sum model_mtp.safetensors.
- All 15 tensors verified
bfloat16, shape-equal, and bit-equal to the TeichAI Fable-Distill source (strict audit, not a count check); the built model_mtp.safetensors sha256 matches the family canonical hash.
Reproduction contract
The immutable source is TeichAI/Qwen3.8-27B-Fable-Distill commit ac2b133895580be05ee90a6a7cdcc23cdf998bd0. The exact selective W8A16 configuration is recorded in recipe.yaml and config.json. A conforming rebuild must take native MTP from that source, reproduce the source-derived packed projection set, and prove all 15 MTP tensors bit-identical before resharing for publication.
Status
v0.2 revision (2026-08-18). This checkpoint is built on the v0.2 pipeline: every preserved (non-quantized) tensor is a bit-exact copy of the TeichAI source — machine-audited (STRUCTURAL AUDIT PASS including the value round-trip and preserved bit-identity checks; MTP AUDIT PASS) — and KLD re-measured on the v0.2 bytes at 0.000700 nats/token, 98.29% top-1 (v0.1: 0.000660/98.50%; both within the family's measured range, and the difference is noise-scale — see the family CHANGELOG.md for the v0.1→v0.2 provenance story). Originally published 2026-08-18 at commit f41c1353d1cf238f631d42a1d0845d0471666f44 (v0.1); the v0.2 revision is a new commit on the same repo — prior commits remain fetchable by hash, so existing downloads keep working. Serving was validated on the v0.1 bytes: boots through llama-swap from the lued/* cache entry, loads the CompressedTensorsWNA16 → MarlinLinearKernel path, detects the native MTP draft model (method=mtp, 3 speculative tokens), and serves a smoke request with the qwen3 reasoning parser. The BF16 parent source remains cached; deletion is the pipeline's final step and has not been requested.
