[!IMPORTANT]
Runtime: standard vLLM
fp8 quant_method (e4m3,
weight_block_size [128, 128], dynamic activations). All evaluation below was run on
tcclaviger/vllm:latest on
RDNA 4 (gfx12xx).
Not validated on any other hardware at this time.
Tess-FP8
Block-wise W8A8 FP8 (e4m3fn) quantization of
Tess-4-27B by Migel Tissera — an
agentic, thinking-native finetune of Qwen3.6-27B. Sibling quants:
Tess-27B-RFI (int8+4-bit hybrid) and
Tess-27B-RFA (all-attention 4-bit).
All credit for the model to its author; this repo only changes the numerics.
Quantization by component
- All attention and MLP linear weights (self_attn q/k/v/o, MLP gate/up/down, GDN linear-attention in_proj_qkv / in_proj_z / out_proj) — FP8 e4m3fn, block-wise 128×128, per-block bf16 dequant scales (
weight_scale_inv), dynamic activation scheme (W8A8).
- MTP speculative-decode head — its MLP and attention projections are FP8 by the same scheme; its fc and norms stay bf16.
- Kept in bf16 (not quantized) — vision encoder, GDN
A_log/conv1d/dt_bias/in_proj_a/in_proj_b, embeddings, norms, and the lm_head.
Tensor layout and naming match the Qwen3.6-27B-FP8 production reference exactly.
Serving context — 512K via YaRN
All evaluation below was run while serving at --max-model-len 524288 (512K tokens),
extended from the native 256K window with YaRN via --hf-overrides:
1{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 2.0,
2 "original_max_position_embeddings": 262144, "mrope_interleaved": true,
3 "mrope_section": [11, 11, 10], "partial_rotary_factor": 0.25,
4 "rope_theta": 10000000}}}
Evaluation results
Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026):
Qwen3.6-27B (bf16 base) →
Tess-4-27B (the tune, bf16) →
Tess-27B-RFI (int8+4-bit hybrid) →
Tess-27B-RFA (all-4-bit) →
Tess-FP8 (this build, W8A8 block-128 FP8),
with
Qwen3.6-35B-A3B (MoE, bf16) as a comparative reference.
Bold marks the best score in each row (ties all bolded).
Quantization cost (W8A8 FP8 block-128) — Tess-4-27B → Tess-FP8
| Metric | Change |
|---|
| Checkpoint size | 55.6 → 29.0 GB (−48%) |
| WikiText-2 perplexity | 6.669 → 6.663 (≈ flat) |
| Codeneedle overall recall | 97.7% → 97.9% (≈ flat) |
| MC accuracy (4 tasks) | ≈ flat (−0.4 to +0.3 pp) |
| Tool-eval / GSM8K / MMLU / IFEval | 87 / 98% / 76% / 90% |
| Decode @ conc 1 (ISL 128) | 59.9 → 86.0 tok/s (+44%) |
| Decode @ conc 50 (ISL 128) | 528 → 560 tok/s (+6%) |
Quality
| Metric | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | Tess-27B-RFA | Tess-FP8 | Qwen3.6-35B-A3B |
|---|
| WikiText-2 PPL (n_ctx 2048, lower is better) | 7.0559 | 6.6691 | 6.6632 | 6.6292 | 6.6627 | 6.5092 |
| ARC-Challenge (acc_norm) | 59.30% | 60.84% | 60.41% | 60.32% | 60.49% | 55.20% |
| ARC-Easy (acc_norm) | 75.93% | 77.53% | 77.40% | 78.87% | 77.82% | 71.13% |
| Winogrande (acc) | 77.51% | 77.43% | 77.51% | 76.80% | 77.66% | 73.40% |
| HellaSwag (acc_norm) | 84.12% | 84.21% | 84.27% | 84.05% | 84.13% | 82.95% |
Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.
Long-context positional recall (codeneedle)
Verbatim function recall under 10K–80K-token contexts.
| Corpus | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | Tess-27B-RFA | Tess-FP8 | Qwen3.6-35B-A3B |
|---|
| Python | 100% | 100% | 100% | 100% | 99.55% | 99.09% |
| C++ | 98.12% | 98.12% | 98.44% | 98.75% | 98.75% | 98.44% |
| Rust | 99.69% | 99.69% | 99.69% | 99.69% | 99.69% | 99.38% |
| JS (~80K tokens) | 93.44% | 93.13% | 93.75% | 93.75% | 93.44% | 92.19% |
| Tools | 98.26% | 99.57% | 99.57% | 99.57% | 99.57% | 93.48% |
| Overall recall | 97.81% | 97.73% | 97.97% | 98.05% | 97.86% | 97.28% |
Tool calling & accuracy benches
| Bench | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | Tess-27B-RFA | Tess-FP8 | Qwen3.6-35B-A3B |
|---|
| tool-eval final (full 69, TC-61 excl) | 86 | 85 | 87 | 86 | 87 | 90 |
| GSM8K (50q) | 98.0% | 94.0% | 98.0% | 98.0% | 98.0% | 96.0% |
| MMLU (50q) | 74.0% | 76.0% | 82.0% | 80.0% | 76.0% | 64.0% |
| IFEval (20 prompts, prompt-level) | 90.0% | 90.0% | 90.0% | 95.0% | 90.0% | 90.0% |
Decode throughput — tok/s output (ISL 128 / ISL 512)
vllm bench serve, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.
| Concurrency | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | Tess-27B-RFA | Tess-FP8 | Qwen3.6-35B-A3B |
|---|
| 1 | 57.0 / 61.4 | 59.9 / 63.8 | 75.3 / 69.3 | 67.9 / 58.2 | 86.0 / 85.1 | 91.9 / 114.4 |
| 10 | 304.5 / 233.6 | 289.6 / 227.8 | 281.5 / 219.2 | 292.2 / 194.8 | 280.6 / 269.2 | 434.9 / 440.3 |
| 25 | 424.4 / 321.7 | 492.0 / 341.1 | 429.1 / 284.0 | 369.0 / 260.3 | 533.2 / 355.4 | 688.9 / 563.7 |
| 50 | 556.0 / 349.8 | 527.9 / 345.6 | 452.8 / 295.0 | 422.6 / 278.4 | 560.3 / 394.1 | 889.8 / 702.9 |
MTP draft acceptance by work category
Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggregation.
| Work category | Overall acceptance | Pos 1 | Pos 2 | Pos 3 | Pos 4 | Pos 5 |
|---|
| JSON generation | 96.7% | 99.4% | 98.8% | 98.0% | 94.9% | 92.5% |
| Math | 79.9% | 96.2% | 88.5% | 79.9% | 70.9% | 64.2% |
| Code | 63.8% | 89.5% | 74.4% | 62.0% | 51.2% | 42.0% |
| Creative English | 62.1% | 88.7% | 72.9% | 59.8% | 48.5% | 40.7% |
Notes
All builds serve on the
tcclaviger/vllm:latest image, which has kernel tunes baked in.
TunableOp is untuned — GEMMs run on default heuristic-determined values.
Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.
Credits
-
Tess-4-27B by Migel Tissera (
migtissera/Tess-4-27B) — the model quantized here:
1@misc{tissera2026tess4,
2 title = {Tess-4-27B},
3 author = {Migel Tissera},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/migtissera/Tess-4-27B}}
6}
-
codeneedle (positional recall) originally by Alexander Ziskind, expanded test suite by tcclaviger (Rob Smith).
-
Tool-calling scenarios (incl. TC-61) run on tool-eval-bench by SeraphimSerapis (Tim Messerschmidt), scenario methodology adapted from ToolCall-15 by stevibe.
-
GSM8K, MMLU, and IFEval run via tool-eval-bench's built-in accuracy benchmarks at their defaults: GSM8K 8-shot CoT, MMLU 5-shot, IFEval zero-shot.
