Built on the Qwen3.6-27B (64-layer, original) base, extended to 80 layers (~34B parameters) and distilled from Fable-5 Agentic Traces. Fine-tuned via QLoRA (4-bit NF4, rank=64) on 4,665 Fable-5 Chain-of-Thought trajectories, then merged with MTP speculative decoding heads.

Model Card

Property	Value
Base model	Qwen3.6-27B (64-layer, original) → Qwen3.6-34B-80L-BF16-v2
Architecture	Qwen3_5ForConditionalGeneration
Parameters	~34B (80 layers, hidden=5120)
Attention heads	24 (4 KV Grouped)
Context length	262,144 (256K)
Vocabulary size	248,320
Attention mode	Hybrid Linear/Full (full every 4 layers)
MTP speculative decoding	✅ (15 mtp weights)
Precision	BF16

Distillation Data

Item	Value
Data source	Glint-Research/fable-5-traces
Samples	4,665 entries
Content	Fable-5 Agentic CoT trajectories (context + completion)
Training config	QLoRA rank=64, α=64, 2 epochs, lr=2e⁻⁴, max_len=4096

Training Details

Quantization: bitsandbytes 4-bit NF4 + double quantization (QQ)
LoRA targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, and MoE projection layers
Optimizer: paged_adamw_8bit, cosine schedule, 5% warmup
Gradient checkpointing: enabled (use_reentrant=False)
Effective batch size: 4 × 4 = 16

Key Features

Fable-5 deep reasoning — Inherits Fable-5's CoT chain-of-thought reasoning and tool-use patterns.
80-layer efficient architecture — Stronger expressiveness at 34B parameters with 80 layers.
Hybrid attention — Alternating Linear/Full design for higher inference efficiency on long sequences.
MTP speculative decoding — Native MTP weights; vLLM deployment achieves ~78% acceptance rate, 2× throughput.
BF16 full precision — MergeLoRA merged without quantization loss, ready for deployment and re-fine-tuning.

Quick Start

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hiebo/Qwen3.6-34B-80L-Fable-5-Heretic"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

prompt = "What is quantum computing?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

vLLM Launch Configuration

bash

vllm serve hiebo/Qwen3.6-34B-80L-Fable-5-Heretic \
    --dtype bfloat16 \
    --max-model-len 262144 \
    --enforce-eager \
    --gpu-memory-utilization 0.95 \
    --swap-space 16 \
    --disable-log-stats

Supported Parameters

Parameter	Default	Description
`--model`	(required)	Model ID or local path
`--dtype`	`auto`	Data type: `bfloat16`, `float16`, `float32`, `auto`
`--max-model-len`	`4096`	Maximum context length (supports 256K)
`--max-num-seqs`	`256`	Maximum number of concurrent sequences
`--gpu-memory-utilization`	`0.90`	Fraction of GPU memory to use (0–1)
`--swap-space`	`4`	CPU swap space in GB
`--tensor-parallel-size`	`1`	Tensor parallelism degree (multi-GPU)
`--pipeline-parallel-size`	`1`	Pipeline parallelism degree
`--enforce-eager`	`False`	Force PyTorch eager mode (recommended for BF16 models)
`--disable-log-stats`	`False`	Disable statistics logging
`--served-model-name`	`None`	Override served model name
`--port`	`8000`	API server port
`--host`	`0.0.0.0`	API server host
`--tokenizer-mode`	`auto`	Tokenizer mode: `auto`, `slow`, `mistral`
`--quantization`	`None`	Quantization: `fp8`, `awq`, `gptq`, `squeezellm`
`--load-format`	`auto`	Load format: `auto`, `sharded_state`, `safetensors`, `npcache`
`--kv-cache-dtype`	`auto`	KV cache data type: `auto`, `float16`, `bfloat16`
`--enable-chunked-prefill`	`False`	Enable chunked prefill for long sequences
`--max-num-batched-tokens`	`None`	Max batched tokens per step
`--seed`	`0`	Random seed for reproducibility
`--distributed-executor-backend`	`ray`	Distributed backend: `ray`, `mp`, `mp_rank`
`--trust-tokenizers`	`False`	Trust remote tokenizer files
`--revision`	`main`	Model revision (branch/tag/commit)

nwzjk/Qwen3.6-34B-80L-Fable-5-Heretic

Qwen3.6-34B-80L-Fable-5-Heretic