Qwen3.5-0.8B (text) fine-tuned for long meeting-transcript summarization and insight
extraction in Traditional Chinese and English, for
VoxSumDroid. Trained with quantization-aware
training so it survives int4 weights on device.
Why this base model
Nine candidates were benchmarked on the three constraints that actually bind on a phone
(prefill speed, Chinese token efficiency, RSS). Qwen3.5-0.8B won all three:
Qwen3.5-0.8B
best rival
prefill @16k, 8 CPU threads
491 tok/s
457 (granite-4.0-h-1b)
zh-TW tokenizer
1.67 chars/token
0.98 (Falcon-H1), 0.69 (Granite)
3-h zh meeting, prefill (ARM-derated)
~4 min
9–12 min
KV cache @32k, q4
197 MB
—
Its hybrid architecture (only 6 of 24 layers hold a KV cache; the other 18 are
Gated-DeltaNet linear attention with constant state) is what keeps memory and prefill low
at long context. Rejected: Falcon-H1-1.5B (1.84 GB @64k, slowest prefill),
Granite-4.0-h-1b (0.69 chars/token on zh), Granite-3.1-1b-a400m MoE (−54% prefill decay by
16k), T5Gemma-2 enc-dec (no runtime support, 32k ceiling, degenerate at 270M),
Gemma-3-1B (32k native), Qwen3.5-2B (1.68 GB, over budget).
The app's per-task prompts (summarize / title / action items / topics / open questions /
risks, bullet-executive-narrative styles, en+zh with cross-lingual clause) are also
supported — see the training repo.
Training
Distillation from Qwen3.5-9B over 3,891 meetings (QMSum, MeetingBank, DialogSum,
VCSum→OpenCC s2twp).
Faithfulness filtering — every one of 39,731 targets judged for factual support and
polarity inversion; 6,414 dropped (16.1% scored faith≤3; 300 were outright
inversions). This was the single largest quality lever.
Rejection-sampling fine-tune on the model's own long-document outputs: 389 samples
the judge scored faith 4.73 with zero inversions, upweighted 6×. Their rejected
counterparts averaged faith 1.85 and contained 121 inversions.
Known limitations — please surface these honestly in UI
Faithfulness degrades past ~12k tokens. Measured on the shipped checkpoint:
faith 4.05 / 1.8% inversion at 4–12k tokens, versus 3.36 / 15.3% inversion above 12k,
where 46% of outputs score faith≤2.
Chunking at ~10–12k tokens with a hierarchical reduce keeps the model in its reliable
regime; the map/reduce prompts are in the training mix for exactly this.
Owner attribution can slip on long transcripts (assigning an action to the wrong
speaker). Action items are a draft to correct, never an audit trail.
Cross-lingual (en→zh-TW) faithfulness is the weakest axis; language routing is
reliable (0.98) but details drift.
