Haan v1 — frozen-backbone lineage
Haan is a Korean-first full-duplex speech model: a Qwen3-8B backbone reading
Moshi-style multi-stream frames (Mimi codec, 12.5 Hz, 8 codebooks per audio
stream) across four channels — native (inner) text, surface text, self audio,
user audio. This repository preserves the frozen-backbone lineage: the
pretrained spans are restored to their original weights and held, and only
the interfaces around them learn.
Both checkpoints are bf16, from_pretrained-loadable
(HaanForConditionalGeneration), 40 backbone layers.
Layout
| path | what it is |
|---|
init/ | the composed starting point, before any frozen-run step |
phase2/ | step 1000 of the frozen run, the lineage's selected checkpoint |
How init/ was composed
Weight surgery over the full-training lineage's phase-2 checkpoint
(RetentionLabs/haan-v1-full, phase2/) and the pristine assembly:
- layers 0–1: the trained checkpoint's first blocks
- layers 2–37: pristine Qwen3-8B, byte for byte — the span the run then froze
- layers 38–39: copies of the pristine last block
- depth decoder: pristine Moshi, except the input projections and role
embeddings the run had fitted
- audio tables, role FiLM, text heads, text embedding above the vocabulary
boundary: the trained checkpoint's
Training resumed at phase 2 with --freeze-layers 2:38 and the depth stack
held (drift on the frozen spans measured 0.0).
Phase-2 selection
The run went to step 4000, milestone-scored every 200 steps. Step 1000 is
where duplex control peaked before capacity drifted to the native channel:
| step | stream ppl (ko/en) | surface ppl (ko/en) |
|---|
| 800 | 154 / 234 | 52 / 469 |
| 1000 | 137 / 191 | 55 / 414 |
| 1200 | 120 / 183 | 57 / 675 |
| 4000 | 268 / 369 | 104 / 2415 |
Stream perplexity bottoms at ~1200 but the surface channel is already
collapsing there; step 1000 is the balance point. Free-running duplex had
not emerged by step 4000 (turn overlap ~0.99 throughout), so this checkpoint
is preserved as the lineage's best state, not as a working duplex model.
Sibling repository
RetentionLabs/haan-v1-full holds the full-training lineage this one was
composed from.
