A DoRA finetune of Qwen3-1.7B on a personal corpus, merged into the base
weights and exported to GGUF at full precision. Private.
F16 is the archival artifact, not the fast one. It is unquantized, so it
is memory-bandwidth bound exactly like bf16 — measured 16 tok/s on an RTX
4050 where the bf16 transformers version did ~57. Its purpose is to be the
lossless source for llama-quantize, and a checkpoint that does not decay.
What it was trained on
22.64M tokens, 25,700 records:
source
tokens
share
what it is
rosetta
10.18M
45.0%
Rosetta Code — 1,182 tasks across 991 languages
soup
9.07M
40.0%
personal Claude sessions, tool calls preserved
graphsoup
3.39M
15.0%
the same corpus rearranged by graph traversal
soup is six months of the author's own conversations with Claude across
two dialects (claude.ai export + Claude Code JSONL), with the agentic layer
— tool calls and results — deliberately kept rather than stripped. It is
roughly 59% tool traffic by bytes.
graphsoup is that corpus walked through a symbolic link graph (JANUS),
producing sequences of real chunks in orders that never occurred. Real
material, novel adjacency, no model authorship.
Measured against the untuned base on a hand-rolled bench:
base
tuned
posture (reaches for a shell)
0/3
3/3
accuracy
4/8
4/8
wrong answers
10
3
deferred to a tool
0
6
The headline is the third row. The tuned model answers wrong about a third
as often, and spends the difference reaching for a tool rather than guessing.
Base: "I cannot run a shape audit without access." Tuned: emits a Bash call.
The 45% Rosetta share is visible and probably too high. Asked "what is
17 times 23" it writes a C# program rather than answering 391. Asked to run
a shape audit it writes Go. If you retrain, turn Rosetta down first.
Known defects
No record separator. Training records were concatenated with nothing
marking the boundaries, so block edges taught transitions between unrelated
documents as continuations. Conversations got partial cover from
<|im_end|>; nothing else did.
Packing slices mid-record every 2048 tokens. 98.8% of records fit
whole, so this hits the long tail — which is where the longest sessions
live, including one at 187k tokens.
weight_decay = 0.0. Never set.
Regurgitation
Probed with three corpus-specific prefixes. No verbatim training data
returned — only format and register ([THINKING] markers, connective-tissue
phrases). Three probes is not an audit. Kept private accordingly.
Provenance
Trained 2026-08-21 on a rented RTX 5090 (vast.ai), ~96 min, roughly $1.30.
Merged and converted on the laptop. Full trail in the source repo:
runs/2026-08-21/ and KNOWN-DEFECTS.md.
