Views
No views yet
Qwen/Qwen3-1.7B on 781 rows of
kelsbeans/digestive-coach-dataset
(nested subset of one shuffle; fixed family-disjoint valid set of 81).
Merged fp16 — loads with plain transformers, no peft. Adapter: kelsbeans/qwen3-1.7b-digestive-coach-n781-adapter.0b008de7d11cfab1b4c7a135dbb91b51cb8dc6b975863851837a7cc853b7052704e8ef7b6f7f8690| metric | base Qwen3-1.7B | this model |
|---|---|---|
| parse rate | 12.5% | 100.0% |
| schema validity | 2.5% | 100.0% |
| spec adherence | 0.0% | 20.0% |
| field accuracy | 26.7% | 39.7% |
| dimension | base | this model |
|---|---|---|
| integrated_voice | 2.88 | 4.78 |
| honest_uncertainty | 2.19 | 4.09 |
| no_manufactured_agreement | 2.62 | 4.47 |
python eval.py --model kelsbeans/qwen3-1.7b-digestive-coach-n781 --base Qwen/Qwen3-1.7B --eval-set eval_scenarios.jsonl --max-new-tokens 1600
(harness in the project repo at the pinned eval-code hash; raw per-example
transcripts ship as JSONL in runs/).