Views
No views yet
live-run/RESULTS.md), trained on the
Amber Framework knowledge pack dataset.mlx-community/SmolLM-135M-Instruct-4bit (135M params, 4-bit, MLX)mlx_lm (rank 8, 8 layers, lr 1e-4, batch 2, 2000 iterations,
62.8 s wall on an M1 Max 32 GB) — full config in adapter_config.jsonadapters.safetensors:
8f88465b98309f4001d85b53eae959e4f22dae3155fc1e43a60f5b90deba6c76 — matches the hash
recorded in the repo's RESULTS.md, so you can verify this is the artifact the numbers
were measured on. No --seed was passed during training (mlx-lm default), so bit-exact
retraining is not guaranteed; the hash identifies this exact artifact.| Eval set | Metric | Base | This adapter |
|---|---|---|---|
| memorization (trained-fact probe) | recall, normalized substring (20) | 1/20 | 11/20 |
| multiple choice (30) | 2/30 | 6/30 | |
| heldout_sources (control) | recall (20) | 2/20 | 0/20 |
| multiple choice (30) | 7/30 | 7/30 |
reports/base.eval.json and
reports/adapter.eval.json (question, gold answer, full
completion, verdict for every one of the 100 items).1pip install "mlx-lm" "transformers<5"
2python -m mlx_lm generate --model mlx-community/SmolLM-135M-Instruct-4bit \
3 --adapter-path . \
4 --prompt "In Amber, what does the pipeline block in config/routes.cr do?" \
5 --temp 0