la-math-v9 — Qwen3-4B fine-tuned for linear algebra
A LoRA fine-tune of Qwen3-4B specialized in linear algebra: determinants, inverses,
eigenvalues, systems of equations, rank/spaces, factorizations, matrix operations,
and linear algebra theory (definitions, theorems, examples).
Best LA specialist so far in the la-math series (v5 → fix2 → v9).
Evaluation
416-pair held-out linear algebra set, SymPy-graded with an identical harness across all models
(284 items computationally gradable; the rest are theory/definitional).
| Model | Accuracy (regraded) |
|---|
| la-math-v9 | 75.0% |
| la-math-v5-fix2 | 73.5% |
| la-math-v5 | 72.8% |
| la-math-v5-fix1 | 71.4% |
| Qwen3-4B base | 67.8% |
Category breakdown (v9): matrix_ops 100%, orthogonality 100%, lin_trans 88%, inverses 85%,
determinants 84%, rank_spaces 67%, systems 65%, least_squares 60%, eigen 58%.
Training
- Base: Qwen3-4B (MLX, 4-bit quantized)
- Data:
la_data_v8 — 9,703 training pairs (curated from v5 + gap-fill items + PDF-extracted
theorems/definitions/examples from 8 standard linear algebra textbooks), audited self-consistent,
answers capped at 1,024 tokens
- Method: LoRA, 16 layers, rank from mlx_lm defaults (0.182% trainable params)
- Fresh run: 9,703 iters (1 epoch), batch size 1, LR 1e-5, max seq 1,024, grad-checkpoint, mask-prompt
- Fused from the iter-7000 checkpoint (val-loss floor 0.047) — late-epoch overfit avoided
- Fused + requantized to 4-bit (group size 64)
Usage
MLX (mlx-lm):
1from mlx_lm import load, generate
2model, tokenizer = load("j01001100/la-math-v9")
3response = generate(model, tokenizer, prompt="Find the inverse of [[2,1],[5,3]]", max_tokens=768)
LM Studio: local model, 2.28 GB, runs on Apple Silicon. For a tutor that verifies its own
answers, wrap with inference-time SymPy verification (generate → verify → retry).
License & attribution
Apache-2.0. Base model: Qwen/Qwen3-4B (Apache-2.0). Fine-tune by jason (j01001100).
