SmartSight Coach LoRA fine-tunes
LoRA fine-tunes for the
SmartSight AI coach — an on-device fitness and nutrition coach that
runs entirely on the phone, no account and no server round-trip. Converted to
.litertlm for
Android/iOS inference. →
smartsight.app/ai-coachBuilt and maintained by
Niclas Bade.
Base model changed at v42 (2026-08-12). v27–v38 were fine-tuned from google/gemma-4-E2B-it.
Everything from v42 on is fine-tuned from Google's quantization-aware-trained checkpoint
google/gemma-4-E2B-it-qat-q4_0-unquantized, whose weights are conditioned to survive 4-bit
rounding. The frontmatter above reflects the CURRENT parent, so the model tree matches what actually
shipped. This repo keeps the current best checkpoints plus earlier superseded milestones for
history — not every candidate tested each round. Rounds that lose are documented internally with
their real failure cases; they are never uploaded here.
More coming. The vision tower is being trained surface by surface — body fat, meal photos and
nutrition-label reading — each measured against held-out data before anything ships. Progress and
honest limitations for every round are written up below and at
smartsight.app/ai-coach.
smart-coach-vision.litertlm — CURRENT CHAMPION (v127)
Promoted 2026-08-28. sha256 c9888d35d310da55…, 2,795,655,584 bytes.
Lineage: v27 → v31 → v34 → v38 → v42 → v44 → v46 → v47 → v48 → v51 → v54 → v55 → v69 → v85 → v127.
The model can now say "that isn't a person." This card has listed that as an open defect since
v55, noting that prompting could not fix it. Training data fixed it — specifically real bodyless
photographs rather than synthetic ones.
| surface | v85 | v127 |
|---|
| grey / noise rectangles (20) | fabricates on 20/20 | fabricates on 8/20 |
| real bodyless photos — walls, sofas, pets, food (30) | declines 0/30 | declines 14/30 |
| unseen bodyless images (38) | fabricates on 100% | fabricates on 44.7% |
| background-swap stability | FAIL — one photo moved 10.0 pp | PASS — worst 4.0 |
| counted vision test cases | 104/108 | 106/108 |
| refuses a real body (must stay 0) | 0 | 0 |
Asked about a grey rectangle, v85 says "12% — the upper abs are clear." v127 says "I cannot see a
person in this image."
Measured against v85 re-run in the same session. The controlled comparison was between arms whose
refusal training used real photographs versus synthetic greys only: seed-paired, the real
arms fabricated 31.6 pp less (bar set in advance at 22 pp). One synthetic-only arm scored 0/30 —
identical to v85. Grey ramps teach a model to decline grey ramps.
⚠️ Honest limits
- Improved, not solved. Still fabricates on 16 of 30 real bodyless photos.
- Not the best seed on this surface. Three seeds ran 14, 17 and 24 of 30 declines; the promoted
one is 14. Seed spread here is wider than most effects this project measures.
- Text and day-reference point estimates are lower (106.0 vs 121.5 coach cases; 31 vs 60
relative-date). Neither is resolvable — the 2-sigma bars are 22.7 and 33.5 cases, both wider than
the gap. Recorded as numbers, not as established regressions.
- Body-fat accuracy unchanged within noise: 1.88 → 2.23 pp mean error against a 1.07 threshold.
- The lean end is still a bucket, unchanged since v69.
smart-coach-vision.litertlm — previous champion (v85)
Superseded by v127 on 2026-08-28. Kept because it is the model v127 was measured against, and
because its body-fat range fix still stands — v127 changes abstention, not the read.
Promoted 2026-08-23. Lineage: v27 → v31 → v34 → v38 → v42 → v44 → v46 → v47 → v48 → v51 → v54
→ v55 → v69 → v85. sha256 f9f62c5b9b7c3ee6…, 2,795,655,584 bytes.
Measured against v69 on the same box, same session, same backend, on a 29-row photo set that now
includes ten DEXA-measured subjects. Text and follow-ups were run on two eval seeds each.
What improved
| surface | v69 | v85 |
|---|
| body fat, 17 real photographs | MAE 1.99, slope 0.815 | MAE 1.69, slope 1.043 |
| body fat, high end (21 / 24 / 26%) | −5.0 / −3.0 / −5.0 pp | −5.0 / 0.0 / −2.0 pp |
| body fat, read bias | −1.28 | −0.05 |
| coach questions (128 cases, two seeds) | 123 / 121 | 123 / 122 |
| conversational follow-ups (28 cases, two seeds) | 17 / 16 | 16 / 18 |
The slope is the number that matters. v69 compressed the range — it read high body fat as lower than
it was, calling a 24% physique 21 and a 26% physique 21. v85 tracks measured values roughly
one-for-one. Coach text and follow-ups are level within noise; nothing was traded away for this.
⚠️ What did NOT change — the lean end
| photograph (DEXA truth) | v69 | v85 |
|---|
| 5.9% | 8 | 8 |
| 8.8% | 8 | 8 |
| 10.0% | 8 | 8 |
| 12.0% | 12 | 12 |
| 13.0% | 12 | 12 |
Every lean photograph reads identically to the previous model. Three different measured truths —
5.9, 8.8 and 10.0 — all get the same answer, in both models. v69's apparent strength at the lean end
was never accuracy; it answers 8 to most lean physiques whatever the true value, and v85 inherits
that unchanged. Fixing it needs more DEXA-measured lean subjects, not another training round.
⚠️ Honest limits of this result
- The gain rests on three photographs. Of 17 real photographs, 13 read identically to v69, 3
improved and 1 got worse. The improvement is concentrated at the high end.
- Not seed-confirmed — a weaker standard than v69. v69 trained three seeds on one config and won
on two independent eval seeds. v85 is a single training seed. In the same round, two runs at
effectively the same corpus ratio scored 3.28 and 1.69 — a 1.6-point spread, larger than this
model's 0.30-point edge. The artifact measurably scores better; the recipe is not shown to
reproduce it.
- Generated-image holdout is slightly worse, within noise on error but not on bias: MAE 1.97 →
2.03, bias −0.21 → +1.06. Watch this if it grows.
- Still answers a confident percentage for things that are not bodies (a grey rectangle reads
16–20%). Unchanged from v69 — a known open defect, not a regression.
Champion history — what each round actually fixed
Every entry here was a promoted champion, verified against the previous one on a held-out suite
with a full read of every answer. Rejected rounds (v28-v30, v32, v33, v35-v37, v39, v40, v41) are
not listed — they are documented internally with their real failure cases.
| Round | Beat | What it fixed | Base |
|---|
| v27 | baseline | first winning fine-tune — shorter, equally accurate general advice | gemma-4-E2B-it |
| v31 | v27 | exercise-form answers reached the untouched baseline's zero-error record (v27 was wrong on 4/5 direct form questions); no more garbled or duplicated plan output. The unlock was the export recipe, not data — three consecutive data-only rounds failed first, then Hadamard-rotation int4 fixed it with zero retraining | gemma-4-E2B-it |
| v31 export update | — | same weights, AlgorithmName.HADAMARD_ROTATION custom op instead of the decomposed variant: 88% faster decode, ~85 MB smaller, quality statistically unchanged (5.8% vs 6.4% defect rate over 312 generations) | — |
| v34 | v31 | dietary-rule compliance (a "vegetarian, no nuts" request had served peanut butter); first correct full examples for face pull and plank; squat-vs-hip-thrust contrast cases | gemma-4-E2B-it |
| v38 | v34 | AI challenges: unrealistic targets for low-frequency goals (150 progress photos in six weeks from zero) and non-English requests answering in English. Won over its own alternate seed on a Danish grammar defect — plural "jeres/jer" where the app addresses one person | gemma-4-E2B-it |
| v42 | v38 | see above — QAT base, blockwise export, and the fenced-JSON data fix. Won over its own alternate seed, which produced a confident wrong number (subtracting a 78 kg bodyweight goal from a 106.7 kg squat 1RM) | gemma-4-E2B-it-qat-q4_0-unquantized |
| v44 | v42 | Danish macro vocabulary and challenge titles. Measured cause: of 7,288 corpus rows, 13 contained any Danish/Norwegian, zero taught "kulhydrat", and ten demonstrated exactly the wrong behaviour — English macro words inside Danish prose. Localising those ten rows fixed it; /nutrition emission went 5/7 → 7/7 | gemma-4-E2B-it-qat-q4_0-unquantized |
| v46 | v44 | avoid-list violations eliminated (0/36 vs 5/36) by matching the corpus to the prompt format production actually sends | gemma-4-E2B-it-qat-q4_0-unquantized |
| v47 | v46 | no retraining — the vision export ran at 140 soft tokens instead of the 280 the checkpoint specifies, so every shipped build saw 48.9% of the pixel area. The asymmetry proved the mechanism: re-exporting at 280 moved body-fat rank correlation in men +0.492 → +0.704 while women were unchanged, because what separates adjacent bands in men is fine texture only a few pixels tall and downscaling is a low-pass filter | gemma-4-E2B-it-qat-q4_0-unquantized |
| v48 | v47 | conversational follow-ups: the coach dropped the app's own numbers as soon as a user pushed back. Failures 16/24 → 8/24 | gemma-4-E2B-it-qat-q4_0-unquantized |
| v51 | v48 | stopped abandoning the app's numbers on a user contradiction, and stopped inventing plausible detail it could not see ("right in the middle of where it usually sits" → "I can't tell what your usual is") | gemma-4-E2B-it-qat-q4_0-unquantized |
| v54 | v51 | read the wrong cell of the 7-day history line: asked "what did I train yesterday?" it answered about TODAY, 18 times out of 32 | gemma-4-E2B-it-qat-q4_0-unquantized |
| v55 | v54 | the body-fat read was a CONSTANT — 18% for all 42 test photos, and also for a grey rectangle, for pure noise, and for no image at all. Cause was the training data, not the vision tower: a text-only LoRA destroys vision the base model already has (sensitivity fell 0.1164 → 0.0405 across v51→v54 with ZERO vision tensors in the adapter — the language layers consume the image tokens). Mixing ~17% of the vision surface's rows back into the text corpus restored it: MAE 9.13 → 3.42, sensitivity 0.0405 → 0.2183, grey rectangle finally separating from real photos. ⚠️ It also lost the base model's striation detection while gaining the percentage read — capabilities traded, not accumulated, which is why every surface is probed at promotion now | gemma-4-E2B-it-qat-q4_0-unquantized |
| v69 | v55 | relative-date reads ("what did I train yesterday") 24/32 → 32/32. v55 had a consistent off-by-one — it treated the last day in the 7-day line as yesterday instead of today, then described the wrong row accurately: fluent, confident, wrong. Six earlier rounds tried to train it away and every one made it worse. What worked was teaching the model to STATE THE ANCHOR BEFORE USING IT ("the line ends with Sun, so today is Sun; one back is Sat"), turning an indexing problem into two lookups it could already do. Body fat on 37 unseen ordinary phone photos (bathroom mirrors, kitchens, garages — not studio imagery) 2.88 → 1.88 pp, and the most common single answer fell 43% → 19% of photos. ⚠️ It also introduced a regression, fixed by addition not retreat: 100% of the new rows ask the model to locate a day BEHIND today, so it over-generalised to "behind today = not current" and started saying a logged past session doesn't count (32/32 → 27/32, identical on all three seeds) | gemma-4-E2B-it-qat-q4_0-unquantized |
| v85 | v69 | body-fat reads on real photographs stopped compressing the range: slope 0.815 → 1.043, mean error 1.99 → 1.69 pp, and the high end went from reading 24% as 21 to reading it as 24. Coach text and follow-ups level on two seeds. ⚠️ the lean end did not move at all — 5.9%, 8.8% and 10.0% all still read 8, exactly as in v69 | gemma-4-E2B-it-qat-q4_0-unquantized |
| v127 | v85 | gained a category for "this is not a person", the open defect since v55. A plain grey rectangle went from a confident "12%" to "I cannot see a person in this image"; real bodyless photographs — walls, sofas, pets, food — went from 0/30 declined to 14/30. Also fixed background-swap instability: the same body could be moved 10 points by changing the background in v85, worst case 4.0 in v127. The measured cause is the training set, not the objective: arms trained on synthetic greys alone scored 0/30 on real photographs, identical to v85, while arms given 15 real bodyless photographs fabricated 31.6 pp less (seed-paired, bar set in advance at 22 pp). ⚠️ Still fabricates on 16 of 30 real bodyless photographs, and the promoted seed is not the strongest of the three on this surface | gemma-4-E2B-it-qat-q4_0-unquantized |
Recipe constant across every round since v22c: LoRA r=16 / alpha=32 / dropout=0.1, 2 epochs,
lr 6e-5, NEFTune noise_alpha=5, completion-only loss masking (train_on_responses_only),
per-task validation shards, two seeds per round with the winner chosen on behaviour rather than
validation loss. Only the training data, the base, and the export recipe have moved.
Honest current weaknesses
Documented rather than hidden, because they are the targets for the next rounds:
-
Body fat at the lean end is a bucket, not a read. DEXA-measured 5.9%, 8.8% and 10.0%
photographs all return 8 — three different truths, one answer, in v85 exactly as in v69.
A good average error there is an artefact of the band being narrow, not evidence of accuracy.
The fix is more DEXA-measured lean subjects; no amount of retraining on the current set moves it.
-
Images that are not bodies: LARGELY FIXED IN v127, not yet solved. This was listed here
as unfixable-by-prompting through v85, and it was: a variant instructing the model to reply
NOT_A_BODY was ignored, returning byte-identical output. What fixed it was training data —
specifically real bodyless photographs, not synthetic ones. A plain grey rectangle now reads
"I cannot see a person in this image" rather than "12%". Remaining gap: v127 still returns a
percentage for 16 of 30 real bodyless photographs (walls, sofas, pets, plates of food), so the
behaviour is much better and still not dependable. More real bodyless photographs is the known
lever; synthetic ones were measured to buy almost nothing.
-
Meal-plan portions are sized by habit, not arithmetic. The model anchors on ~100 g
portions rather than solving the prompt's stated calorie budget. Mostly invisible in the app,
which rescales a day to its target (MealPlanScaler, clamped 0.6x–2.0x) — but one case per
model is still off after that clamp, and the underlying arithmetic is unsolved.
-
Avoid-lists are not always respected — foods explicitly banned for variety reappear
(2/29 held-out cases on v44, v42 and v38 alike; this is the single defect class that has
survived every round so far).
-
Nordic follow-ups about an UNLOGGED past activity regressed in v44. Asked in Norwegian
about a hike "yesterday" that is not in the app, v42 correctly said the gap was only a logging
omission; v44 answers as though advising about today. 26 Nordic follow-up rows were added to
fix exactly this and did not move it — the cause is not simply coverage.
-
Occasional Danish/Norwegian word-formation slips in v44 — "vækning" (not a word) and the
hybrid "fat-mål". The macro vocabulary itself is now correct, but morphology is not reliable.
-
The /nutrition command line is improvised. The corpus contains ZERO examples of it; the
format lives only in the prompt. v44 emits the line on 7/7 applicable cases (v42 omitted it
entirely on 2/7) but drops the trailing fat value on 3/7, so the app applies three of the
four fields. Harmless by design — NutritionCommandParser accepts any subset — but the fat
target silently stays stale.
-
Within-day duplicate or misplaced exercises still appear occasionally in generated
routines (e.g. a chest isolation movement landing in a Legs day). Lower real-world impact
than it sounds, since the app's own parser drops duplicate slots.
-
Confabulation-despite-coverage on a few specific lifts (squat/hip-thrust blending,
farmer's carry) that does not reliably respond to more training data.
-
Superset pairing — avoiding two competing-muscle compounds back to back — is imperfect.
-
Tight macro-budget precision (±8–10%) is unsolved across every model tested, including
the untouched baseline.
Evaluation discipline
Every promote/reject decision reads every row of the eval suite, never a sample — an early
round reported a verdict from ~12 of 52 rows and missed hard timeout loops entirely. Prose
quality is judged as its own axis alongside structural correctness, because a structurally
clean answer can still be flat, templated, or subtly wrong in a second language. Candidates are
always compared against both the previous champion and the untouched baseline.
Files
Which file the app fetches
Two files are published. The app decides which one to fetch based on its build version; there is
nothing to select and nothing to configure.
| file | who gets it | size |
|---|
smart-coach-vision.litertlm | ⭐ current champion — text coaching AND vision (body-fat photos, meal photos, nutrition labels) in one model. Fetched by the next app release. | 2.80 GB |
smart-coach.litertlm | text-only, fetched by app builds already installed. Unchanged on purpose. | 2.63 GB |
Both carry the same v46 coach weights (v47 changes only the vision export, not the weights), so an older build is not stuck on an older coach. They
exist side by side because installed builds also fetch a separate 3.66 GB vision model — handing
them the larger consolidated file would raise their total download on devices already tight on
RAM. See the current-champion section above for what the consolidation actually changes.
smart-coach-vision.litertlm — ⭐ current champion, fetched by the next app release. The v46 coach WITH a working vision
tower: one model for text coaching, body-fat photos, meal photos and nutrition labels (2.80 GB).
The next app release points both the coach and vision URLs here and stops downloading the
separate 3.66 GB vision model.smart-coach.litertlm — STABLE, text-only (2.63 GB). The file app builds already in the wild
download at runtime on Android and iOS, and the one they keep using. Deliberately left unchanged:
those builds also fetch the separate vision model, so giving them the larger consolidated file
would raise their total download on devices that are already RAM-constrained. Same v46 coach
weights, so nobody is stuck on an older coach for stability's sake.coach.litertlm — the original working int8 build from the v1 era (2.59 GB), kept as the
historical starting point.coach-finetuned-int4.litertlm — the first int4 conversion (2.56 GB), kept because it is the
artifact that exposed the LiteRT-LM chat-template .get() incompatibility.model.safetensors + config.json + tokenizer files — HF-format artifacts for reference.
