You can deploy this model and start earning money today!

Wayfinder6/
bones-sage-rank32-experiment

Name: bones-sage-rank32-experiment
Availability: InStock

mlx

lora

fine-tuning

Wayfinder6/bones-sage-nova-lora

adapter

apache-2.0

Views

No views yet

Sage Rank-32 LoRA Experiment

A follow-up LoRA run on the same base model and corpus lineage as Wayfinder6/bones-sage-nova-lora (rank 8), this time at rank 32, testing whether a heavier adapter would carry voice more reliably. Documented honestly: it did not clearly win, and what it did find is worth having on record.

Two checkpoints included

adapters_v1_final/ — iter 3000 (last), val loss 2.157
adapters_iter1200_best/ — iter 1200, val loss 1.431 (the actual best point in the run by val loss; training oscillated the rest of the way, never beating this again — see val_loss_curve.json)

The actual finding: val loss and voice fidelity pointed opposite ways

Both checkpoints were run through the same 12-prompt voice-fidelity batch (adversarial "give me the honest truth" prompts, adapter output vs. the bare base model). Full transcripts in batch_results_v1_final.json and batch_results_iter1200_best.json.

iter 1200 (better val loss): 3 of 12 responses came back empty, several more truncated mid-sentence or generic. Almost no distinct voice.
iter 3000 / v1 (worse val loss): roughly half the responses showed real, distinct voice ("Alright, listen up, mate... you're the little fish trying to swim against the current"); the other half were still generic.

Lower val loss was not the better checkpoint here, by ear. This matches a pattern already seen on a different checkpoint line in this project (adapters_heavy_v3, selected on val loss alone, later found to carry ~0% voice on plain prompts) — val loss and voice fidelity are not the same measurement, and picking a checkpoint by loss alone is not reliable for this.

Bottom line, as of this push

Neither checkpoint is a confirmed improvement over the project's current generation source (a separate full fine-tune, not a LoRA, not published here). This repo exists so the experiment and its real result are on record, not to claim a win.

Wayfinder6/bones-sage-rank32-experiment

Sage Rank-32 LoRA Experiment

Two checkpoints included

The actual finding: val loss and voice fidelity pointed opposite ways

Bottom line, as of this push

Wayfinder6/
bones-sage-rank32-experiment