You can deploy this model and start earning money today!

lewisdog/
lfm2.5-230m-cogs-ingest-GGUF

Name: lfm2.5-230m-cogs-ingest-GGUF
Availability: InStock

Views

No views yet

lfm2.5-230m-cogs-ingest

A 230M ingest student for the Cogitarium wiki pipeline: distils one raw captured document into structured JSON (extract / suggest_links / contradiction / page_update).

Serving pins (important)

temperature 0, repeat_penalty 1.0 (NO penalty). A repeat penalty produces schema-valid but empty/degenerate JSON on extraction at this size — the extract task copies input tokens and a penalty starves it.
Cap max_new_tokens and/or constrain with a JSON grammar: the extract task can run away under pure greedy (well-formed but unterminated JSON). This is a decoding artifact, not a format defect — do NOT reach for a repeat penalty to fix it.

Sanity eval (5 samples / 4 task types, temp 0, rep 1.0)

quant	strict JSON / keys	note
F16 / Q8_0	4/5	`suggest_links`, `page_update`, `contradiction` perfect; `extract` may not terminate under greedy
Q4_K_M	3/5	quant cliff — `page_update` loses the `section_md` key

Recommended quant: Q8_0 (233 MB, 723 tok/s on GB10). Q4_K_M only where size dominates and page_update is not used. Training matched the Qwen3-1.7B token accuracy (0.752 vs 0.756) at 7x fewer params.

This repo

GGUF quant ladder for llama.cpp (F16 / Q8_0 / Q4_K_M). See the quant table above for the recommended pick and per-quant behavior. Run with --temp 0 and the serving pins noted above.

Base model & license

Fine-tuned from LiquidAI/LFM2.5-230M. Use is governed by the LFM Open License v1.0 (lfm1.0) — see the LICENSE in the base repo. This derivative complies with and inherits those terms; attribution to LiquidAI is retained above.

Provenance

LoRA SFT (TRL) on the Cogitarium distillation datasets, DGX Spark (GB10). Full methodology, loss curves, eval harnesses and per-quant results: see the project RESULTS.md. This is the "fast/small tier" of the Cogitarium model picker; the Qwen3-1.7B students remain the quality tier.

lewisdog/lfm2.5-230m-cogs-ingest-GGUF