LiquidAI LFM2.5-1.2B-Base after Continued Pre-Training on a fictional domain corpus (Orbital Mining Corporation technical docs + Mars Express telemetry).

Overview

Stage: Continued Pre-Training (full fine-tune, CLM)
Lineage: LFM2.5-1.2B-Base → CPT (this model)
Method: Continued Pre-Training (next-token, full fine-tune). Raw telemetry rewritten to grounded explanatory prose [wrap]; 7% general replay to limit forgetting [replay].
Domain: fictional — Orbital Mining Corporation (OMC) technical docs + Mars Express telemetry.

Training


Corpus	~12.2M tokens (84% OMC / 9% telemetry prose / 7% FineWeb-Edu replay)
Objective	CLM (next-token), full fine-tune, bf16
LR / schedule	1e-4 cosine, warmup 0.03
Epochs / seq	2 epochs, seq 4096, eff_batch 32

Evaluation

Metric	Value	Note
domain eval loss	1.531	monotone decreasing, no overfit
general (FineWeb) loss	1.494	improves — replay works
telemetry loss	0.443	—

Eval is a held-out, deterministic verifiable harness (synthetic tasks); baseline = the pre-SFT ParamΔ model. See noval-corp/scripts/eval_agentic.py.

Intended use & limitations

Domain-knowledge backbone. Not instruction-tuned — does not follow chat instructions on its own (see the -instruct-paramdelta / -agentic siblings).

Limitations:

Fictional domain — the model 'knows' an invented world (OMC / Mars Express).
No chat template / instruction following (it is a base-style CPT checkpoint).
Sparse one-off entities (component IDs, mission names) are not reliably learned.

Innovations tested

Log → grounded prose — raw telemetry deterministically templated to NL with verbatim numbers (zero teacher, zero hallucination) [wrap].

Citations

WRAP (rephrase-to-prose) — Maini et al., 2024. arXiv:2401.16380
CPT replay (anti-forgetting) — Ibrahim et al., 2024. arXiv:2403.08763

Card generated by noval-corp/scripts/gen_model_cards.py (standardized across the noval-corp model family).

atenareply/lfm2.5-1.2b-noval

LFM2.5-1.2B Noval — Domain CPT