A ~125.8M parameter Llama-architecture base language model trained from scratch
on a legal/financial corpus (US case law + SEC filings + a fineweb-edu slice),
with a custom 16,384-token byte-level BPE tokenizer.
Params: ~125.8M (12L / 768d /
12h, context 1024)
Vocab: 16384 (byte-level BPE)
Training data: ~2.0B tokens (~40% case law / ~40% SEC / ~20% web), deduplicated
and decontaminated against CaseHOLD/LexGLUE.
Objective: causal language modeling.
This is a base model (no instruction tuning). It is a research artifact.
