A 126M-parameter legal/financial Q&A model, supervised
fine-tuned from nitin1pawar/slm125m-base — a base model
that was itself pretrained from scratch (corpus, tokenizer and weights) for $31.76.
Trained in RAFT style: each example pairs a question with retrieved passages, and a
quarter of the training set contains passages that do not answer the question, so the
model learns to say so instead of inventing an answer.
1messages =[2{"role":"system","content":"You are a legal and financial assistant. ..."},3{"role":"user","content":"Passage 1:\n...\n\nQuestion: ..."},4]5prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
When the passages do not contain the answer the model replies:
The passage does not provide enough information to answer this question.
Training data
10,000 synthetic Q&A pairs generated with gpt-5-mini from the model's own
pretraining corpus (US case law + SEC filings + educational web text), then graded by an
LLM judge on groundedness, correctness, answerability and form. Judge pass rate
88.4%. Questions were deduplicated by exact hash and by 5-gram
Jaccard overlap.
Passages were drawn from the deduplicated and decontaminated corpus, so CaseHOLD and
LexGLUE case_hold remain genuinely held out.
slice
share
closed_book
1,584
instruction
787
negative
2,511
positive
5,118
Training
148 steps over 1 epochs
(9.7M tokens, 62s on 1×H100),
full-parameter (no LoRA), AdamW, peak LR 8e-05 cosine to 0, bf16.
Loss is computed on answer tokens only — question and context tokens are masked out.
125M parameters. It produces fluent but frequently incorrect legal and financial text, its
knowledge is bounded by a 2.07B-token corpus, and the training answers were written by
another language model rather than by lawyers. Context is capped at 1,024 tokens.
Not legal or financial advice.
