A 46.7M-parameter English language model trained from scratch — architecture, tokenizer,
training code and behaviour data are all original. No existing weights were fine-tuned,
distilled or merged.
It is a research artifact and a small, honest chat model: it knows who it is, follows a
chat format, and — most usefully — reads a supplied context and answers from it rather
than from memory. At 47M parameters its own world knowledge is thin, so retrieval is the
intended way to use it.
Runs on CPU. Commands inside the chat: /ctx <text> supplies a retrieval context,
/reset clears history, /q quits.
Prompt format
The model was trained with four control tokens and no system prompt — the persona lives
in the weights.
<|user|>
{question}
<|sane|>
{answer}<|end|>
With retrieval, the context goes before the question:
<|context|>
Search results: The Ridgeloom V4 weighs 34 kg and sells for 690 euros.
<|user|>
How much does the Ridgeloom V4 weigh?
<|sane|>
Sampling defaults that work well: temperature 0.7, top_k 50, top_p 0.92,
repetition_penalty 1.12. For factual questions over a context, drop the temperature to
0.3 or use greedy decoding.
What it does and does not do
Works:
knows what it is and does not claim to be human;
reads <|context|> and copies the right value out of it, including noisy search-result
text with distractors, and says so when the answer is not in the passage;
holds a short multi-turn conversation within the 1024-token window;
simple string and format tasks.
Does not work:
facts without a context are unreliable. This is a capacity wall, not a bug to be
fine-tuned away — 47M parameters trained on 4.4 B tokens cannot store much;
arithmetic beyond the trivial;
long free-form generation drifts;
anything outside English.
Evaluation
No standardised benchmark numbers were recorded for this model — it was evaluated with a
held-out behavioural probe only. Its successor, Sane-118M,
was measured and scores roughly at GPT-2-small level; Sane-47M sits below it.
If you benchmark it, please open a discussion — the numbers are welcome.
Training data
Pre-training and instruction data (see ATTRIBUTION.md in this repository):
The behaviour pool — identity, retrieval-reading, refusal-of-roleplay — is procedurally
generated by the author and contains no third-party text.
ODC-By requires attribution; ATTRIBUTION.md is that attribution and must travel with the
model if you redistribute it.
Bias, risks and limitations
The model inherits whatever biases exist in web text filtered for educational quality. It
has no safety tuning, no refusal training for harmful requests, and no guardrails: treat its
output as untrusted text. It has no memory between sessions and no access to the internet —
anything time-sensitive must be supplied in <|context|>.
License
Released under the Sane Community License 1.0 (see LICENSE). In short: free to use,
modify and redistribute, including commercially, until the product built on it earns more
than 1 000 USD per month — beyond that, contact the licensor. Attribution notice
required: "Powered by Sane-47M."
Citation
bibtex
1@misc{sane47m,
2 title = {Sane-47M: a small language model trained from scratch},
3 author = {S.C.},
4 year = {2026},
5 url = {https://huggingface.co/sekund0chka/sane-47m}
6}
