A 105M-parameter bilingual causal language model with dual-mode reasoning, built on the Codon stack.
MotifA1 is a compact, CPU-friendly causal language model trained for bilingual (Chinese / English) instruction following. It supports both an explicit thinking mode (chain-of-thought wrapped in [cot_start] ... [cot_end]) and a direct non-thinking mode, switchable at inference time.
Model Summary
Field
Value
Parameters
105.41 M
Vocabulary
8,192 (BPE, packed)
Architecture
Causal Transformer (decoder-only)
Position Encoding
RoPE, base = 500,000
Training Context
4,096 tokens
Languages
中文 / English
Modes
Thinking / Non-thinking
Runtime
CUDA / CPU
Precision
fp32 / bf16
License
See repository
Why RoPE base 500k
A RoPE base of 500k flattens the rotary frequency spectrum, which gives MotifA1 headroom to extend its context window beyond the 4,096 it was trained on via interpolation-style scaling, without retraining the position basis from scratch.
Installation
pip install codon-model==0.0.5
Required artifacts:
motifa1_sft.safetensors — model weights
motif.vocab — packed tokenizer (vocab + chat template + config in one zip)
Quickstart
Load the model
python
1from codon.motif import MotifA1
23model = MotifA1().load_pretrained('motifa1_sft.safetensors').to('cuda')4print(model.count_params(human_readable=True))# -> 105.41 M
CPU users: replace 'cuda' with 'cpu'. Inference works out of the box, just slower.
chunk.is_cot — whether the current span is inside a chain-of-thought block
chunk.cot_ended — fires once when the model exits thinking mode and begins the user-facing answer
chunk.content — the decoded text fragment
This lets you render reasoning in a separate visual channel (e.g. dim blue) and the final answer in normal style.
Modes
MotifA1 follows a chat template with explicit reasoning markers.
Thinking mode — the model first generates content between [cot_start] and [cot_end], then produces the final answer. Recommended for math, multi-step reasoning, code planning.
Non-thinking mode — the model emits an empty [cot_start][cot_end] block and answers directly. Recommended for chit-chat, translation, short-form generation, and latency-sensitive applications.
The chat helper exposes mode switching; consult the Codon docs for the parameter form your version exposes.
Training
Stage 1 — Pretraining at 4,096-token context, bilingual corpus.
Stage 2 — SFT on a curated mix of single- and multi-turn dialogues, with thinking and non-thinking samples blended.
Schedule Linear warmup → cosine annealing to 10% of peak LR.
Precision bf16 autocast.
Known Limitations
Long-range attention is weak. Even with RoPE base 500k allowing window extension, retrieval and reasoning over long spans (>2k effective tokens) degrade noticeably. Treat MotifA1 as a short- to mid-context model in practice.
Scale-bound knowledge. At ~105 M parameters, factual recall is limited. Pair with retrieval for knowledge-heavy tasks.
Vocabulary is compact. With 8,192 BPE tokens, rare scripts, niche jargon, and long URLs may be tokenized inefficiently.
Hallucination. Like all LMs of this scale, MotifA1 can produce confident but incorrect answers. Verify safety-critical outputs.
Intended Use
Personal assistants, on-device chat, edge deployment
Education and research on small LMs, dual-mode reasoning, and RoPE scaling
A base model for further fine-tuning at modest compute budgets
Not intended for high-stakes decisions (medical, legal, financial) or as a sole knowledge source.
