Nano.Deep.Reasoner.11m-HyperMini
An approximately 11,094,003-parameter decoder-only
adaptive recurrent reasoning language model.
Architecture
- Parameters: 11,094,003
- Context: 1096
- Vocabulary: 16,000
- Hidden size: 240
- Base Transformer blocks: 6
- Attention heads: 8
- Head dimension: 30
- Intermediate size: 1072
- Shared recurrent reasoning block
- Learned latent reasoning memory: 8 tokens
- Adaptive reasoning depth: 2-16
- Gated recurrent memory updates
- Adaptive halting controller
- Verification head
- Revision head
- RoPE
- Tied input/output embeddings
- Padding-aware causal attention
- Explicit padded-state suppression
Reasoning tokens
Training
Dataset:
Plans11/Organized_PreTrain_1k_Context
Each session contains up to 200,000 NEW examples.
Examples are protected by SHA-256 hashes.
Session reservations are committed before training so a hard
Kaggle interruption cannot cause the same reserved examples
to be selected again.
Resume safety
The checkpoint contains:
- model.safetensors
- optimizer.pt
- rng_state.pt
- training_state.json
- example_ledger.json
- tokenizer.json
- tokenizer_config.json
- token_id_manifest.json
- config.json
Dataset fingerprint and tokenizer artifact hashes are verified
before resume.
Current state
Completed sessions:
5
Unique examples reserved/trained:
200,000
Unique completed examples:
460,000
Global optimizer steps:
7,189
Last session loss:
0.17293200694084168
