Views
No views yet

| Model Specification | OneGenomeRice (OGR) |
|---|---|
| Model Scale | |
| Total Parameters | 1.25B |
| Activated Parameters | 0.33B |
| Architecture | |
| Architecture | MoE |
| Number of Experts | 8 |
| Selected Experts per Token | 2 |
| Number of Layers | 12 |
| Attention Hidden Dimension | 1024 |
| Number of Attention Heads | 16 (GQA, 8 KV groups) |
| MoE Hidden Dimension (per Expert) | 4096 |
| Vocabulary Size | 128 (padded) |
| Context Length | up to 1Mb |