Views
No views yet
| Property | Value |
|---|---|
| Total parameters | 450,993,024 (~451M) |
| Active parameters | ~250M (MoE: 4 experts × 2 active) |
| Layers | 10 |
| Hidden size | 1,152 |
| Attention | GQA — 18 query heads / 6 KV heads, head_dim 64, RoPE θ=10K |
| Mamba SSM | layers 3 & 7 · d_state=16 · d_conv=4 · expand=2 |
| MoE | layers 6–9 · 4 experts × 2 active · hidden 2,304 |
| Context length | 256 tokens |
| Vocabulary | 65,536 (SentencePiece BPE) |
| Languages | العربية + English |
| File | Description |
|---|---|
ivis-400m-pytorch.pt | Full PyTorch checkpoint (state_dict + model_info) |
ivis-400m.gguf | GGUF container (f16, 910MB, all 135 tensors) |
config.json | Architecture hyperparameters |
tokenizer.model / tokenizer.vocab | SentencePiece BPE tokenizer (AR+EN) |
infer_numpy.py | Local inference engine — pure numpy, no PyTorch needed |
convert_to_gguf.py | Torch-free .pt → GGUF converter |
1pip install gguf sentencepiece numpy # no torch required!
2
3python infer_numpy.py --gguf ivis-400m.gguf \
4 --tokenizer tokenizer.model \
5 --prompt "البرمجة هي" --tokens 40
6
7# measured on CPU: d_model=1152 layers=10 heads=18/6 moe=4@L6+
8# generation speed ≈ 7 tokens/s (numpy, single thread)
9
10## Related | نماذج مرتبطة
11
12- [`nepetai/ivis-50m-pilot`](https://huggingface.co/nepetai/ivis-50m-pilot) — same architecture family at 55M scale, **fully validated end-to-end** with measured training curve (loss 9.20→0.92 over 300 steps), GGUF conversion and a pure-numpy inference engine that runs without PyTorch.
13
14## Citation
15
16```bibtex
17@misc{ivis-nepetai-400m,
18 title={Ivis-400M-Nepetai: Hybrid Transformer-Mamba-MoE Language Model},
19 author={Nepetai Arabic},
20 year={2026},
21 url={https://huggingface.co/nepetai/ivis-400m-gpu}
22}