Views
No views yet
3 Mamba2 + 1 MLA + MoE (2 experts, top-1) model with 26.73M parameters
1# Install blazr (recommended inference server)
2cargo install blazr
3
4# Generate text
5blazr generate --model abal1000x/nano-start_64_26m_f32 --prompt "Hello, world!"
6
7# Start API server
8blazr serve --model abal1000x/nano-start_64_26m_f32 --port 80801# Basic generation
2blazr generate --model abal1000x/nano-start_64_26m_f32 --prompt "Your prompt here" --max-tokens 100
3
4# With sampling parameters
5blazr generate --model abal1000x/nano-start_64_26m_f32 \
6 --prompt "Once upon a time" \
7 --max-tokens 200 \
8 --temperature 0.8 \
9 --top-p 0.91# Start the server
2blazr serve --model abal1000x/nano-start_64_26m_f32 --port 8080
3
4# The server provides OpenAI-compatible endpoints:
5# - POST /v1/completions
6# - POST /v1/chat/completions
7# - GET /v1/models1from openai import OpenAI
2
3client = OpenAI(base_url="http://localhost:8080/v1", api_key="unused")
4
5# Chat completion
6response = client.chat.completions.create(
7 model="abal1000x/nano-start_64_26m_f32",
8 messages=[{"role": "user", "content": "Hello!"}],
9 max_tokens=100
10)
11print(response.choices[0].message.content)1# Using huggingface-cli
2huggingface-cli download abal1000x/nano-start_64_26m_f32 --local-dir ./model
3
4# Then run locally
5blazr generate --model ./model --prompt "Hello!"This model requires blazr for inference.
| Property | Value |
|---|---|
| Architecture | 3 Mamba2 + 1 MLA + MoE (2 experts, top-1) |
| Parameters | 26.73M |
| Hidden Size | 128 |
| Layers | 4 |
| Vocab Size | 100315 |
| Max Sequence Length | 64 |
| Precision | FP32 |
| License | Apache-2.0 |