Qwen3.5 Tiny Test Model
A tiny Qwen3.5 hybrid MoE model for testing and validation purposes.
This model has random weights and is not trained. It exists to validate the
architecture implementation and hub upload pipeline.
Architecture
- Type: Hybrid MoE (Gated DeltaNet + Gated Attention)
- Parameters: 138,261,536 total
- Layers: 8 (6 DeltaNet + 2 Full Attention)
- Experts: 8 routed (top-2) + 1 shared
- Embedding dim: 256
- Vocab size: 248,320
- Context: 4096 tokens
