You can deploy this model and start earning money today!

lovedheart/
Qwen3-Next-REAP-20B-A3B-Thinking-GGUF

Name: Qwen3-Next-REAP-20B-A3B-Thinking-GGUF
Availability: InStock

Views

No views yet

Qwen3-Next-REAP-20B-A3B-Thinking has the following specifications:

Type: Causal Language Models
Number of Parameters: 20B in total and 3B activated
Hidden Dimension: 2048
Number of Layers: 48
Hybrid Layout: 12 * (3 * (Gated DeltaNet -> MoE) -> 1 * (Gated Attention -> MoE))
Gated Attention:
Number of Attention Heads: 16 for Q and 2 for KV
Head Dimension: 256
Rotary Position Embedding Dimension: 64
Gated DeltaNet:
**Number of Linear Attention Heads: 32 for V and 16 for QK
**Head Dimension: 128
Mixture of Experts:
**Number of Experts: 128 (uniformly pruned from 512)
**Number of Activated Experts: 10
**Number of Shared Experts: 1
Context Length: 262,144 natively and extensible up to 1,010,000 tokens
Compression Method: REAP (Router-weighted Expert Activation Pruning)
Compression Ratio: 75% expert pruning
Specialized: Math, Physics, Control Engineering, Scientific Writing

lovedheart/Qwen3-Next-REAP-20B-A3B-Thinking-GGUF