Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Qwen3-Next-REAP-40B-A3B-Instruct-GGUF – AI Model by lovedheart | AlphaNeural AI
You can deploy this model and start earning money today!
lovedheart
/
Qwen3-Next-REAP-40B-A3B-Instruct-GGUF
like
0
gguf
text-generation-inference
Qwen/Qwen3-Next-80B-A3B-Instruct
quantized
apache-2.0
endpoints_compatible
us
conversational
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
qwen3-next-instruction
Qwen3-Next-REAP-40B-A3B-Instruct
has the following specifications:
Type:
Causal Language Models
Number of Parameters
: 40B in total and 3B activated
Hidden Dimension
: 2048
Number of Layers
: 48
Hybrid Layout
: 12 * (3 * (Gated DeltaNet -> MoE) -> 1 * (Gated Attention -> MoE))
Gated Attention
:
Number of Attention Heads
: 16 for Q and 2 for KV
Head Dimension
: 256
Rotary Position Embedding Dimension
: 64
Gated DeltaNet
:
**Number of Linear Attention Heads: 32 for V and 16 for QK
**Head Dimension: 128
Mixture of Experts
:
**Number of Experts: 256 (uniformly pruned from 512)
**Number of Activated Experts: 10
**Number of Shared Experts: 1
Context Length
: 262,144 natively and extensible up to 1,010,000 tokens
Compression Method
: REAP (Router-weighted Expert Activation Pruning)
Compression Ratio
: 50% expert pruning
