Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
llama-3.1-8b-instruct-4x2-moe – AI Model by Jongbin-kr | AlphaNeural AI
You can deploy this model and start earning money today!
Jongbin-kr
/
llama-3.1-8b-instruct-4x2-moe
like
0
transformers
safetensors
upcycled_llama
text-generation
llama
moe
sparse-upcycling
conversational
custom_code
meta-llama/Llama-3.1-8B-Instruct
finetune
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Llama 3.1 8B Instruct 4x2 MoE
This checkpoint sparse-upcycles every Llama 3.1 8B Instruct decoder MLP into four initially identical experts with dropless top-2 routing.
Load it with
AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True)
.
