Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
SEWY2-640M-untrained – AI Model by Aarushhh | AlphaNeural AI
You can deploy this model and start earning money today!
Aarushhh
/
SEWY2-640M-untrained
like
0
transformers
safetensors
Sewy_v2
text-generation
conversational
custom_code
cc-by-nc-sa-4.0
autotrain_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Sewy2 (untrained) 640m
It is a new MoE architecture which uses the following:
DeepseekV3
nGPT
ResFormer
NeuTRENO (as in resformer)
Tanh logit softcapping (as in Gemma2)
Architecture:
32 Layers
32 Heads
32 KV heads
64 experts
8 experts per token
