Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
vanilla-340M-4096-model – AI Model by zaydzuhri | AlphaNeural AI | AlphaNeural AI
You can deploy this model and start earning money today!
zaydzuhri
/
vanilla-340M-4096-model
like
0
safetensors
transformer
2504.20966
2508.19228
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This model is from the paper arxiv.org/abs/2504.20966
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Also used in arxiv.org/abs/2508.19228
Token Order Prediction
See code:
https://github.com/zaydzuhri/softpick-attention
This model is only usable through these repositories:
https://github.com/zaydzuhri/flash-linear-attention/tree/softpick-attention
https://github.com/zaydzuhri/flame/tree/softpick-attention
