Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
scaled-softpick-340M-4096-batch16-steps100000-20250423-192416 – AI Model by zaydzuhri | AlphaNeural AI
You can deploy this model and start earning money today!
zaydzuhri
/
scaled-softpick-340M-4096-batch16-steps100000-20250423-192416
like
0
2504.20966
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This model is from the paper arxiv.org/abs/2504.20966
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
See code:
https://github.com/zaydzuhri/softpick-attention
This model is only usable through these repositories:
https://github.com/zaydzuhri/flash-linear-attention/tree/softpick-attention
https://github.com/zaydzuhri/flame/tree/softpick-attention
