Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
GSA-PT-Llama-3.2-1B-chunk4-chunk4 – AI Model by gist-sparse-attention | AlphaNeural AI
You can deploy this model and start earning money today!
gist-sparse-attention
/
GSA-PT-Llama-3.2-1B-chunk4-chunk4
like
0
safetensors
llama
gsa
gist-sparse-attention
long-context
meta-llama/Llama-3.2-1B
finetune
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
GSA-PT-Llama-3.2-1B-chunk4-chunk4
This model is
continued pretrained
from
meta-llama/Llama-3.2-1B
using
Gist Sparse Attention (GSA)
with chunk size
chunk4-chunk4
.
Paper
GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding
Model Details
Field
Value
Base model
meta-llama/Llama-3.2-1B
Training type
Continued Pretraining
Chunk size
chunk4-chunk4
Architecture
Llama-3.2-1B
