gist-sparse-attention/

GSA-FT-Llama-3.2-1B-data

This is the supervised fine-tuning dataset used for training GSA (Gist Sparse Attention) models based on Llama-3.2-1B. The answers to the questiosn are generated by Deepseek-V3.2 Each sample is formatted with GSA gist tokens for instruction fine-tuning.

GSA: Gist Sparse Attention via Learnable Compression and Selective Unfolding

yuzhenm/GSA-FT-Llama-3.2-1B-chunk8 yuzhenm/GSA-FT-Llama-3.2-1B-chunk16… See the full description on the dataset page: https://huggingface.co/datasets/gist-sparse-attention/GSA-FT-Llama-3.2-1B-data.