You can deploy this model and start earning money today!

sxiong/
DHSA-Qwen2.5-3B-Instruct-BF16

Name: DHSA-Qwen2.5-3B-Instruct-BF16
Availability: InStock

Views

No views yet

DHSA-Qwen2.5-3B-Instruct-BF16

This repository hosts the predictor weights for Qwen2.5-3B-Instruct (BF16) used in Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference (ICML 2026 Spotlight).

Checkpoint Schema

Every checkpoint contains:

Field	Description
`predictor_config`	Explicit predictor architecture and inference parameters
`state_dict`	Trained predictor weights in FP32
`sample_prototypes`	BF16 prototypes used for nearest-sample matching
`density_config_overrides`	Optional density-specific dynamic budget ranges

Usage

First, clone our repository and download the checkpoint files from Hugging Face:

bash

git clone https://github.com/xiongsiheng/DHSA
cd DHSA

hf download sxiong/DHSA-Qwen2.5-3B-Instruct-BF16 \
  --local-dir checkpoints/DHSA-Qwen2.5-3B-Instruct-BF16

Pass the desired checkpoint to the evaluation command:

bash

CUDA_VISIBLE_DEVICES=0 python run_ruler.py \
  --model_name Qwen/Qwen2.5-3B-Instruct \
  --data_dir data/RULER \
  --sparsity-mask DHSA_learned_topK_static \
  --predictor-checkpoint checkpoints/DHSA-Qwen2.5-3B-Instruct-BF16/predictor_static.pt \
  --density 0.125 \
  --q-block-size 128 \
  --k-block-size 32 \
  --save_dir results_ruler

Citation

@inproceedings{xionglong,
  title={Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference},
  author={Xiong, Siheng and Zou, Joe and Fekri, Faramarz and Cho, Yae Jee},
  booktitle={Forty-third International Conference on Machine Learning}
}