You can deploy this model and start earning money today!

sxiong/
DHSA-Llama-3.1-8B-Instruct-NF4

Name: DHSA-Llama-3.1-8B-Instruct-NF4
Availability: InStock

Views

No views yet

DHSA-Llama-3.1-8B-Instruct-NF4

This repository hosts the predictor weights for Llama-3.1-8B-Instruct (NF4) used in Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference (ICML 2026 Spotlight).

Checkpoint Schema

Every checkpoint contains:

Field	Description
`predictor_config`	Explicit predictor architecture and inference parameters
`state_dict`	Trained predictor weights in FP32
`sample_prototypes`	BF16 prototypes used for nearest-sample matching
`density_config_overrides`	Optional density-specific dynamic budget ranges

Usage

First, clone our repository and download the checkpoint files from Hugging Face:

bash

git clone https://github.com/xiongsiheng/DHSA
cd DHSA

hf download sxiong/DHSA-Llama-3.1-8B-Instruct-NF4 \
  --local-dir checkpoints/DHSA-Llama-3.1-8B-Instruct-NF4

Pass the desired checkpoint to the evaluation command:

bash

CUDA_VISIBLE_DEVICES=0 python run_ruler.py \
  --model_name meta-llama/Llama-3.1-8B-Instruct \
  --use_quant \
  --data_dir data/RULER \
  --sparsity-mask DHSA_learned_topK_static \
  --predictor-checkpoint checkpoints/DHSA-Llama-3.1-8B-Instruct-NF4/predictor_static.pt \
  --density 0.125 \
  --q-block-size 128 \
  --k-block-size 32 \
  --save_dir results_ruler

Citation

@inproceedings{xionglong,
  title={Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference},
  author={Xiong, Siheng and Zou, Joe and Fekri, Faramarz and Cho, Yae Jee},
  booktitle={Forty-third International Conference on Machine Learning}
}