When Does One-Shot Pruning Beat Iterative Optimisation? Second-Order Correction for Sparse LLMs on Neuromorphic Hardware
Kimia Gholami et al., NeurIPS 2026 submission

Performance

Metric	Value
WikiText-2 PPL (dense)	14.18
WikiText-2 PPL (sparse)	1,952
Sparsity	80%
ARC-Easy	0.269
ARC-Challenge	0.222
HellaSwag	0.258
PIQA	0.527
WinoGrande	0.491
LAMBADA	0.000
Average	0.294

Best PPL among all methods at 80% sparsity on HGRN-1.3B (SparseGPT: 2,811; AWP: 17,756; Wanda: 75,620; RIA: 26,817). At this extreme compression level all methods produce largely incoherent output; this model is provided primarily for hardware sparsity efficiency benchmarking rather than language quality.

Sparsity structure

Semi-structured sparsity: exactly k = floor(d_in * 0.8) weights are zeroed per output row in every linear layer. Zero weights map directly to zero spike events on Intel Loihi.

Loading

python

import torch, fla
from fla.models.hgrn import HGRNConfig, HGRNForCausalLM
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer

AutoConfig.register("hgrn", HGRNConfig, exist_ok=True)
AutoModelForCausalLM.register(HGRNConfig, HGRNForCausalLM, exist_ok=True)

model_id = "ikimyaii/hgrn-1.3B-obs-cancel-block-80pct"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).cuda()

Install FLA first:

pip install flash-linear-attention