Paper: Sparse Finetuning for Inference Acceleration of Large Language Models
Code: https://github.com/neuralmagic/deepsparse/tree/main/research/mpt

This model was produced from a MPT-7B base model finetuned on the GSM8k dataset for 2 epochs and quantized.

GSM8k zero-shot accuracy with lm-evaluation-harness : 27.8%

All MPT model weights are available on SparseZoo and CPU speedup for generative inference can be reproduced by following the instructions at DeepSparse

For general questions on these models and sparsification methods, reach out to the engineering team on our community Slack.