1from transformers import AutoModelForCausalLM
2model = AutoModelForCausalLM.from_pretrained("wpferrell/llama-3-8b-instruct-bigsmall")
It works exactly like loading the original model. No code changes needed.
Every weight is mathematically identical to the original model.
1from bigsmall import BigSmallStreamingModel
2
3model = BigSmallStreamingModel.from_pretrained(
4 "wpferrell/llama-3-8b-instruct-bigsmall",
5 device="cuda",
6 lru_max_vram_gb=2.0,
7)
Uses up to ~12- less VRAM than standard loading by streaming layers on demand.
1import bigsmall
2state_dict = bigsmall.stream_from_hub("wpferrell/llama-3-8b-instruct-bigsmall", device="cpu")
Decompresses directly from the HuggingFace CDN over HTTP range requests. With the default cache=False, no .bs file is ever written to disk (V10).
1import bigsmall
2from safetensors.torch import save_file
3
4# bigsmall decompress works on local .bs files, not Hub repos, so
5# stream the weights from the Hub and write them out as safetensors.
6state_dict = bigsmall.stream_from_hub("wpferrell/llama-3-8b-instruct-bigsmall", device="cpu")
7save_file(state_dict, "llama-3-8b-instruct-bigsmall.safetensors")
This is a lossless-compressed copy of
meta-llama/Meta-Llama-3-8B-Instruct. All credit to the original authors. The weights are unchanged.
1pip install "bigsmall>=4.0"
2bigsmall compress my-model/ -o my-model.bs
1@misc{bigsmall2026,
2 title={BigSmall: Lossless Neural Network Weight Compression},
3 author={Ferrell, Will},
4 year={2026},
5 doi={10.5281/zenodo.20279247},
6 url={https://doi.org/10.5281/zenodo.20279247}
7}
