Views
No views yet
EleutherAI/pythia-12b-deduped.pytorch_model-*.bin files exist there. This caused multi-process inference
loads to bottleneck on disk because pickle deserialization is in-process work that
doesn't share well via OS page cache. Loading 8 sweep procs of pythia-12b-deduped
from .bin took ~50 min on local NVMe; safetensors zero-copy mmap drops that to ~5 min.EleutherAI/pythia-12b-deduped — converted with:1model = AutoModelForCausalLM.from_pretrained("EleutherAI/pythia-12b-deduped", torch_dtype=torch.float16)
2model.save_pretrained(out_dir, safe_serialization=True, max_shard_size="10GB")1@misc{biderman2023pythia,
2 title={Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling},
3 author={Stella Biderman and Hailey Schoelkopf and others},
4 year={2023},
5 eprint={2304.01373},
6 archivePrefix={arXiv},
7}