OA-EM (Output-Aware Expectation-Maximisation) is a Hessian-weighted EM algorithm that significantly improves codebook initialization for additive quantization, particularly at extreme compression rates. See the paper
Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization for details.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 "kennedyian94/Llama-3.2-3B-AQLM-OA-EM-2Bit-2x8",
5 trust_remote_code=True,
6 torch_dtype="auto",
7 device_map="auto"
8)
9tokenizer = AutoTokenizer.from_pretrained("kennedyian94/Llama-3.2-3B-AQLM-OA-EM-2Bit-2x8")
10
11inputs = tokenizer("The meaning of life is", return_tensors="pt").to(model.device)
12outputs = model.generate(**inputs, max_new_tokens=100)
13print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Ian W. Kennedy and Nafise Sadat Moosavi, University of Sheffield
1@article{kennedy2026oaem,
2 title={Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization},
3 author={Kennedy, Ian W. and Moosavi, Nafise Sadat},
4 journal={arXiv preprint arXiv:2604.08118},
5 year={2026}
6}
