Large language model (LLM) inference on consumer-grade hardware remains bottlenecked by autoregressive decoding, where each token requires a separate forward pass through the model. Speculative decoding mitigates this by using a small draft model to propose multiple tokens per step, but introduces a verifier overhead that can negate gains on memory-bandwidth-constrained systems such as Apple Silicon. We present PIPO-Hayula, an implementation of the Pair-In-Pair-Out (PIPO) multi-token prediction
1@techreport{hayulalab2026pipohayulainference,
2 title={Hayula Research Paper — Hayula Research},
3 author={Hayula AI Lab},
4 year={2026},
5 url={https://huggingface.co/hayulalab/pipo-hayula-inference-paper}
6}