GGUF quantizations of GestaltLabs/Ornstein-3.5-9B-V2 — the reinforcement-learning post-training (V2) of Ornstein 3.5 9B. Each quant has a sibling .sha256 checksum, and a separate vision projector (mmproj) ships the multimodal tower for image/video input.

Text:

Multimodal (image/video) — add the vision projector:

Quality and speed scale with quant size; Q4_K_M is a strong default for ~8 GB of VRAM/RAM.

Every quant embeds the model's native multi-token-prediction (MTP) draft head (GGUF block_count 33), so self-speculative decoding runs from a single file — no separate draft model needed:

I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded — balancing GPU costs against a student budget. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.

Apache 2.0 — inherited from the Qwen 3.5 9B base release.