To prove this extreme compression didn't lobotomize the model, we subjected the ~11GB ONYX-mini quant to a grueling 20-minute agentic stress test designed to break long-horizon reasoning. We tasked the model with writing a compilable Rust program to calculate quantum harmonic oscillator eigenvalues via nalgebra matrix diagonalization, forcing it to execute the code via bash, read the compiler errors, and autonomously debug its own Unicode and eigenvalue-sorting bugs. After successfully patching the math to achieve <0.05% analytical error, the model seamlessly passed the entire technical explanation through a three-step translation gauntlet into formal Japanese, Spanish, and colloquial Egyptian Arabic. The fact that a heavily crushed 35B MoE running on a consumer laptop GPU can hold the working memory required to write physics code, run cargo build, fix its own mistakes, and perfectly execute multilingual code-switching in just 20 minutes proves that ONYX doesn't just shrink the weights—it preserves the model's agentic soul.

🪨 ONYX

Adaptive Precision Engine

This repo contains ONYX Quants of OrionLLM/GRM-3.2-Sky

⚙️ The ONYX Architecture

🧠 Dynamic Layer Sensitivity

Replaces hardcoded edge boundaries with real activation variance measurements. ONYX autonomously identifies critical layers (like mid-network attention blocks) and protects them dynamically.

🎯 Router-Weighted Imatrix

Captures MoE router probabilities and multiplies them into activation scales. This forces the quantizer to aggressively crush "cold" experts while fiercely protecting "hot" ones within the same tensor block.

🏗️ Architecture-Agnostic

Dynamically reads HuggingFace modules and the generated F16 GGUF to map tensors. No hardcoded regex. Works out-of-the-box on Llama, DeepSeek, and custom hybrid SSM/MoE architectures.

Tier Name	Target Quality	Target Size	Middle Layer Strategy
🪨 `quality`	Q8 Match	~21 GB	IQ4_XS
⚖️ `balanced`	Q6 Match	~24 GB	Q5_K
📦 `compact`	Q4 Match	~16 GB	Q3_K
🚀 `mini`	Q2 Match	~12 GB	IQ2_XXS

📚 Credits & Foundations

👉 APEX Quantization Method
Ettore Di Giacinto & Richard Palethorpe (LocalAI Team). ONYX evolves the layer-wise precision gradients and MoE-aware tensor classification outlined in the APEX technical paper into a fully dynamic, data-driven engine.

👉 Bartowski and Lamim
For the excellent semantic imatrix calibration dataset that powers ONYX's activation scaling.

👉 llama.cpp
Georgi Gerganov and contributors for the foundational inference and quantization engine.

👉 HuggingFace Accelerate
For the init_empty_weights() context manager that makes the 0-RAM "Ghost Model" possible on consumer hardware.

Support the Project

A coffee in Ethereum would be cool! Although I don't drink coffee—I think it tastes like burnt water—but a pink lemonade would be fire! 🔥

0xDEE7fa8C421BD038D32e4441ea1aDe72fE973706

recommended sampling parameters:

--temp 0.85 --top-p 0.95 --top-k 40 --min-p 0.05 --repeat-penalty 1.1 --presence-penalty 0.0