⚡ Fuel the Lab: Keep the Optimization Loops Running
Every single ZeroDigest YMQ-MTP release is handcrafted and manually calibrated via intensive importance-matrix sweeps to protect critical logic pathways. This project is entirely independent research—no automation bots, no corporate backers, and no external funding. Running multi-hour compute arrays consumes massive local infrastructure overhead out-of-pocket. Consider checking out our compiler or supporting our compute costs!
⚖️ An Architecture-Aware, AutoRound-Inspired MoE Mixed Precision Layout
This repository features advanced, custom architecture-aware quantizations of Ornith-1.5-35B (Uncensored) processed directly from official raw BF16 source files using the custom YMQ-Compiler (v2.0) log-space framework.
These builds natively preserve the delicate Mixture-of-Experts routing topology and utilize high-context optimization parameters tailored for demanding local agent execution environments (such as RooCode/Aider).
Near lossless. Solid performance scaling across heavier consumer setups.
📉 Perplexity Evaluation Metrics (WikiText-2)
The following metrics show the mathematical quality preservation of the YMQ-Compiler log-space cluster analysis compared to standard linear quantization layouts. Tested natively via llama-perplexity over a 4096 context window using the official WikiText-2 test corpus.
💡 The MoE Compression Breakthrough: YMQ vs. Standard Quants
Standard quantization pipelines act like a blunt hammer. They apply a uniform, flat bit-depth across every layer of the model, which completely breaks the delicate routing paths of Mixture-of-Experts (MoE) architectures.
Independent local testing using the official llama-perplexity harness exposes the massive optimization gap between standard, flat layouts and the architecture-aware YMQ-Compiler:
Standard Flat Q4_K_M (~21.0 GB):
Hits a high 13.4757 perplexity score on the test corpus.
Starves the core attention entry channels and native MTP speculator tracking paths of bit-depth resolution.
Model experiences severe tracking fatigue under stress.
YMQ-Compiler M Preset (~16.0 GB):
Scores a spectacular 12.4310 perplexity score on the exact same corpus.
⚖️ YMQ vs. Uniform Quantization (The AutoRound Philosophy)
Standard quantization pipelines apply a blunt, uniform bit-depth across every single layer in a model. This blunt approach completely collapses the delicate routing structures of large Mixture-of-Experts (MoE) architectures, starving critical logic anchors of necessary precision while bloating file sizes with idle parameters.
The YMQ-Compiler implements a post-training optimization philosophy similar to advanced weight-tuning frameworks like Intel's AutoRound:
Targeted Bit Isolation: Instead of applying a flat matrix mask, YMQ operates strictly in Log-Space. It surgically identifies the core attention entry lanes and high-leverage routing networks, locking them behind heavy Q5_K and Q6_K high-fidelity safety shields.
Expert Layer Flattening: It takes the massive pool of background expert weights (ffn_*_exps) and compresses them aggressively using dense, non-linear grids (IQ2_XS and IQ4_NL). Because these background parameters make up the majority of the file footprint but are rarely active at the same time, the compiler shaves off gigabytes of background noise without breaking the model's main train of thought.
The result is a highly stable, custom mixed-precision portfolio that matches the low perplexity and high instruction clarity of premium optimized configurations, while allowing 16GB and 24GB single-GPU setups to stream massive context windows with absolute structural peace of mind!
🛠️ The YMQ Compilation Architecture
Standard quantization pipelines treat network tensors like a flat dataset, applying destructive blanket low-bit compression to delicate tracking networks. The YMQ-Compiler solves high-context logic decay by parsing model files dynamically via an automated, multi-tiered protection matrix:
Log-Space Gap Detection Clustering: Instead of flat percentage thresholds, the engine computes statistical cluster variances in log-space, successfully isolating intermediate logical reasoning spikes and elevating them to stable non-linear 4-bit (IQ4_XS) formats, while compressing idle fact-storage layers to aggressive 2-bit baselines.
Fading Boundary Tapering: Recognizes the extreme fragility of initial token entry data vectors, forcing an input wave cushion (L00=IQ4_NL → L01=IQ4_XS → L02=IQ3_XXS) that gradually stabilizes parameters before hitting the fallback pools.
Asymmetric Vocabulary Shielding: Fixes tied-weight boundary errors by mapping the final logit classification exit heads to robust configurations to completely eliminate formatting loops and API tag leakage under deep contexts.
Compact vision projection for VRAM-constrained setups. Retains strong image understanding at reduced footprint.
Pass via --mmproj <path-to-file> in your llama-server invocation (see example above).
☕ Support & Future R&D
If the YMQ-Compiler builds saved your context window from collapsing or optimized your active development cycle speeds, consider buying a coffee to fund further low-level optimization research. Your support keeps the server nodes baking future model scales!
The compiler pipeline automation engine, setup thresholds, and structural mapping rules are open-source. To view the implementation details or compile your own custom models natively using this profile layout, visit the official development hub:
