⚡ Fuel the Lab: Keep the Optimization Loops Running
Every single ZeroDigest YMQ-MTP release is handcrafted and manually calibrated via intensive importance-matrix sweeps to protect critical logic pathways. This project is entirely independent research—no automation bots, no corporate backers, and no external funding. Running multi-hour compute arrays consumes massive local infrastructure overhead out-of-pocket. Consider checking out our compiler or supporting our compute costs!
⚖️ An Architecture-Aware, AutoRound-Inspired Mixed Precision Layout
This repository features advanced, custom architecture-aware quantizations of Qwen3.8-27B (Uncensored) processed directly from official raw BF16 source files using the custom YMQ-Compiler (v2.0) log-space framework.
These builds natively support parallel multi-token prediction (MTP) speculation engines and utilize high-context optimization parameters tailored for demanding code development API execution environments (such as RooCode/Aider).
ZeroDigest YMQ Logo
📊 Quantization Preset Tier Details
Preset Tier
Total Size
Target Usage / Memory VRAM Profile
Cognitive Real-World Coding Quality
XXS-Pro
~9.3 GB
⚠️ Experimental Low-VRAM Sandbox
The Ultra-Compact Frontier. Perplexity = 8.2084. Packs the 27B dense matrix into a sub-10GB footprint. In intense multi-stage agent workflows, it can occasionally trigger context amnesia or formatting loops, but features heavily insulated upper routing tracks to protect basic logical structures.
XXS
~9.8 GB
Absolute VRAM Squeeze / 12GB Card Lifeline
Massive structural quantization noise. Best restricted to low-context, single-turn instructions. Fits 12GB cards with context cache breathing room.
XS-TI
~10.2 GB
⚡ 12GB Card Lifeline (High-Context)
The 12GB Context Champion. Safely pins core attention layers to a stable IQ3_XXS gradient while crushing non-critical auxiliary arrays to IQ2_XS.
XS-Pro
~11.0 GB
⚡ Dedicated 12GB VRAM Champion
High compression economy baseline. Optimized to prevent API degradation during deep context tasks.
S-Pro
~12.5 GB
💎 16GB Workstation Driver
Pristine conversational fluidness, using full attention armor.
M-TI
~12.6 GB
💎 Premium 16GB GPU Workspace Driver
The 16GB Workstation Choice. Employs a robust Q5_K/IQ4_XS mixed-precision gradient that protects logical reasoning focus while leaving over 3.4GB of VRAM wide open.
M
~14.5 GB
The Ultimate Coding Sweet Spot (Recommended)
Elite logical stability. Complete logic clarity. It crushes standard industry 4-bit alternatives.
L-TI
~14.1 GB
🔥 Heavy 16GB Workstation Choice (Precision)
The 16GB High-Fidelity Champion. Features a robust Q6_K/IQ4_NL armored peak [local]. Minimizes information entropy drift to keep structural logic stable over long, multi-turn agent execution loops.
Mathematical saturation ceiling. Full precision logic tracks for massive multi-file codebase operations.
🎯 Quick Selection Guide
Use Case
Recommended Preset
Why
12GB GPU
XS-TI or XS-Pro
Best 12GB card performance with high-context support
16GB GPU (Recommended)
M-TI or S-Pro
The sweet spot for coding tasks with optimal VRAM headroom
24GB GPU
L-TI or M
~14GB models fit well on 24GB VRAM with headroom for context
Low-VRAM / Experimental
XXS-Pro
Sub-10GB footprint for testing and lightweight workflows
📉 Perplexity Evaluation Metrics (WikiText-2)
The following metrics demonstrate the mathematical quality preservation of the YMQ-Compiler log-space cluster analysis compared to standard linear quantization layouts. Tested natively via llama-perplexity over a 4096 context window using the official WikiText-2 test corpus.
Legacy S Preset Deprecated: The older, standard S configuration (~12.2 GB) has been officially removed from the repository.
Upgrade to S-Pro (~12.5 GB): We have replaced it with the newly engineered S-Pro preset.
Legacy XS Preset Deprecated: The older, standard XS configuration (~11.0 GB) has been permanently removed from the repository tree.
Upgrade to XS-Pro (~10.5 GB): We have officially replaced it with the newly engineered XS-Pro preset. Score 7.1665 vs old XS score 8.1516
💡 The Multi-Tier Grid Breakthrough: Standard S vs. S-Pro
During intensive local workspace validation passes, our architecture-aware YMQ-Compiler successfully mapped out a radical new bit-allocation matrix. By splitting the layer distribution, we created a premium, high-fidelity alternative to our standard budget tier:
Standard S Preset (~12.2 GB): Perplexity = 8.0351. Features a balanced log-space gradient. Highly capable of handling single-turn scripts and quick edits. Successfully ingested a clean 91kb codebase chunk to resolve deep priority-ordered dictionary bugs natively.
S-Pro Preset (~12.5 GB): Perplexity = 7.0687. By aggressively compressing auxiliary tensor lanes down to Q2_K but raising the background baseline floor to IQ3_XXS, S-Pro eliminates a massive wave of background quantization noise—dropping paper perplexity by a massive ~1.0 point while only adding a few megabytes of file weight.
The practical result is a premium, low-overhead everyday driver for 16GB GPU setups. Backed by full Q5_K reasoning armor.
💡 The Uncensored Performance Breakthrough
Notice that the Uncensored M preset achieves an elite score of 6.8176, outperforming even the original base model's score (6.8413). This occurs because removing the artificial refusal safety layers allows the model's underlying Attention and Mamba SSM weights to predict text paths with absolute, unrestricted mathematical clarity.
By pairing JonathanColetti's pristine abliteration weights with the YMQ-Compiler's log-space gate insulation, this preset matches the raw reasoning power of the massive 19GB XL file while clawing back a clean 5 Gigabytes of VRAM overhead cache space for local RooCode/Aider coding loops!
💡 Engineering Notes on the XXS-Pro Layout
The XXS-Pro preset is a highly aggressive exploration pass utilizing an optimized mixed-precision architecture template:
HIGH="IQ3_XXS" (3.0 BPW) -> MID="IQ3_XXS" -> LOW="IQ2_S" (2.5 BPW) -> FLOOR="IQ2_XXS" (2.06 BPW).
By adding custom FLOOR_TARGET parameters, we aggressively crushed the auxiliary and background matrix noise to stay beneath a hard 9.5 GB memory limit. While this compression level introduces enough quantization noise to challenge complex multi-file edit loops, our log-space steering gate protection allows the model to retain surprisingly strong English language capabilities and shorter script tracking entirely within low-VRAM graphics memory buffers!
⚖️ YMQ vs. Uniform Quantization (The AutoRound Philosophy)
Standard quantization pipelines apply a blunt, uniform bit-depth across every single layer in a model. This wastes valuable VRAM on silent background layers while starving critical logic anchors of necessary precision.
The YMQ-Compiler implements a philosophy similar to advanced weight-tuning frameworks like Intel's AutoRound:
Targeted Bit Allocation: It strips bits away from low-leverage background tensors and automatically re-allocates that saved VRAM budget straight into full high-fidelity shields for the model's highest cognitive spikes and boundary pathways.
Instant Optimization: Instead of running heavy, days-long optimization training loops, YMQ achieves a highly accurate mixed-precision layout instantly by analyzing layer importance metrics in log-space.
The result is a custom mixed-precision portfolio that matches the low perplexity and high context stability of premium optimized quants (like AutoRound), while maintaining an ultra-lightweight, high-speed single-GPU cache footprint.
🛠️ The YMQ Compilation Architecture
Standard quantization pipelines treat network tensors like a flat dataset, applying destructive blanket low-bit compression to delicate tracking networks. The YMQ-Compiler solves high-context logic decay by parsing model files dynamically via an automated, multi-tiered protection matrix:
Log-Space Gap Detection Clustering: Instead of flat percentage thresholds, the engine computes statistical cluster variances in log-space, successfully isolating intermediate logical reasoning spikes and elevating them to stable non-linear 4-bit (IQ4_XS) formats, while compressing idle fact-storage layers to aggressive 2-bit baselines.
Fading Boundary Tapering: Recognizes the extreme fragility of initial token entry data vectors, forcing an input wave cushion (L00=IQ4_NL → L01=IQ4_XS → L02=IQ3_XXS) that gradually stabilizes parameters before hitting the fallback pools.
Dedicated Gate Insulation: Hard-shields volatile parallel Transformer Multi-Head Attention and Mamba Linear State Space Model (SSM) routing paths, keeping context tracking perfectly noise-free.
Asymmetric Vocabulary Shielding: Fixes tied-weight boundary errors by mapping the final logit classification exit heads to robust configurations to completely eliminate formatting loops and API tag leakage under deep contexts.
Native Next-N Speculative Stripping: Processed with advanced pre-tokenizer stripping to ensure zero index offset drift or layer-shifting risks across hybrid configurations.
Compact vision projection for VRAM-constrained setups. Retains strong image understanding at reduced footprint.
Pass via --mmproj <path-to-file> in your llama-server invocation (see example above).
☕ Support & Future R&D
If the YMQ-Compiler builds saved your context window from collapsing or optimized your active development cycle speeds, consider buying a coffee to fund further low-level optimization research. Your support keeps the server nodes baking future model scales!
The compiler pipeline automation engine, setup thresholds, and structural mapping rules are open-source. To view the implementation details or compile your own custom models natively using this profile layout, visit the official development hub:
