⚠️ Preliminary release. Quantized from the bf16 GGUF master of keithnull/Qwen3.6-35B-A3B-REAM-192-heretic. The only validation performed so far is (a) Heretic's first-token KL + refusal-marker eval against its 100-prompt evaluation split during the optimization run, (b) a five-prompt qualitative chat spot-check on the bf16 weights, and (c) basic generation-coherence smoke tests on the resulting GGUFs. No perplexity, KL-divergence, or academic-benchmark numbers have been measured against this APEX recipe yet — the published APEX results are on the un-abliterated Qwen3.5-35B-A3B reference, not this model. Treat this release as a preview while those evaluations are pending. Card will be updated with hard numbers as they land.
Same size class as standard Q4_K_M, recommended daily driver. Q4_K (edge) + Q3_K (middle) routed experts + Q6_K shared + Q4_K attention + imatrix scales. APEX claims this beats Q4_K_M-class quants on perplexity AND HellaSwag at the same footprint.
The "I-" prefix in APEX nomenclature means "imatrix-calibrated" — it changes the scale values stored in K-quant blocks but does not change the tensor formats.
A note on filenames
APEX GGUFs are mixed-precision — every file contains multiple llama.cpp quant types simultaneously (e.g. Q6_K for edge layers, Q5_K for near-edge, IQ4_XS for middle-layer routed experts, Q8_0 for shared experts, etc). HuggingFace's hardware-compatibility widget and various model-listing surfaces auto-detect a file's quantization by scanning the filename for a recognized K-quant or IQ-quant token (Q4_K_M, Q3_K_S, IQ4_XS, etc). Because no single token is strictly correct for a mixed-precision file, the parser would otherwise show "Unknown" / "Cannot determine variant" and the model wouldn't surface on hardware-filtered model lists.
To work around this, each file in this repo has a parser-friendly K-quant suffix appended that matches its dominant middle-layer expert quantization (since middle layers comprise the bulk of routed-expert parameters and most strongly determine the file's effective BPW). For example: APEX I-Compact uses Q4_K-edge + Q3_K-middle experts, so its file is named ...-APEX-ICompact-Q3_K_L.gguf. The Q3_K_L tag isn't exactly the file's contents — the actual tensor types are mixed — but it's the closest single recognizable token for HF's parser, and the BPW lands close to Q3_K_L's 4.27.
Pay attention to the APEX tier name (Mini / Compact / I-Compact / I-Balanced / I-Quality) for the real description of what's inside each file. Treatt the "Q_#" suffix purely as a UX hint for HF's display layer.
Vision support
These GGUFs are text-only weights, but the underlying REAM-192 architecture preserves the Qwen3.6-VL vision tower. Pairing with the standard Qwen3.6-VL projector enables image input via llama-server's OpenAI-compatible chat-completions endpoint.
The compatible projector is mirrored here for convenience:
Both REAM (this model) and REAP (atbender's source) preserve the unmodified Qwen3.6-VL vision encoder, hidden-state dimension, and tokenizer — neither method touches the vision tower or the input-embedding layer. The mmproj's image-token embeddings land in the same hidden-state space the LM was originally trained against. The APEX recipe further keeps shared experts and attention layers at high precision (Q6–Q8) across every tier, so vision-LM alignment survives even at the most aggressive (Mini-IQ2_S) routed-expert quantization.
This is a cross-recipe pairing without any vision-specific fine-tuning. A REAM-native projector would likely improve quality further, but is not yet released.
Validation (2026-05-12, n=1, qualitative)
Single-rater smoke test on a 590×904 JPEG product photo of a Canon PowerShot G5, running APEX Mini-IQ2_S + this projector on Apple M3 Metal:
Correctly identified the camera (Canon PowerShot G5) and attached flash (Canon Speedlite 420EX).
Read fine print: 5.0 MEGA PIXELS, CANON ZOOM LENS 7.2-28.8mm 1:2.0-3.0.
Noted the CENEO watermark across the middle of the image.
~40 s end-to-end via an agent harness (pi-mono) on the 10 GB Mini quant.
No quantitative VLM benchmarks (MMMU, MM-Vet, OCR-Bench, etc.) have been run against this pairing yet. The above is a smoke test, not a result. The same probe was previously validated against the REAP-26B-A3B Q4_K_M native pairing on 2026-05-01 — REAM Mini matches REAP-native on this image.
Then POST OpenAI-compatible chat completions with an image_url content block (base64 data URI, data:image/jpeg;base64,…). Confirmed working on JPEG and PNG; WebP is rejected by llama-server's mtmd loader at the time of writing — convert to JPEG/PNG first.
For grounding-heavy tasks, the upstream Qwen-VL guidance applies: add --image-min-tokens 1024 if accuracy on text reading or bounding-box-style queries drops.
Refusal and capability metrics (inherited from source bf16)
These numbers are from the Heretic optimization run on the bf16 weights, prior to APEX quantization. APEX-specific refusal/KL deltas (whether Q3-tier or IQ-tier middle layers change the refusal-suppression effectiveness) have not been measured. We do not expect significant change but it is on the validation TODO list.
Quantization recipe
Source: Qwen3.6-35B-A3B-REAM-192-heretic-bf16.gguf (51 GB / 16.01 BPW), the bf16 master from the standard GGUF repo.
Imatrix calibration corpus: bartowski's calibration_datav3.txt. APEX's own pipeline references a calibration_v1.2.txt that isn't bundled in the repo; we used the de-facto community standard instead. Quality difference is reportedly small per APEX's own ablations.
Imatrix generation: 32 chunks × 2048 ctx (corpus length capped the chunk count), all 40 transformer blocks GPU-resident on H100.
Profile-to-config mapping: APEX --profile compact / i-compact / i-balanced / i-quality / mini (the qwen36_heretic_*.txt configs are byte-identical to qwen36_35b_*.txt — no heretic-specific tuning beyond the layer-gradient strategy that's universal across the Qwen3.5/3.6 family).
Quick start
Inference (any APEX tier)
bash
1./llama-cli \2 -m Qwen3.6-35B-A3B-REAM-192-heretic-APEX-ICompact-Q3_K_L.gguf \3 -p "Write a Python function that returns the nth Fibonacci number."\4 -n 1000 --temp 0.7 --n-gpu-layers 999
(Use -n 1000 minimum since Qwen3.6's default thinking mode emits long [Start thinking] ... [End thinking] blocks before the answer. Pass /no_think in the prompt to skip thinking for quick sanity checks.)
Preliminary local code-gen smoke test (ICompact only)
A small-scale single-run shakedown of the ICompact-Q3_K_L tier against a real coding workload, run via llama-server from TheTom-llama-cpp-turboquant with KV cache q8_0/turbo3, ngram-simple speculative decode, free-form thinking with --reasoning-budget 1500, and ctx 32768. Date: 2026-05-09.
This is not a benchmark. Sample sizes are tiny (2 standalone tasks + 3 agentic tasks), n=1 per task, single rater. It exists to confirm the recipe survives an agentic workload without obvious breakage, ahead of the larger evals on the validation TODO list.
2× standalone TypeScript code-gen (mulberry32 SeededRng + LRUCache<K,V>) | 33/33 vitest tests pass on the produced sources |
3× agentic tasks against a ~360-test repo (vitest sanity / Box-Muller nextGaussian + stats test / multi-file readonly name field on post-processing Effect classes + tsc --noEmit) | 3/3 clean diffs; vitest pass; tsc --noEmit clean |
Pre-bench probes terminated cleanly (finish_reason=stop, no V-quant attention loop). On the nextGaussian task the agent produced library-grade Box-Muller — caching the second sample of each pair and patching fork() to copy the cache — without being prompted for either detail. The test (10 000 samples; assert |mean| < 0.1 and |stddev − 1| < 0.1) passed first try.
What this can support saying: the I-Compact recipe runs end-to-end through Read/Grep/Edit/Bash + self-verify loops and produces diffs that pass vitest and tsc on the cases tried.
What it cannot support saying: anything about quality vs other APEX tiers (not benched here), vs vanilla REAM Q4_K_M on agentic loops (no comparison run yet), or about academic-eval scores (still deferred).
Per-task diffs, raw stdout/stderr, and the produced source snapshots are uploaded under bench/2026-05-09-icompact-smoke/ in this repo.
