Ornith-1.0-35B oQ8 Text-Only (Optimized for Apple Silicon)
This repository contains a custom-quantized, text-only configuration of the Ornith-1.0-35B Mixture-of-Experts (MoE) model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.
🎯 Why This Was Created
Ornith-1.0-35B is a state-of-the-art, self-improving MoE model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories.
This specific build was converted using oMLX v0.4.5.dev1 to address long-context deployment constraints on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):
The MoE High-Fidelity Precision: As a 35B Mixture-of-Experts model, Ornith-1.0-35B strikes an incredible balance between deep reasoning and high throughput. Quantizing it to oQ8 (8-bit) provides near-lossless code syntax retention and complex instruction adherence, matching the original model's quality while still saving valuable Unified Memory to support deep context windows during complex multi-file repository tasks.
Overcoming the 128k Context Wall: Traditional backends often choke or suffer severe latency degradation when context sizes scale out. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to fluidly ingest giant codebases.
Prefill Speedup via float16: While the base weights are distributed in bfloat16, this build explicitly targets Apple Silicon hardware by using float16 for non-quantized weights, unlocking a ~20% faster prefill speed on M1/M2 Max chips.
MTP Note: Unlike some Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (mtp.*) headers. As a result, native MTP decoding is not available for this model.
🚀 Key Differences
Feature / Attribute
Standard Ornith-1.0-35B
This Custom Build (oQ8-fp16)
Native MTP Heads
Not present in base architecture
Not Available (No base MTP tensors)
Vision Model (VLM)
N/A (Text-only coding agent)
Stripped/Text-Only
Quantization Method
Standard Uniform / HF / Unsloth
oQ8 (Dynamic mixed-precision calibration)
Non-Quant Weight DType
bfloat16
float16 (~20% faster prefill on M1/M2 Silicon)
💻 Hardware & RAM Recommendations
Mac Hardware Configuration
RAM Recommendation
Status / Performance Expectation
M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)
48GB Unified Memory
Supported — Fits under 48GB, but leaves limited headroom for massive context windows. 32GB/36GB variations are not recommended.
M1 / M2 / M3 / M4 Max / Ultra
64GB Unified Memory
Recommended — Fits comfortably with ample space for moderate context lengths (~64k-128k).
M2 Max / M3 Max / Ultra (96GB / 128GB / 192GB)
96GB+ Unified Memory
Optimal / Best Experience — Highly recommended configuration. Allows operating at the full context boundary (262k) with zero system memory pressure.
🛠️ Quantization Settings
This model was quantized using oMLX v0.4.5.dev1 with the following specification:
Source Model:deepreinforce-ai/Ornith-1.0-35B
Sensitivity Model:None
oQ Level:oQ8
Text Only:Enabled
Preserve MTP weights:Disabled (Not present in source architecture)
Non-quant weight dtype:float16
⚙️ Optimized oMLX Settings (v0.4.5)
To seamlessly route this model through agentic development workspaces like OpenCode, apply the following server specifications in your oMLX dashboard:
Model Basic Settings
Reasoning Parser:qwen3(Isolates the <think> ... </think> blocks securely away from IDE syntax parsers)
Tool Call Parser:qwen3_xml
CTX Window:262,144
Max Tokens:32,768
Temperature:0.6(Use 1.0 if attempting to perfectly replicate official benchmark environments)
Burst Decode:Aggressive (Coalesces tokens for maximized typing speeds)
Initial Cache Blocks:256
SSE Keepalive Mode:Chunk
🩹 Repair Script (repair_moe_experts.py)
The weights in this repository are already fixed and load correctly — you do not need to run this for normal use.
It's included only for anyone who cached an earlier broken download of this repo, or who runs into the same issue when quantizing another MoE model with a similar per-expert weight layout. Symptom: loading fails with an error like:
Received 92160 parameters not in model:
language_model.model.layers.0.mlp.experts.0.down_proj.biases, ...
Cause: Ornith-1.0-35B ships its MoE experts as separate per-expert tensors (mlp.experts.<0-255>.{gate,up,down}_proj), but mlx-lm's qwen3_5_moe loader only understands the fused switch_mlp layout — so an affected build never quantizes them into a loadable shape, and its config.json also carries stale pre-sanitize key names for the per-path quantization overrides.
Requires only mlx (any environment with mlx-lm/omlx installed has it). It streams the weights shard-by-shard, stacks the per-expert tensors into switch_mlp (bitwise-identical values — no requantization), and fixes the config.json key names. The repair happens in place: new shards are written alongside the originals, verified bitwise against the source tensors, and only then swapped in — a failure at any point leaves the original model untouched. It needs free disk roughly equal to the model size while running, and is safe to re-run (exits early on an already-repaired model).
🌡️ Thermal Optimization Notice
Sustained execution across massive context windows heavily taxes the Apple Silicon GPU/CPU complexes, causing rapid heat buildup. Because Apple's default fan profiles emphasize near-silent operation, they delay ramping up system fans until thermal throttling has already begun to affect generation tokens-per-second (TPS).
To protect performance integrity during prolonged coding sessions, it is highly recommended to run a custom fan utility to enforce proactive, aggressive cooling curves:
