Ornith 1.0 35B Q4_K_M, MoEspresso package
This repository holds a MoEspresso package, and MoEspresso is the only engine that serves it:
https://github.com/steadfastgaze/MoEspresso.
The weights are GGUF K-quant tensors repacked into safetensors shards, described tensor by tensor in a content-hashed manifest.
llama.cpp, transformers, and mlx-lm will not load these files.
Install and serving instructions live in the MoEspresso repository.
Quality
| check | scope | result |
|---|
| Reference next-token loss (lower is better) | Fixed 13,215-token code corpus | This package: 1.1965; bartowski Q4_K_M in llama.cpp: 1.1946; difference: +0.0019 (+0.16%). |
| Greedy token agreement | Six fixed prompts, up to 64 generated tokens each | Five prompts matched at every position (198/198 tokens); the free-form sixth first differed at token 8. |
| Fixed-answer questions | 15 independently verified answers | Initial run: 15/15; later fixed-seed runs: 14/15 and 15/15. |
| Mixed tasks | Four reasoning problems, three executable coding tasks, and two exact-match questions over a 35,729-token document | 9/9: reasoning 4/4, executable code 3/3, long-context exact match 2/2. All generated programs passed their hidden tests. |
The answer checks used fixed seeds and the model author's recommended sampling settings.
The numerical reference above is the source Q4_K_M GGUF, not the original bf16 model.
Speed
Measured on an Apple M3 Max with 128 GB of unified memory, q8 KV cache - MoEspresso engine.
| context | prefill | decode |
|---|
| 4K | 1107.6 t/s | 77.84 tok/s |
| 8K | | 72.57 tok/s |
| 37K | 826.4 t/s | 53.86 tok/s |
Longest context reached in testing: 160,965 tokens, grown in one session.
Recipe
The package is built from three inputs.
qwen_kquant_recipe_report.json sidecar records the full recipe, including input hashes and per-shape codec assignments. package_manifest.json records the sha256 of every file; after downloading, moespresso-verify checks the package against it in one command.
The routed experts ship byte for byte from bartowski's Q4_K_M file: gate and up projections in q4_k on all 40 layers, down projections in q4_k on 20 layers and q6_k on the other 20. Nothing in those tensors was re-quantized.
The 252 dense tensors take a different path. They are encoded fresh from the bf16 release under bartowski's imatrix, with a heavier codec mix: 94 tensors in q8_0, 96 in q6_k, 10 in q5_k, 52 in q4_k. Another 361 small tensors (norms, biases) pass through in f32.
Files
Six safetensors shards (19.91 GiB, 733 tensors) plus the sidecars the engine reads: package_manifest.json (per-file sha256 and per-tensor decisions), package_plan.json (the allocation the writer consumed), agentic_profile.json (validated tool-calling defaults), and expert_hotlist.json (cold-start expert prewarm ranking). Tokenizer files and the chat template are vendored from the original release.
Credits
This quantization is derived from bartowski's quantization work; the expert bytes in this package ship unchanged and the imatrix guiding the dense encoding as well. The model is Ornith 1.0 by deepreinforce-ai. Everything in the chain is MIT: the original release, bartowski's GGUFs, and this package.
