qwen36-reap-2k-mlx-q8 is an MLX-formatted checkpoint optimized for Apple silicon, published by groxaxo.
It is intended for open-source evaluation, reproducible experimentation, and compatible local or
hosted inference workflows. The wording below is deliberately limited to what can be verified
from this repository's metadata and artifacts.
Use an up-to-date MLX-compatible runtime on Apple silicon and point it at this repository:
mlx_lm.generate --model groxaxo/qwen36-reap-2k-mlx-q8 --prompt "Write a concise technical summary."
Embedding and audio repositories may require the task-specific MLX package documented by the
upstream project.
Compatibility and responsible use
Use a runtime that explicitly supports this format, architecture, and modality.
Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
Review the source model card and license before redistribution or deployment.
Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.
Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.
Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for
testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.
The same model from the story — the one that grew up on the filtered public
internet, pruned but whole, with one chapter of structured-reasoning traces
folded into its final layer — here pressed into a lighter form.
Where the bf16 release carries every weight at full precision, this is the
8-bit reading: the text and mixture-of-experts weights quantized to 8-bit
affine (group size 64), the embeddings and output projection kept at full
bf16 so the vocabulary stays sharp, and the vision tower preserved exactly as
in the source — bit for bit, untouched by the compression. It thinks the same
thoughts; it just carries them in a smaller satchel.
Built by fusing the rank-four LoRA into the base and quantizing in a single
pass, so no precision was lost to an intermediate round-trip. Higher fidelity
than the 4-bit release, lighter than full precision — for when you want the
model's best foot forward without paying for the whole library.
Technical details
Base:RangerX-Qwen3.6-35B-REAP-Pruned-ratio-0.5 — Qwen3.6 35B MoE (128 experts, 8 active), REAP-pruned at ratio 0.5, multimodal (text + vision)
Precision: text/MoE weights 8-bit affine (group_size 64); embed_tokens and lm_head kept at bf16; vision tower bf16 (preserved bit-identical from source)
~8.9 bits per weight overall
Adapter: LoRA, rank 4, scale 8.0, last full-attention layer only (layer 39), 300 iterations at lr 1e-5
Merge: adapter fused via mlx_lmfuse(), then quantized in the same pass; vision tower re-attached verbatim
Loads strictly under mlx_lm 0.31.3 with zero missing or unexpected weights (full multimodal: text + vision)
Built via the same fuse + quantize pipeline as the verified-working Q4 release
Note on generation testing
Generation was not verified on the build machine (24 GB unified memory): the
Q8 weights (~20 GB) plus vision tower exceed available GPU memory at runtime.
The model strict-loads cleanly and the weights are produced by the same proven
fuse + quantize path as the Q4 (which generates correctly). Verify generation
on a host with ≥ 32 GB unified memory.
