GGUF quantizations of the official Motif-Technologies/Motif-3 checkpoint for the Motif-3-enabled llama.cpp runtime.

These files were converted directly from the official BF16 weights at source revision 1695f5aa6d97cccc8623a03f93c9c7d5fbb14d45. They are not requantizations of an FP8 or another reduced-precision checkpoint.

Motif-3 is a 314B-total / 13.2B-active decoder-only MoE model. The upstream model card and MIT license remain the authoritative sources for architecture details, intended use, and limitations.

Each variant is split into 14 GGUF shards. Download all shards for one variant and pass only its 00001-of-00014.gguf file to llama.cpp; the remaining shards are discovered automatically.

Q8_0 provides the closest weight reconstruction. Q6_K and Q5_K_M are the practical high-fidelity choices when their memory footprint is acceptable. Lower-bit variants trade progressively more reconstruction precision for capacity.

The comparison uses deterministic, evenly spaced, complete GGML rows sampled from every tensor changed from BF16 by quantization: 2 rows from each of 314 quantized tensors, or 2,030,592 scalar values per variant. Quantized rows are reconstructed with the reference dequantizers in the same llama.cpp GGUF implementation and compared directly with the corresponding official-source BF16 rows. Global cosine is accumulated over all sampled values; relative RMSE is ||BF16 - dequantized||₂ / ||BF16||₂.

These are weight-reconstruction measurements, not task benchmark scores. The reproducible measurement script is included as compare_cosine.py.

The label describes the target type for the large quantizable tensors, not a promise that every tensor uses that type. The Motif-3 converter deliberately preserves sensitive GDLA attention tensors in BF16 and control/norm tensors in F32. Each variant contains 2,162 tensors in total:

The current Motif-3 GGUF converter covers the main causal model but does not export the upstream MTP auxiliary head. Consequently, the upstream self-speculative MTP path is not available in these GGUF files.

Motif-3 support is not yet part of ordinary llama.cpp releases used for this conversion. Use the Motif3 branch at the pinned commit below and apply the included tokenizer patch. The patch adds the exact Motif-3 pre-tokenizer and was checked to apply cleanly to that commit.

Example server command for a multi-GPU machine:

--fit on lets llama.cpp leave layers in system memory when the model plus runtime buffers do not fit fully in aggregate VRAM. Q8_0 was loaded and served on the four-H100 system below with automatic host offload.

The BF16-to-GGUF conversion and CPU quantization used local NVMe storage and high parallel CPU utilization. CUDA was used for the runtime build and inference validation; standard llama.cpp GGUF quantization itself is CPU-side.

The model architecture and original weights are by Motif Technologies. The source model is distributed under the MIT license. This repository contains format-converted and quantized derivatives under the same license terms; consult the upstream repository for complete model documentation.