Serpen/

Motif-GGUF

This repo contains quants of Motif 3 Beta. More specifically a converted BF16 GGUF for creating quantizations and a somewhat optimized Q5_K_M. These ggufs WILL NOT load on mainline llama.cpp.

You have to build this pr. The PR now supports MLA. This needs reconverted GGUFs.

Alternatively if you don't wish to redownload, you can use llama-tensor-split --merge to merge your already downloaded split ggufs, then use the motif3_add_mla_tensors.py script which performs the tensors surgery on the merged single file gguf, which will then support MLA.

Usage: python3 motif3_add_mla_tensors.py merged.gguf out.gguf