python llama.cpp/convert_hf_to_gguf.py ./ubs_autotest --outfile ubs_autotest-bf16.gguf --outtype bf16
llama-quantize ubs_autotest-bf16.gguf ubs_autotest-Q4_K_M.gguf Q4_K_M

Architecture qwen35moe requires a recent llama.cpp build.

Usage

llama.cpp:

llama-cli -m ubs_autotest-Q4_K_M.gguf -p "Hello"

Ollama:

ollama run hf.co/Catter58/ubs_autotest-4bit:Q4_K_M

Note

The source model is multimodal (image-text-to-text). This GGUF contains the text weights only; the vision encoder is not included.