granite-4.0-1b-cn-gguf

Files

• granite-4.0-1b-cn-f32.gguf: merged F32 GGUF exported from the local MLX LoRA run.
• granite-4.0-1b-cn-q8_0.gguf: merged Q8_0 GGUF quantized from the merged model.

Source run

• Training output: mlx/output/granite_4_0_1b_tc_trs_1epoch_alltargets_8k_bs2/
• Dataset: mlx/data/granite_4_0_1b_tc_trs/
• Recipe: LoRA all eligible targets, rank=8, scale=16, batch_size=2, iters=530, max_seq_length=8192
• Merge path:
  • export LoRA GGUF from a transposed PEFT-style adapter shim
  • merge with local Granite base GGUF via llama-export-lora
  • quantize merged model to Q8_0 with llama-quantize

Runtime status

• Direct native llama.cpp CLI smoke tests succeeded for both merged artifacts.
• The exact local Granite base GGUF also succeeds in direct native llama.cpp.

Important disclaimer

• This Granite GGUF does not currently work in LM Studio on our test machine.
• LM Studio returned @@@@@@@@... with empty tool_calls for:
  • the exact local base Granite Q8_0 GGUF
  • this merged Q8_0 GGUF
  • this merged F32 GGUF
  • unsloth/granite-4.0-1b q8 GGUF
• Treat LM Studio Granite support as currently broken or incompatible here.
• Recommended runtime for this release: direct native llama.cpp.

Eval note

• Earlier trusted native Granite evals were run on the corrected direct llama.cpp path using Granite's own tokenizer chat template and XML tool-call parsing fixes.
• Later local llama-cpp-python and LM Studio runtime checks diverged and should not be treated as equivalent validated Granite runtimes.