File	Size	Use
`nix-reviewer-1.5b-Q4_K_M.gguf`	~986 MB	Recommended — fits 16 GB xnode CPU inference, ~5–10 s/review
`nix-reviewer-1.5b-f16.gguf`	~3.1 GB	Full precision; for local GPU or quantization experiments

Use via Ollama

bash

ollama pull hf.co/OpenxAILabs/nix-reviewer-1.5b-GGUF:Q4_K_M
ollama run hf.co/OpenxAILabs/nix-reviewer-1.5b-GGUF:Q4_K_M '{ pkgs, ... }:
{
  environment.systemPackages = with pkgs; [ vim vvim ];
}'

Expected output:

json

[{"line": 4, "severity": "error",
  "message": "`vvim` is not a nixpkgs attribute — did you mean `vim`?"}]

Use via llama.cpp

bash

./llama-cli -m nix-reviewer-1.5b-Q4_K_M.gguf \
  --system "You are nix-assistant. Review the Nix config and output ONLY a JSON array: [{\"line\":int,\"severity\":\"error\"|\"warning\"|\"hint\",\"message\":str}]" \
  -p "{ pkgs, ... }: { environment.systemPackages = with pkgs; [ vim vvim ]; }"

System prompt (important)

The model expects this exact system prompt to emit the JSON contract:

You are nix-assistant. Review the Nix config and output ONLY a JSON array: [{"line":int,"severity":"error"|"warning"|"hint","message":str}]

Other prompts will work but output shape is less reliable.

Quantization details

Source: merged fp16 weights (LoRA adapter merged into base)
Converter: llama.cpp's convert_hf_to_gguf.py
Q4_K_M quantization: performed by Ollama's ollama create -q Q4_K_M
Expected quality drift vs. fp16: <3% on our benchmark (precise measurement in v0.2 release notes)

License

Apache-2.0. See LICENSE + NOTICE on the main model repo.