Property	Value
Base model	google/gemma-3-4b-it
Parameters	4B
Quantization	Q4_K_M (4-bit, medium-high quality)
Approximate size	~2.4 GB
Quantization tool	llama.cpp
Primary use case	On-device Android CV analysis (CVPilot)

Why Q4_K_M?

Q4_K_M uses a mixed quantization strategy (some layers at Q4, key layers at Q6) which gives significantly better output quality than naive INT4 while staying at roughly the same file size.

Usage

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="alefalab/gemma-3-4b-it-Q4_K_M-GGUF",
    filename="gemma-3-4b-it-Q4_K_M.gguf",
    n_ctx=4096,
)

response = llm.create_chat_completion(messages=[
    {"role": "user", "content": "Analyze this CV for a Software Engineer role: ..."}
])
print(response["choices"][0]["message"]["content"])

Android Integration

This model is used in the CVPilot Android app via llama.cpp Android bindings. The GGUF format is inference-engine agnostic — compatible with llama.cpp, llama-android, Ollama, LM Studio, and any other GGUF-compatible runtime.

License

This model inherits the Gemma Terms of Use.