You can deploy this model and start earning money today!

Avtrkrb/
granite-claude-h-350m-GGUF

Name: granite-claude-h-350m-GGUF
Availability: InStock

Views

No views yet

granite-claude-h-350m-GGUF

GGUF quantizations of:

Avtrkrb/granite-claude-h-350m

These files are intended for inference using:

Typical variants included:

Quant	Use Case
Q4_K_M	Best size / quality balance
Q5_K_M	Higher quality
Q6_K	Near-lossless for most use cases
Q8_0	Highest quality quantized version

Merged model:

Dataset:

bash

./llama-cli \
  -m granite-claude-h-350m-Q4_K_M.gguf \
  -p "Explain quantum tunneling."

For most users:

Q4_K_M

offers the best balance between:

This repository follows the licensing terms of the original Granite model.