You can deploy this model and start earning money today!

KavinduHansaka/
granite-4.0-h-tiny-gguf

Name: granite-4.0-h-tiny-gguf
Availability: InStock

gguf

text-generation

ibm-granite/granite-4.0-h-tiny

Views

No views yet

granite-4.0-h-tiny-gguf

🚀 GGUF-quantized release of IBM Granite 4.0 H Tiny, converted for efficient local inference.

This repository provides GGUF-format versions of the original
ibm-granite/granite-4.0-h-tiny model, enabling fast execution with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

Model Overview

Base Model: IBM Granite 4.0 H Tiny
Model Type: Decoder-only Transformer
Format: GGUF
Pipeline: Text Generation
Language: English

Designed for lightweight, low-latency inference while preserving strong instruction-following capabilities.

What This Repository Contains

✅ GGUF-converted model weights
✅ Quantized for CPU / low-VRAM environments
✅ Ready for local & edge inference
❌ No fine-tuning or weight modification (format conversion only)

Usage

Ollama

Create a Modelfile:

FROM granite-4.0-h-tiny.gguf

Run:

bash

ollama create granite-tiny -f Modelfile
ollama run granite-tiny

LM Studio / GUI Tools

Open LM Studio
Load the .gguf file
Select a llama.cpp backend
Start inference

Quantization Notes

GGUF quantization provides:

Reduced memory usage
Faster load times
Compatibility with CPU-only systems

Recommended usage:

Q4 / Q5: Best balance of speed & quality
Q8: Higher quality, more memory

Intended Use Cases

Text generation
Review & sentiment analysis
QA automation pipelines
Agentic systems (RAG, MCP, LangGraph, CrewAI)
Offline / embedded AI applications

Attribution

Original Model:
IBM Granite Team
https://huggingface.co/ibm-granite/granite-4.0-h-tiny
GGUF Conversion & Packaging:
Kavindu Hansaka Jayasinghe

License

This repository follows the MIT license, consistent with the original model.

Please review the base model license before commercial usage.

Disclaimer

This is an unofficial GGUF conversion.
IBM is not affiliated with or responsible for this release.

KavinduHansaka/granite-4.0-h-tiny-gguf