SmolLM2-135M-Instruct-Q4_K_M-GGUF – AI Model by jc-builds | AlphaNeural AI

You can deploy this model and start earning money today!

jc-builds/
SmolLM2-135M-Instruct-Q4_K_M-GGUF

like

0

HuggingFaceTB/SmolLM2-135M-Instruct

endpoints_compatible

Views

No views yet

SmolLM2-135M-Instruct Q4_K_M GGUF

This is a Q4_K_M quantized GGUF conversion of HuggingFaceTB/SmolLM2-135M-Instruct optimized for on-device inference with llama.cpp.

Model Details

Property	Value
Original Model	SmolLM2-135M-Instruct
Parameters	135 million
Quantization	Q4_K_M (4-bit, medium quality)
File Size	~101 MB
Context Window	8,192 tokens
Architecture	LLaMA-style transformer
Training Data	2 trillion tokens

Intended Use

This model is optimized for:

Mobile/Edge Deployment: Runs on ANY iOS device, even the oldest
llama.cpp Integration: Compatible with llama.cpp and its bindings
On-Device AI: Private, offline inference without cloud dependencies

Capabilities

Ultra-Tiny: Smallest model available, instant responses
Works Everywhere: Runs on any device with minimal resources
Basic Q&A: Good for simple chat and quick interactions
Minimal Battery Usage: Extremely efficient
Trained on 2T Tokens: Impressive capability for its tiny size

Usage with llama.cpp

./llama-cli -m SmolLM2-135M-Instruct.Q4_K_M.gguf -p "Your prompt here" -n 512

License

This model inherits the Apache 2.0 license from the original SmolLM2 model.

Attribution

Original Model: SmolLM2-135M-Instruct by Hugging Face
Quantization: jc-builds