Qwen3-4B-Q4_K_M-GGUF – AI Model by jc-builds | AlphaNeural AI

You can deploy this model and start earning money today!

jc-builds/
Qwen3-4B-Q4_K_M-GGUF

like

0

endpoints_compatible

Views

No views yet

Qwen3-4B Q4_K_M GGUF

This is a Q4_K_M quantized GGUF conversion of Qwen/Qwen3-4B optimized for on-device inference with llama.cpp.

Model Details

Property	Value
Original Model	Qwen3-4B
Parameters	4 billion
Quantization	Q4_K_M (4-bit, medium quality)
File Size	~2.5 GB
Context Window	32,768 tokens
Architecture	Qwen3 (RoPE, SwiGLU, RMSNorm, GQA)

Intended Use

This model is optimized for:

Mobile/Edge Deployment: Runs efficiently on iOS devices with 8GB+ RAM (iPhone 15 Pro, 16, etc.)
llama.cpp Integration: Compatible with llama.cpp and its bindings
On-Device AI: Private, offline inference without cloud dependencies

Capabilities

Complex reasoning and analysis
Code generation and debugging
Mathematical problem solving
Creative writing
Multilingual support (29+ languages)
"Thinking mode" for step-by-step reasoning

Usage with llama.cpp

./llama-cli -m Qwen3-4B-Q4_K_M.gguf -p "Your prompt here" -n 512

License

This model inherits the Apache 2.0 license from the original Qwen3-4B model.

Attribution

Original Model: Qwen3-4B by Qwen Team, Alibaba Cloud
Quantization: jc-builds