You can deploy this model and start earning money today!

roshan-soni/
qwen-3.5-2b-q4-k-m

Name: qwen-3.5-2b-q4-k-m
Availability: InStock

Views

No views yet

Qwen3.5-2B Q4_K_M GGUF

This repository contains a GGUF quantized version of Qwen3.5-2B optimized for local inference with llama.cpp and compatible applications.

Base Model: Qwen3.5-2B
Format: GGUF
Quantization: Q4_K_M
Intended Use: Local inference, experimentation, educational projects, and edge deployment

The model was converted from the original Hugging Face checkpoint to GGUF format and quantized using llama.cpp's Q4_K_M quantization scheme.

Q4_K_M provides an excellent balance between:

making it suitable for laptops, desktops, and mobile devices.

File	Description
qwen-q4_k_m.gguf	Q4_K_M quantized GGUF model

bash

./llama-cli \
  -m qwen-q4_k_m.gguf \
  -p "Explain machine learning in simple terms."

This repository contains a quantized derivative of the original Qwen3.5-2B model.

Please refer to the original model repository for training details, benchmarks, limitations, and licensing information.

This repository only provides a GGUF quantization of the original model. All credit for model training and development belongs to the Qwen team.