Qwen3.5-9B — GGUF (IQ4_NL)
📊 Performance Metrics
- Hardware: Intel(R) Xeon(R) CPU @ 2.20GHz (4 vCPUs)
- Size: 5.07 GB
- Speed (Generation): 2.38 tokens/sec
- Speed (Prompt): 5.13 tokens/sec
- KV Cache Usage: 0.0143 GB
- Quantization: IQ4_NL
🔷 Model Overview
This repository contains a GGUF quantized version of:
- Base Model: Qwen3.5-9B
- Format: GGUF (optimized for llama.cpp inference)
- Precision: IQ4_NL
- Efficiency Score: 0.4695 (TPS/GB)
GGUF format provides:
- Fast loading via memory mapping
- Single-file model distribution
- Cross-platform compatibility
- Efficient inference with llama.cpp
📦 Files
| File | Description |
|---|
Qwen3.5-9B-IQ4_NL.gguf | Quantized GGUF model file |
⚙️ Technical Details
| Parameter | Value |
|---|
| Architecture | Qwen3.5-9B |
| Format | GGUF |
| Precision | IQ4_NL |
| Runtime | llama.cpp |
| Benchmark Hardware | Intel(R) Xeon(R) CPU @ 2.20GHz (4 vCPUs) |
| Context Latency | 69.85s |
| Memory (KV) | 0.0143 GB |
⚡ Why GGUF?
GGUF is designed for efficient inference:
- Optimized for llama.cpp
- Supports CPU and GPU inference
- Single-file deployment
- Memory-mapped loading for speed
- Ideal for edge / local environments
⚠️ License & Usage
This is a converted derivative model.
- You must comply with the original model license of Qwen3.5-9B
- This is not an official release
- No additional rights are granted
- Original ownership remains with the base model creator
🚀 Quick Start (llama.cpp)
./llama-cli -m Qwen3.5-9B-IQ4_NL.gguf -p "Explain AI simply"
