Views
No views yet
compressed-tensors format. The lm_head layer is kept in FP16 to preserve output quality.| Variant | Repo | Description |
|---|---|---|
| Merged FP16 | NightPrince/Qwen3-4B-Islamic-Arabic | Canonical merged model, FP16, ~7.6 GB — drop-in for transformers or vLLM |
| LoRA Adapter | NightPrince/Qwen3-4B-Islamic-Arabic-LoRA | PEFT adapter only, 264 MB — apply on top of Qwen/Qwen3-4B |
| INT4 Quantized (this model) | NightPrince/Qwen3-4B-Islamic-Arabic-INT4 | W4A16 compressed-tensors for fast vLLM serving, 2.5 GB |
| MLX 4-bit | NightPrince/Qwen3-4B-Islamic-Arabic-mlx-4Bit | Apple Silicon / MLX — native Mac inference, 4-bit quantized |
| GGUF | NightPrince/Qwen3-4B-Islamic-Arabic-GGUF | llama.cpp / Ollama / LM Studio — Q4_K_M (2.3 GB), Q8_0 (4.0 GB), F16 (7.5 GB) |
| Dataset | NightPrince/islamic-arabic-qa | 17,944 train / 2,101 val / 1,042 test — Islamic Arabic Q&A pairs |
1# Install vLLM
2pip install vllm
3
4# Serve the INT4 model — fits on a single 11 GB GPU
5vllm serve NightPrince/Qwen3-4B-Islamic-Arabic-INT4 \
6 --quantization compressed-tensors \
7 --dtype float16 \
8 --enforce-eager \
9 --max-model-len 4096 \
10 --port 8000--enforce-eagerdisables CUDA graph capture, which is recommended for compressed-tensors quantized models to ensure compatibility. You may omit it on newer vLLM versions if throughput matters more.
1from openai import OpenAI
2
3client = OpenAI(base_url="http://localhost:8000/v1", api_key="token-abc123")
4
5SYSTEM_PROMPT = (
6 "أنت مساعد عالم إسلامي متخصص. "
7 "أجب على الأسئلة بدقة استناداً إلى القرآن الكريم والسنة النبوية والفقه الإسلامي الكلاسيكي. "
8 "استشهد بالمصادر حيثما أمكن. كن موجزاً لكن شاملاً."
9)
10
11response = client.chat.completions.create(
12 model="NightPrince/Qwen3-4B-Islamic-Arabic-INT4",
13 messages=[
14 {"role": "system", "content": SYSTEM_PROMPT},
15 {"role": "user", "content": "ما هي شروط صحة عقد البيع في الفقه الإسلامي؟"},
16 ],
17 max_tokens=512,
18 temperature=0.7,
19 top_p=0.9,
20)
21print(response.choices[0].message.content)1# Two GPUs for higher throughput
2vllm serve NightPrince/Qwen3-4B-Islamic-Arabic-INT4 \
3 --quantization compressed-tensors \
4 --dtype float16 \
5 --enforce-eager \
6 --tensor-parallel-size 2 \
7 --max-model-len 8192 \
8 --port 8000| Property | Value |
|---|---|
| Quantization scheme | W4A16 (4-bit weights, 16-bit activations) |
| Format | compressed-tensors (vLLM native) |
| Quantization tool | llm-compressor |
| lm_head | Kept in FP16 |
| Quantized size | ~2.5 GB |
| Source model | NightPrince/Qwen3-4B-Islamic-Arabic (FP16, 7.6 GB) |
| Configuration | VRAM Required |
|---|---|
| Single GPU (INT4) | ~3–4 GB (fits on 8 GB GPU) |
| Single GPU + long context (8K) | ~6–8 GB |
| Recommended minimum | 1× 8 GB GPU |
Quantized with llm-compressor W4A16 scheme. Thelm_headlayer is kept in FP16 to preserve logit quality. This model is designed for vLLM with--quantization compressed-tensorsand is not compatible withtransformersquantization backends (GPTQ, AWQ). For CPU or llama.cpp inference, use the GGUF variant instead.
1@misc{alnwsany2026qwen3islamicarbic,
2 author = {Yahya Alnwsany},
3 title = {Qwen3-4B-Islamic-Arabic: QLoRA Fine-Tuning of Qwen3-4B on Islamic Arabic Q\&A},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/NightPrince/Qwen3-4B-Islamic-Arabic}},
6 note = {Base model: Qwen/Qwen3-4B. Dataset: NightPrince/islamic-arabic-qa.}
7}