A fine-tuned version of Qwen3-4B specifically designed for Islamic Q&A with chain-of-thought reasoning capabilities. This model can provide detailed explanations about Islamic concepts, jurisprudence, theology, and Quranic interpretations.

This qwen3 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model Features

Chain-of-Thought Reasoning: Shows detailed thinking process before providing answers
Islamic Knowledge: Specialized in Islamic concepts, jurisprudence, and theology
Flexible Output: Can enable/disable thinking process based on user preference
Streaming Support: Real-time response generation

Quick Start

Installation

pip install transformers torch soundfile ipython bitsandbytes

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
import torch

# Load the fine-tuned model
model_id = "Ellbendls/Qwen3-4b-Quran-LoRA-Fine-Tuned"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Ask a question
messages = [
    {"role" : "user", "content" : "What is Sabr?"},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True, # Must add for generation
    enable_thinking = True, # Disable or Enable thinking
)

from transformers import TextStreamer
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 256, # Increase for longer outputs!
    temperature = 0.7, top_p = 0.8, top_k = 20, # For non thinking
    streamer = TextStreamer(tokenizer, skip_prompt = True),
)

Parameters

enable_thinking = True - Shows the model's reasoning process
enable_thinking = False - Only shows the final answer
max_new_tokens - Controls response length (default: 256)
temperature - Controls randomness (0.1-1.0, default: 0.7)

Example Output

<think>
Sabr is a crucial concept in Islam that means patience, perseverance, and endurance in the face of difficulties. It is one of the essential pillars of faith, as emphasized in the Quran. Sabr is not just about enduring hardship but also about maintaining faith and trust in Allah's plan.
</think>

Sabr is a vital virtue in Islam that teaches patience and perseverance in the face of challenges.

Training Details

Training Method: LoRA (Low-Rank Adaptation) fine-tuning
Dataset: 857 examples from musaoc/Quran-reasoning-SFT
Training Steps: 30 (demonstration model)
LoRA Configuration:
- Rank: 32
- Alpha: 32
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Batch Size: 2 per device with gradient accumulation steps of 4
Learning Rate: 2e-4
Optimizer: adamw_8bit

Limitations

Religious Guidance: This model provides educational information about Islamic concepts but should not be considered as official religious guidance
Scholarly Consultation: For important religious matters, always consult qualified Islamic scholars
Training Data: Model responses are based on training data and may not cover all Islamic perspectives

Citation

bibtex

@misc{qwen3-4b-quran-lora,
  title={Qwen3-4B-Quran-LoRA-Fine-Tuned},
  author={Ellbendls},
  year={2025},
  howpublished={\url{https://huggingface.co/Ellbendls/Qwen3-4b-Quran-LoRA-Fine-Tuned}},
}

This model is designed for educational and research purposes. For authoritative Islamic guidance, please consult qualified scholars and authentic Islamic sources.