This model is a fine-tuned Qwen2.5-3B-Instruct model specialized in detecting hallucinations in Retrieval-Augmented Generation (RAG) answers in both English and Arabic.

It acts as an LLM judge that determines whether an answer is fully supported by the retrieved context.

🎯 Task

Given:

Context (retrieved documents)
Question
Answer (generated by an LLM)

The model outputs:


PASS  → Answer is grounded in the context
FAIL  → Answer contains hallucinations or unsupported claims

🏗 Base Model

Model: Qwen/Qwen2.5-3B-Instruct
Fine-tuning: LoRA → merged into base weights
Languages: English + Arabic
Training Objective: Hallucination classification for RAG systems

⚙️ Inference Format

Prompt Template:


You are a system that detects hallucinations in RAG answers.

Decide whether the answer is fully supported by the context.
Reply with only one word: PASS or FAIL.

[CONTEXT]
{context}

[QUESTION]
{question}

[ANSWER]
{answer}

Judgment:

💻 Example (Python)

python

import requests

API_URL = "YOUR_HF_ENDPOINT_URL"
HF_TOKEN = "hf_xxx"

headers = {
    "Authorization": f"Bearer {HF_TOKEN}",
    "Content-Type": "application/json"
}

def judge(context, question, answer):
    prompt = f"""You are a system that detects hallucinations in RAG answers.

Decide whether the answer is fully supported by the context.
Reply with only one word: PASS or FAIL.

[CONTEXT]
{context}

[QUESTION]
{question}

[ANSWER]
{answer}

Judgment:"""

    payload = {
        "inputs": prompt,
        "parameters": {
            "max_new_tokens": 5,
            "do_sample": False,
            "temperature": 0.0
        }
    }

    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()[0]["generated_text"]

📊 Training Data

The model was trained on a labeled dataset of RAG examples from HaluBench:

Field	Description
Context	Retrieved passages
Question	User query
Answer	LLM-generated answer
Label	PASS / FAIL

The dataset is balanced between grounded and hallucinated answers.

🚀 Intended Use

✅ Evaluating RAG pipelines ✅ LLM-as-a-judge research ✅ Automatic hallucination detection ✅ Benchmarking grounding quality

❌ Not for open-ended chat ❌ Not a knowledge source

🧩 Deployment

Optimized for low-latency inference using Hugging Face Text Generation Inference (TGI) endpoints.

👤 Author

Ahmed Abolfadl B.Sc. Computer Science & Engineering — German University in Cairo Research focus: ML, AI, Data Science

📅 Model Version

Uploaded on: 2026-01-26

aabolfadl/balash-faty-2

🧠 Balash Faty — RAG Hallucination Judge (EN/AR)