This repository contains a LoRA fine-tuned version of google/gemma-7b-it for generating English multiple-choice questions that follow the official Egyptian Ministry of Education examination style.

Model Information

Base Model: google/gemma-7b-it
Fine-tuning: LoRA (PEFT)
Framework: Unsloth
Language: English
Task: Instruction Fine-Tuning

Training Configuration

Parameter	Value
LoRA Rank	16
LoRA Alpha	32
Dropout	0.05
Learning Rate	1e-4
Epochs	3
Max Sequence Length	2048

Example

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "tokhey/egyptian-mcq-generator-gemma-7b"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Generate 10 official Egyptian Ministry English MCQs about Past Perfect. Return only valid JSON."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1500,
    temperature=0.7,
)

print(
    tokenizer.decode(
        outputs[0][inputs.input_ids.shape[1]:],
        skip_special_tokens=True,
    )
)

Output Format

json

{
    "statement": "...",
    "correct_answer": "...",
    "plausible_distractors": [
        "...",
        "...",
        "..."
    ],
    "explanation": "..."
}

License

This model is derived from google/gemma-7b-it and follows the Gemma license.