This model is a fine-tuned version of Gemma-3 trained on the RACE (ReAding Comprehension from Examinations) dataset using Unsloth.
It specializes in multiple-choice reading comprehension tasks that require reasoning and explanation.

The model was optimized using Generalized Reinforcement Policy Optimization (GRPO) with a custom reward function encouraging correct, confident, and concise answers in the form:

🧩 Model Details

Field	Description
Base Model	`Gemma-3`
Fine-tuning Framework	Unsloth
Task Type	Multiple-Choice Reading Comprehension
Dataset	RACE
Language	English
Reward Function	Custom GRPO (`mc_reward_grpo`)
License	Gemma License
Training Objective	Reinforcement Fine-tuning for reasoning clarity and correctness

⚙️ Reward Function

python

import re, math
LETTER_RE = re.compile(r'final(?:\s*answer)?\s*:\s*([A-D])', re.IGNORECASE)
CONF_RE = re.compile(r'(\d{1,3})\s*%')

CORRECT = 1.5
INCORRECT = -1.0
MALFORMED = -1.5

def mc_reward_grpo(completions, answer=None, **kwargs):
    penalty = float(kwargs.get("malformed_penalty", MALFORMED))
    texts = [str(c.get("content", c)) if isinstance(c, dict) else str(c) for c in completions or []]
    golds = [answer] if isinstance(answer, str) else list(answer or [None]*len(texts))
    golds = (golds * math.ceil(len(texts) / len(golds)))[:len(texts)]

    rewards = []
    for txt, g in zip(texts, golds):
        m = LETTER_RE.findall(txt or "")
        conf_match = CONF_RE.search(txt)
        confidence = float(conf_match.group(1)) / 100 if conf_match else 1.0
        length = len(txt.split())

        if not m:
            reward = penalty * (0.5 if "final" in txt.lower() else 1)
        else:
            pred = m[-1].upper()
            reward = CORRECT if g is None or pred == str(g).upper() else INCORRECT

        clarity_bonus = max(0, 1 - length / 200)
        reward = reward * confidence + 0.3 * clarity_bonus
        rewards.append(round(reward, 3))

    return rewards

🏆 Reward Components

✅ Correctness: +1.5 for correct answers, −1.0 for incorrect
⚠️ Malformed penalty: −1.5 if the answer lacks Final answer:
💬 Confidence scaling: adjusts reward based on confidence percentage (e.g. “80%”)
🧾 Clarity bonus: rewards concise and focused completions

🧪 Training Configuration

Setting	Value
Optimizer	AdamW
Learning Rate	5e-6
Epochs	1
Reward Type	Scalar
Engine	Unsloth Reinforcement Fine-tuning
Hardware	1xT4 16GB

📈 Training History

Reward Function Training History

💬 Example Usage

python

prompt = """
Passage: The fox saw the grapes hanging high and decided they were sour.
Question: What does the phrase "sour grapes" mean?
Choices:
A. The fox liked the grapes.
B. The fox couldn’t reach the grapes, so he pretended not to care.
C. The grapes were actually sour.
D. The fox wanted to share the grapes.

Final answer:
"""

response = model.generate(prompt)
print(response)

Example Output:

The phrase "sour grapes" refers to pretending not to care about something you cannot have.
Final answer: B

✅ Intended Use

Educational QA and reading comprehension
Reasoning-based multiple-choice benchmarks
Reinforcement learning and model evaluation research

⚠️ Limitations

May produce verbose or hallucinated explanations.
Limited generalization beyond English comprehension tasks.
Confidence percentages are heuristic, not calibrated.
Not suitable for high-stakes automated testing.

📚 Citation

If you use this model or the reward function, please cite:

latex

@misc{gemma3_race_unsloth,
  title = {Gemma-3 Fine-tuned on RACE with Unsloth and GRPO Reward},
  author = {Samuel Lopera Torres},
  year = {2025},
  howpublished = {\url{https://huggingface.co/Senshi5620/gemma-3-finetune}},
}

Senshi5620/gemma-3-finetune

🧠 Gemma-3 Fine-tuned on RACE (GRPO via Unsloth)