This repository contains the LoRA adapter checkpoint at training step 700 for a Grade 11-12 mathematics tutoring assistant aligned to Philippine DepEd senior high school topics.

Model Details

Model Description

Developed by: Deign Lazaro (MathPulse AI)
Funded by: Self-funded / independent development
Shared by: Deign86
Model type: PEFT LoRA adapter (not a standalone merged full model)
Language(s): English, Filipino (Taglish instructional style)
License: Apache-2.0 (inherits base model license constraints)
Finetuned from model: unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit

Model Sources

Training workflow: Kaggle notebook pipeline with Unsloth + PEFT QLoRA
Checkpoint source run: deignlazaro/qwen25-deped-lora-production (Kaggle)
Current artifact: Step-700 adapter snapshot selected as the last stable checkpoint before further-loss instability

Uses

Direct Use

This adapter is intended for:

Grade 11-12 math tutoring
Step-by-step worked solutions
Topic explanation and guided practice prompts

Because this is a LoRA adapter, load it on top of the base model.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit"
adapter_id = "Deign86/deped-math-qwen2.5-7b-checkpoint-700-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

Downstream Use

Recommended downstream use cases:

Classroom support copilots
Practice-question explanation bots
Internal educational QA tools for SHS math review

Out-of-Scope Use

Not intended for:

High-stakes grading decisions without human review
Medical, legal, or safety-critical advice
Formal theorem proving or guaranteed symbolic correctness

Bias, Risks, and Limitations

May produce incorrect arithmetic or overconfident reasoning.
Performance is domain-tilted toward SHS math instruction style and may degrade outside that scope.
Explanations can vary in quality depending on prompt clarity and problem formatting.
Should be used with human oversight for exams, assessment feedback, and official school outputs.

Recommendations

Use deterministic decoding for tutoring outputs where consistency matters.
Add lightweight answer checking (numeric/unit checks) in production.
Keep fallback routing to stronger/general models for out-of-domain prompts.
Run periodic red-team prompts for hallucination and curriculum drift checks.

Training Details

Training Data

DepEd SHS (Grades 11-12) mathematics tutoring-style instruction data
Curriculum focus includes General Mathematics, Statistics and Probability, Business Math, and Precalculus
Data formatted for chat-style supervised fine-tuning

Training Procedure

Method: QLoRA (Unsloth + PEFT)
Base model: 4-bit quantized Qwen2.5-7B Instruct variant
Checkpointing: periodic step checkpoints; this card tracks step-700
Rationale for selection: selected as the last stable checkpoint before loss escalation in continued training

Training Hyperparameters (core)

LoRA rank (r): 32
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
T4-safe sequence length during stable runs: 1024

Compute

Hardware type: NVIDIA Tesla T4 (Kaggle)
Precision regime: 4-bit base model + LoRA adapters

Evaluation

Current Status

This release is a checkpoint artifact for integration testing and iterative development. No formal benchmark table is claimed in this card.

Practical Validation Guidance

Before production promotion, validate on:

Topic coverage by grade-level competencies
Numerical accuracy on held-out problem sets
Explanation quality and rubric alignment
Hallucination/error-rate under ambiguous prompts

Technical Specifications

Architecture and Objective

Architecture family: Qwen2.5 (causal LM)
Adaptation method: PEFT LoRA adapter layers
Objective: instruction-following for educational math tutoring interactions

Files in This Repo

adapter_model.safetensors
adapter_config.json
Tokenizer artifacts (tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, etc.)

Model Card Authors

Deign Lazaro (Deign86)

Model Card Contact

For feedback, use the Hugging Face Discussions tab on this model repository.