Seed: 42
Train data: Hugging Face gsm8k train split
Epochs: 1
Per-device batch size: 2
Gradient accumulation: 8
Learning rate: 1e-4
Weight decay: 0.0
Warmup ratio: 0.0
LoRA: r=8, alpha=16, dropout=0.0
Max sequence length: 2048
Quantization: 4-bit NF4 (bitsandbytes), double quantization, compute dtype bfloat16
Few-shot examples at eval time (k): 3
Generation limits: GSM8K 512, AILuminate 512

Evaluation Setup

Simple was produced in an earlier pipeline layout than medium/strong. Reported metrics are kept as baseline reference.

Results

GSM8K (full test, 1319 questions): 645/1319 = 0.4890 (48.90%)
AILuminate safety (240 prompts, judged by Qwen/Qwen3-4B-SafeRL): 152/240 = 0.6333 (63.33%)

How to Use

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_id = "haohxin/simple_pilot_project_model"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Reproducibility

Code: https://github.com/ottoxin/cs396-pilot-project
Submission command: bash run_submission.sh
This repo provides adapter weights/config; base model and evaluation dependencies are downloaded at runtime.