The model is trained using Group Relative Policy Optimization (GRPO) on a subset of the MBPP (Mostly Basic Python Problems) dataset and evaluated on HumanEval and MBPP benchmarks.

🧠 Model Details

Attribute	Value
Base Model	Qwen3-8B
Architecture	Transformer (Causal Language Model)
Parameters	~8.2B
Context Length	32K
Training Method	GRPO (Reinforcement Learning)
Task	Code Generation / Reasoning
Hardware	NVIDIA A100 (40GB)

⚙️ Training Details & Procedure

📂 Dataset

Dataset: MBPP (sanitized subset)
Training Samples: ~120
Task:
- Input: Problem description
- Output: Python function implementation

🧠 Training Method

The model was fine-tuned using Group Relative Policy Optimization (GRPO):

Multiple responses are generated per prompt
Outputs are compared within a group
Relative rewards are assigned
Model is updated to prefer higher-quality outputs

⚙️ Training Configuration

Parameter	Value
Training Steps	200
Generations per Prompt	4
Optimization	GRPO
Framework	Hugging Face TRL
Hardware	NVIDIA A100 (40GB)

🧪 Reward Functions

Execution Reward → rewards correct and executable code
Answer Similarity Reward → compares output with reference solution
SWE Quality Reward → encourages structured and readable code
Length Penalty → penalizes overly long responses

📉 Training Dynamics

Initial reward variance was very low
Limited differentiation between generated outputs
Weak learning signal during early training
Minor improvements observed later

📊 Evaluation Results

Benchmark Performance

Benchmark	Before	After	Change
HumanEval	42.0%	42.0%	No change
MBPP	2.0%	4.0%	+2%

📈 Key Observations

Slight improvement on MBPP
No improvement on HumanEval
Limited generalization due to:
- small dataset
- low reward variance

🚀 Intended Use

✅ Suitable For

Python code generation
Educational and research experiments
RL fine-tuning (GRPO) exploration

❌ Not Suitable For

Production systems
High-reliability coding tasks
Safety-critical applications

💻 Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Madhu2133/qwen3-8b-swe-grpo"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Write a Python function to check if a number is prime"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))