You can deploy this model and start earning money today!

ehzawad/
qwen3_1_7b-gsm8k-grpo

Name: qwen3_1_7b-gsm8k-grpo
Availability: InStock

Views

No views yet

Qwen3 GSM8K SFT + GRPO Adapter

This is a LoRA adapter for Qwen/Qwen3-1.7B trained on the GSM8K dataset. It was trained using an initial Supervised Fine-Tuning (SFT) phase followed by Group Relative Policy Optimization (GRPO).

Training Details

Base Model: Qwen/Qwen3-1.7B
SFT Steps: 350
GRPO Steps: 180
Hardware: NVIDIA L4
Quantization: 4-bit NF4

Complete Inference Code

To use this adapter, load the base model and apply the PEFT adapter:

python

!pip install --upgrade torchao

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen3-1.7B"
adapter_id = "ehzawad/qwen3_1_7b-gsm8k-grpo"

# 1. Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

# 2. Load base model
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 3. Load and apply adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# 4. Prepare prompt
system_prompt = "You are a careful math reasoning assistant. Solve the problem step by step, but keep the solution concise. Use only the needed calculations, avoid repetition, and end with exactly one final answer in the form \\boxed{answer}."
question = "Janet has 3 bags with 4 apples each. She gives away 5 apples and then took back 4. Then ate 3 apples and then friends took away 2 apples and then he boughts 5 apples again. How many remain?"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": question}
]

# 5. Format and Generate
try:
    inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, enable_thinking=True, return_dict=True, return_tensors="pt").to(model.device)
except TypeError:
    inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.6,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[-1]:], skip_special_tokens=True))