You can deploy this model and start earning money today!

Satyamp777/
Qwen-GRPO-Agent

Name: Qwen-GRPO-Agent
Availability: InStock

safetensors

Views

No views yet

Qwen-GRPO-Agent

📌 Model Overview

The Qwen-GRPO-Agent is an advanced reasoning model aligned using Reinforcement Learning from Verifiable Rewards (RLVR).

Instead of relying on human preference data (RLHF), this model was post-trained using DeepSeek's Group Relative Policy Optimization (GRPO) algorithm. It is specifically designed to handle multi-step reasoning tasks by generating an internal chain-of-thought before arriving at a final, verifiable answer.

Base Model: Qwen/Qwen2.5-1.5B-Instruct
Alignment Technique: GRPO (Group Relative Policy Optimization)
Objective: Verifiable mathematical and logical reasoning using structured XML tags (<think> and <answer>).

⚙️ Architecture & Training Setup

This project demonstrates highly optimized LLM post-training within constrained hardware environments, reflecting strong applied AI and data architecture principles.

Frameworks: unsloth, trl, vllm
Reward Functions: 1. Format Reward: Strictly enforces the use of <think> for reasoning steps and <answer> for the final output. 2. Correctness Reward: Evaluates the final extracted output against deterministic unit tests (exact match).
Optimization: Utilized 4-bit quantization and LoRA adapters to maximize compute efficiency during the reinforcement learning loop.

🚀 How to Use the Model

You can load and test this model easily using the transformers library:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Satyamp777/Qwen-GRPO-Agent"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

# Format your prompt
prompt = "If I have 5 apples and buy 3 more, how many do I have? Think step-by-step."
messages = [
    {"role": "system", "content": "You are a helpful AI reasoning agent. Put all your thinking inside <think> </think> tags. Put your final answer inside <answer> </answer> tags."},
    {"role": "user", "content": prompt}
]

# Generate the response
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))