GRPO-trained 4-Agent Multi-Agent System for Enterprise Customer Support
Trained for the PyTorch OpenEnv Hackathon 2026 — Round 2
What This Model Does
This is a merged (LoRA + base) version of unsloth/Meta-Llama-3.1-8B-Instruct, fine-tuned using GRPO (Group Relative Policy Optimization) via TRL on an NVIDIA A100 SXM4 80GB.
The model was trained to operate as 4 specialized agents in a multi-agent negotiation environment:
Agent
Specialization
Trained Success Rate
Technical
App crashes, API failures, data sync
✅ 100%
Billing
Refunds, duplicate charges, subscriptions
⚠️ 67% (hardest domain)
Account
2FA resets, password recovery, login issues
✅ 100%
Manager
QA evaluation, escalation routing
✅ 100%
Team Average
All 4 agents combined
91.8%
The agents use a 3-phase negotiation protocol:
Bidding — each specialist bids confidence [0.0–1.0] for the incoming ticket
Execution — the highest-confidence specialist provides the solution
Resolution — the Manager evaluates and approves/escalates
Training Details
Parameter
Value
Base Model
unsloth/Meta-Llama-3.1-8B-Instruct
Algorithm
GRPO (Group Relative Policy Optimization)
Library
TRL + Unsloth
Hardware
NVIDIA A100 SXM4 80GB
Training Steps
25 steps per agent (100 total)
Batch Size
32
Max Seq Length
2048
Quantization
4-bit (LoRA rank=16, alpha=32)
Final Loss
~1.6e-08 (team average)
Reward System (11 Signals, Anti-Hacking)
Positive:
correct_specialist_bid → +0.30 Agent bids correctly for their specialty
correct_solution → +0.30 Solution matches ground truth
appropriate_confidence → +0.15 Bid calibrated to actual accuracy
solution_format → +0.05 JSON format compliance
team_success_bonus → +0.20 Shared when ticket resolves
Penalties:
wrong_specialist → -0.20 Non-expert won the bid
wrong_solution → -0.20 Solution incorrect for ticket category
overconfident → -0.10 Bid >0.8 but solution was wrong
team_failure_penalty → -0.10 Shared penalty on failure
invalid_bid → -0.05 Confidence outside [0.0, 1.0]
timeout → -0.15 Exceeded MAX_STEPS (10)
How to Use
python
1from unsloth import FastLanguageModel
2import torch
34model, tokenizer = FastLanguageModel.from_pretrained(5 model_name ="RavichandraNayakar/openenv-grpo-merged",6 max_seq_length =2048,7 dtype =None,8 load_in_4bit =True,# ~4-5GB on GPU (fits Colab T4)9)10FastLanguageModel.for_inference(model)1112# Example: Ask Technical Agent to bid on a ticket13messages =[14{"role":"system","content":"""You are the Technical Support Specialist.
15You diagnose app crashes, API outages, and data sync issues.
16Respond ONLY with JSON: {"action_type": "bid", "confidence": <0.0-1.0>, "rationale": "<why>"}"""},17{"role":"user","content":"Ticket: My app keeps crashing. Error code 500."},18]1920inputs = tokenizer.apply_chat_template(21 messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"22).to(model.device)2324with torch.no_grad():25 out = model.generate(inputs, max_new_tokens=100, temperature=0.1, do_sample=True)2627response = tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True)28print(response)29# Expected: {"action_type": "bid", "confidence": 0.95, "rationale": "App crash is my specialty"}
