A LoRA fine-tune of Qwen2.5-1.5B-Instruct trained with GRPO (Group Relative Policy Optimization) on the Coalition Market multi-agent negotiation environment. Agents learn to trade resources, form coalitions, and complete construction goals through emergent cooperation.

Results

Metric	Random Baseline	After GRPO	Improvement
Mean episode reward	9.12	14.20	+5.08 (+55%)
Training steps	—	600	—
Training loss	—	-0.0007	—

The trained model consistently outperforms random policy across all evaluation episodes, with mean reward improving from 9.12 → 14.20.

Environment

Coalition Market is a custom multi-agent environment where 4 LLM agents (A, B, C, D) each have:

A private construction goal (e.g. "Steel Mill") requiring specific goods
A starting inventory of resources
The ability to propose trades, form coalitions, accept/reject offers, and build projects

Agents interact over 10 rounds. The environment is hosted as a HuggingFace Space: 👉 coalition-market-env Space

Training Details

Parameter	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Method	GRPO (TRL v0.24.0)
LoRA rank	16
LoRA alpha	16
Target modules	q/k/v/o/gate/up/down proj
Quantization	4-bit (Unsloth)
Batch size	4 (effective 8 with grad accum)
Learning rate	5e-5 (cosine)
Num generations	4 per prompt
Max prompt length	320 tokens
Max completion length	48 tokens
Training hardware	Kaggle T4 GPU
Training time	~50 minutes

Reward Functions

Three reward signals were combined during GRPO training:

Format reward (+0.5 / -0.3): Checks if the agent output matches a valid action pattern (propose_trade, accept_trade, reject_trade, form_coalition, build_project, pass).

Strategic reward (+0.3): Rewards non-pass valid actions, encouraging active participation over idle behaviour.

Environment reward: Direct reward signal from the Coalition Market environment based on successful trades and goal completions.

How to Load

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "Santhosh-14146/coalition-market-qwen2.5-grpo",
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

Sample Inference

python

import torch

SYSTEM_PROMPT = """You are an agent in a multi-agent resource market.
Complete your construction goal by trading goods with other agents.

ACTIONS — respond with EXACTLY ONE:
  propose_trade(TO, {GOOD: QTY}, {GOOD: QTY})
  accept_trade(TRADE_ID)
  reject_trade(TRADE_ID)
  form_coalition([AGENTS], DESCRIPTION)
  build_project()
  pass()

Output ONLY the action string. No explanation."""

user_prompt = """Round 3/10 | Agent A
Goal: Steel Mill | Requires: Iron:3, Coal:2, Stone:1
Inventory: Iron:3, Coal:2
Missing: Stone:1
Others:
  B: Stone:3, Wood:2
Pending trades for you:
  #abc123: B offers Stone:1 for Iron:1
Action:"""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": user_prompt},
]

input_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

with torch.no_grad():
    out = model.generate(input_ids, max_new_tokens=48, temperature=0.4,
                         do_sample=True, pad_token_id=tokenizer.eos_token_id)

action = tokenizer.decode(out[0, input_ids.shape[-1]:], skip_special_tokens=True)
print(action.strip().split("\n")[0])
# Expected: accept_trade(abc123)

Project Links

🌐 Environment Space: https://huggingface.co/spaces/Santhosh-14146/coalition-market-env
🤗 Model: https://huggingface.co/Santhosh-14146/coalition-market-qwen2.5-grpo
📓 Training Notebook: https://www.kaggle.com/code/santhosh14146/notebookb20f0a67a5/edit

Citation

bibtex

@misc{coalition-market-grpo-2026,
  author    = {Santhosh-14146},
  title     = {Coalition Market: Multi-Agent Negotiation with GRPO},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Santhosh-14146/coalition-market-qwen2.5-grpo}
}

GRPO method from:

bibtex

@article{shao2024deepseekmath,
  title  = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
  author = {Zhihong Shao et al.},
  year   = 2024,
  eprint = {arXiv:2402.03300}
}