LoRA adapter for Qwen/Qwen3.5-4B, trained with TRL GRPO in a multi-turn agentic RL loop on a custom OpenEnv cricket environment. This is the warmup checkpoint (5-over curriculum, 25 GRPO steps). The main 20-over T20 run resumes from this adapter.

Training

Base: Qwen/Qwen3.5-4B (bf16)
LoRA: r=64, alpha=128, dropout=0.05; targets q/k/v/o + gate/up/down → 85M trainable (~1.98% of 4.2B)
Algorithm: GRPO via TRL (environment_factory=CricketCaptainToolEnv)
Curriculum: max_overs ∈ [2,2,2,2,2,3,3,3,4,4,5] sampled per rollout
Steps: 25, batch=4, num_generations=4, max_completion_length=4096, max_tool_calling_iterations=240
Optimizer: lr=5e-6, beta=0.0 (no KL ref), temperature=0.9, top_p=0.95
Reward composite: 0.20·r_result + 0.45·r_cricket + 0.25·r_behavior + 0.10·r_validity
- r_cricket — dense per-ball Dream11 fantasy proxy
- r_behavior — coherence + adaptation + opponent_awareness + regret
- r_validity — fraction of legal tool calls
- r_result — match outcome margin / win bonus (rare in 5-over)

Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", torch_dtype="bfloat16", device_map="auto")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
model = PeftModel.from_pretrained(base, "pratinavseth/cricket-captain-warmup-stage2")

Status

Warmup only. The trained policy that goes to head-to-head eval is the main run checkpoint (resumed from this one, lr=1e-5, 30 steps at 20-over T20, weights rebalanced to 0.35 / 0.30 / 0.25 / 0.10).

Repo: https://github.com/pratinavseth/cricket-captain-llm (OpenEnv hackathon submission).

Framework versions

PEFT 0.19.1