A LoRA adapter on top of Qwen3-4B-Instruct trained with GRPO to play competitive Pokemon Showdown (Gen 4 OU). This is run 3 — the best performing checkpoint from the series.

Links:

GitHub — OpenEnv-WolfeClick — full environment, training pipeline, replay viewer
Blog post — How I turned competitive Pokemon into an RL environment for LLMs
Live demo (HF Space) — watch the model play a recorded battle

Why Pokemon?

Competitive Pokemon isn't just a children's game — it's a partially observable, long-horizon decision problem. Each turn the model must:

Manage hidden information (opponent team, items, abilities)
Make tradeoffs that pay off 5–10 turns later (stat setup, sacrifice plays)
Output valid structured JSON within a constrained legal action space

That makes it a legitimate benchmark for LLM planning and reasoning, and a natural fit for GRPO training with real environment rewards.

Model Details

Property	Value
Base model	`Qwen/Qwen3-4B-Instruct-2507`
Adapter type	LoRA (PEFT)
LoRA rank	32
LoRA alpha	32
Dropout	0.0
Target modules	q/k/v/o proj, gate/up/down proj
Training method	GRPO on live battle trajectories
Format	Gen 4 OU (competitive, older format — more stable)

How to Use

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
tokenizer = AutoTokenizer.from_pretrained("Atharva2099/openenv-smogon-rl")
model = PeftModel.from_pretrained(base, "Atharva2099/openenv-smogon-rl")

# The model expects a structured markdown battle state and outputs a JSON action:
# {"action": "move" | "switch", "choice": "Exact Move or Pokemon Name"}

For full environment setup and battle loop, see the GitHub repo.

Training Pipeline

Base: Qwen3-4B-Instruct
    |
[JSON SFT warm-up]         teach the model to output valid action JSON
    |
[Rollout collection]       live battles against RandomPlayer on local Showdown
    |
[GRPO training]            optimize on real shaped rewards from environment
    |
LoRA checkpoint -> HF Hub

Reward signal (dense, per-turn):

+1.0 per 10% opponent HP dealt / -1.0 per 10% HP lost
+3.0 opponent faint / -3.0 self faint
+0.5 super effective hit / -1.0 immune/no-effect
+0.5 per stat stage gained (capped), +1.0 per 10% healed (capped)
-10.0 illegal action (hallucinated move/pokemon)
-0.05 step penalty (anti-stall)

Checkpoints

Branch	Notes
`grpo-qwen3-4b-run1`	First GRPO run, baseline
`grpo-qwen3-4b-run2`	Tuned reward shaping
`grpo-qwen3-4b-run3`	Best performing (this branch = main)

Limitations

Trained against RandomPlayer — will struggle against strong competitive opponents
Gen 4 OU format only; not tested on other generations or formats
Small model (4B) with limited working memory for very long battles
No search or lookahead — purely reactive turn-by-turn policy

Author

Atharva — Medium | GitHub