CanoPy is a self-playing reinforcement learning Rocket League agent designed for the RLBot Championship 2025.
It uses PPO (Proximal Policy Optimization) to learn 2v2 gameplay through self-play. The agent is trained to play effectively on both blue and orange teams and can generalize to various team compositions.
Model Details
Framework: RLGym + RLBot v5
Algorithm: PPO (via rlgym-ppo)
Team size: 2v2
Action repeat: 8
Observations:DefaultObs with normalized positions, angles, velocities, and boost
Action space: Lookup table actions with repeat frames
Reward shaping: Combined reward including:
Speed toward ball
In-air bonus
Ball velocity toward goal
Goal scoring reward
image/png
image/png
Training Configuration (from config.json)
Number of processes: 4
Minimum inference ratio: 80%
Steps per checkpoint: 1,000,000
PPO batch size: 100,000
PPO minibatch size: 50,000
PPO epochs per update: 2
Experience buffer size: 300,000
Policy network layers: [256, 128]
Critic network layers: [256, 128]
Policy learning rate: 0.0001
Critic learning rate: 0.0001
PPO entropy coefficient: 0.01
Standardize returns: true
Standardize observations: false
Total training steps: 1,000,000,000
Checkpoint directory: ./checkpoints
Intended Use
CanoPy is intended for research, competition, and experimentation within the RLBot framework. It is designed to compete in the ML bot bracket of the RLBot Championship 2025.
Limitations
Performance is dependent on training; untrained or partially trained models may perform poorly.
The bot has been trained for standard Rocket League 2v2 matches; it may not generalize to unusual map sizes, mutators, or game modes.
Does not include human-like strategy beyond what PPO has learned from self-play.
Evaluation
CanoPy can be evaluated using the evaluate() function in the training script. Expected evaluation includes average episode returns and gameplay against copies of itself.
Note: To meet RLBot Championship submission requirements, further testing against Psyonix Pro bots may be necessary.
