In the realm of Ping Pong, Mr. Pong is no ordinary paddle. Feared across the table as the Blue Beast, he commands every rally with ruthless precision, striking despair into the hearts of all who dare face him.
In other words, he is a reinforcement learning (RL) agent trained to compete at a grandmaster level in Ping Pong.
Model Architecture
Base Architecture: MrPongMLPForRL
Hidden Size: 160
Observation Size: 16
Number Of Layers: 3
Action Space: 3 (stay, up, down)
Activation: tanh
Mr. Pong uses a shared Actor-Critic MLP designed for 2D table tennis control. The 16-dimensional observation vector is passed through two hidden layers of 192 units each, with Tanh activations between them. The resulting 192-dimensional representation is shared by both the actor and critic heads. The actor outputs categorical logits across the 3 movement actions (stay, up, down), while the critic uses the same representation to estimate the scalar state value V(s). This shared setup keeps the network small and fast on CPU while letting the model combine ball coordinates, velocities, paddle momentum, raycasted intercept points, and opponent court openings into a single control decision.
Training Configuration
Number of Environments: 12
Rollout Steps: `128
PPO Epochs: 4
Minibatch Size: 64
Gamma: 0.99
GAE Lambda: 0.95
Clip Coefficient: 0.20
Value Coefficient: 0.50
Entropy Coefficient: 0.02
Maximum Gradient Norm: 0.75
Learning Rate: 3.5e-4
Adam Epsilon: 1e-5
Anneal Learning Rate: true
Total Timesteps: 10,000,000
Maximum Rally Steps: 1500
Training Opponents
Realistic Hard Logic (human-like perception horizon and bounce raycasting)
Medium Logic (linear trajectory extrapolation)
Minimax Lookahead (depth 1 and depth 2)
Current Self-Play (mirror matches against the active policy)
Historical Self-Play (past checkpoints sampled between 5 and 75 saves ago)
Easy Logic (reaction delay with targeting noise)
Impossible Hard Logic (0ms reaction time)
Random Agent
Training Results
Opponent Name
Win %
Draw %
Loss %
Record (W / D / L)
Avg Rally
Easy Logic
99.7%
0.0%
0.3%
997W / 0D / 3L
1.7 hits
Medium Logic
99.0%
0.2%
0.8%
990W / 2D / 8L
17.3 hits
Realistic Hard
86.6%
8.1%
5.3%
866W / 81D / 53L
33.9 hits
Impossible Hard
0.0%
98.4%
1.6%
0W / 984D / 16L
64.8 hits
Minimax Depth 1
43.0%
55.8%
1.2%
430W / 558D / 12L
32.2 hits
Minimax Depth 2
78.4%
20.3%
1.3%
784W / 203D / 13L
23.7 hits
Random Agent
100.0%
0.0%
0.0%
1000W / 0D / 0L
0.9 hits
Self-Play Mirror
1.6%
96.1%
2.3%
16W / 961D / 23L
50.5 hits
Mr. Pong has demonstrated the he has mastered the game Ping Pong; losing only <=5% of games.
Inference
First install the required dependencies:
pip install torch transformers
Next, download inference.py and run:
python inference.py
Use --mode play to play against the agent live in the terminal.
Use --mode simulate to run AI match simulations (default).
Add --opponent to choose a specific opponent (realistic_hard, medium, minimax_d1, minimax_d2, impossible_hard, easy, random).
Add --video to export an MP4 recording of the match.
License
Apache 2.0.
Citation
@misc{mrpong,
title = {Mr. Pong: Teaching RL agents to Play Ping Pong},
organization = {FromZero},
authors = {Paul Courneya},
year = {2026},
url = {https://huggingface.co/fromziro/MrPong]
}
