PPO Agent for CartPole-v1
This repository contains a PPO Actor-Critic agent trained on the
Gymnasium CartPole-v1 environment as part of Machine Learning
Experiment 5.
Experiment configuration
- Algorithm: Proximal Policy Optimization (PPO)
- Environment: CartPole-v1
- Framework: PyTorch
- Random seed: 42
- Clip coefficient: 0.2
- Training steps: 19968
- Parallel environments: 4
- Rollout steps: 128
- Learning rate: 0.00025
Evaluation results
- Evaluation episodes: 20
- Mean reward: 442.50
- Standard deviation: 75.60
- Minimum reward: 299.00
- Maximum reward: 500.00
Debugging modifications
The original tutorial used an outdated Gym interface. The implementation
was updated to Gymnasium and modified to support the current reset()
and step() return formats. The Actor-Critic network, PPO training loop,
evaluation procedure, model saving, training visualization, and video
recording pipeline were successfully executed on Google Colab.
Repository contents
ppo_cartpole_task1_seed42.pt: trained PyTorch checkpointresults.json: quantitative evaluation resultstask1_cartpole_training_curve.png: training reward curveppo_cartpole_demo.mp4: evaluation video
