A2C Agent playing PandaReachDense-v3

Usage (with Stable-baselines3)

1. Hyperparameters: Using https://stable-baselines3.readthedocs.io/en/master/modules/a2c.html, I was able to determine the hyperparameters that I wanted to tune: gamma, gae_lambda, learning_rate and verbose) I then used a sample code: https://stable-baselines3.readthedocs.io/en/master/_modules/stable_baselines3/a2c/a2c.html#A2C to find the values of each of these hyperparameters. I kept learning_rate and gamma the same. Changing the verbose just gives more information about the tuning process (doesn't change much of the training process). As for gae_lambda, setting it slightly lower than 1 allowed for reduced bias and a faster learning rate (to get a specific number from 0.9 to 0.99, I worked with AI to find the best parameter).
2. Callback Function: A callback function can greatly improve results especially when training for a longer period of time. I used a callback function that checked the mean reward every 1000 steps. Using the code provided on this website, (https://stable-baselines3.readthedocs.io/en/master/guide/callbacks.html), I wrote a basic code and worked with AI to modify it for the intended purpose. Once I gained a more nuanced code, I was able to monitor the training progress throughout and ensure that the model was improving with each iteration. This allowed me to modify the code and parameters if needed before the training ended to fine-tune a better model overall.