Parameter	Value
Algorithm	PPO
Policy	MlpPolicy
n_steps	32
batch_size	256
n_epochs	20
gamma	0.98
gae_lambda	0.8
ent_coef	0.0
clip_range	0.2
learning_rate	0.001
n_envs	8
total_timesteps	100,000

Usage

python

from stable_baselines3 import PPO
from huggingface_sb3 import load_from_hub

# Download and load the model
checkpoint = load_from_hub(
    repo_id="shubhamshakya21/ppo-CartPole-v1",
    filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)

# Use the model
import gymnasium as gym
env = gym.make("CartPole-v1")
obs, info = env.reset()
for _ in range(1000):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

Training Recipe

Based on the official sb3/ppo-CartPole-v1 recipe from the SB3 RL Zoo. Key design choices:

Short rollout (n_steps=32): fits CartPole's short episodes
High data reuse (n_epochs=20): maximizes learning per rollout
Tuned discount (gamma=0.98): optimized for ~500-step horizon
8 parallel envs: faster data collection

Dependencies

pip install stable-baselines3 huggingface_sb3 gymnasium