Property	Value
Environment ID	`LunarLander-v2`
Observation Space	`Box(8,)`
Action Space	`Discrete(4)`
Vectorized Envs (train)	16 (`DummyVecEnv`)
Observation Normalization	`VecNormalize(norm_obs=True)`
Reward Normalization	`VecNormalize(norm_reward=True)`

Training Hyperparameters

Parameter	Value
Algorithm	PPO
Policy	MlpPolicy
Policy Architecture	`pi=[256, 256], vf=[256, 256]`
Total Timesteps	2,000,000
n_steps	2048
batch_size	128
n_epochs	10
gamma	0.999
gae_lambda	0.98
clip_range	0.2
ent_coef	0.001
learning_rate	3e-4
vf_coef	0.3
max_grad_norm	0.5

EvalCallback: eval every 10,000 steps, 10 eval episodes, saves best model checkpoint.

Usage

python

from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize
from stable_baselines3.common.monitor import Monitor
from huggingface_hub import hf_hub_download
import gymnasium as gym

# Load model
model = PPO.load(hf_hub_download("muhrivandysetiawan/ppo-LunarLander-v2", "PPO-LunarLander-v2.zip"))

# Load normalization stats
vec_path = hf_hub_download("muhrivandysetiawan/ppo-LunarLander-v2", "vec_normalize.pkl")
eval_env = DummyVecEnv([lambda: Monitor(gym.make("LunarLander-v2", render_mode="rgb_array"))])
eval_env = VecNormalize.load(vec_path, eval_env)
eval_env.training = False
eval_env.norm_reward = False

obs = eval_env.reset()
done = False
while not done:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, info = eval_env.step(action)

Notes

No W&B integration in this version — TensorBoard logging used during training (./ppo_lunarlander_tensorboard/).
vec_normalize.pkl must be loaded alongside the model for correct inference; omitting it degrades performance significantly.
Trained on Google Colab with virtual display (pyvirtualdisplay).