ppo-LunarLander-v3-flip – AI Model by KaptainKris | AlphaNeural AI

You can deploy this model and start earning money today!

KaptainKris/
ppo-LunarLander-v3-flip

like

0

stable-baselines3

reinforcement-learning

deep-reinforcement-learning

Views

No views yet

PPO LunarLander flip, recover and land agent

This repository contains a Stable-Baselines3 PPO actor-critic agent trained on a customised LunarLander-v3 environment.

Learned task

The curriculum teaches one policy to:

complete a full rotation in a fixed direction;
recover upright and arrest angular motion;
enter the landing zone;
reduce descent speed and land safely.

Reward configuration version: v5-soft-touchdown-refinement.

Changes in this upload

Continued from the selected Phase D checkpoint.
Added an altitude-dependent vertical-speed target.
Added a near-ground quadratic descent-overspeed penalty.
Added a dedicated in-zone crash penalty.
Reduced risky late horizontal corrections using a deadband.

Reward design

The shaped reward includes:

one-off rotation-progress and flip-completion rewards;
an upright post-flip recovery reward;
horizontal guidance towards the landing zone;
an altitude-dependent vertical-speed target;
attitude and angular-speed control;
a near-ground descent-overspeed penalty;
distinct penalties for off-zone landings and in-zone crashes.

Parameter	Value
`required_rotations`	1
`rotation_direction`	1
`upright_tolerance_radians`	0.3
`recovery_angular_velocity_tolerance`	0.3
`pre_flip_original_reward_weight`	0.15
`post_flip_original_reward_weight`	3
`pre_flip_center_weight`	25
`pre_flip_horizontal_speed_weight`	15
`pre_flip_shaping_weight`	1
`pre_flip_shaping_clip`	10
`flip_corridor_half_width`	0.35
`flip_entry_penalty_weight`	600
`rotation_progress_bonus`	300
`flip_completion_bonus`	500
`recovery_bonus`	250
`flip_landing_bonus`	2000
`landing_without_flip_penalty`	600
`no_flip_terminal_penalty`	500
`failed_landing_penalty`	900
`outside_zone_landing_penalty`	1200
`in_zone_crash_penalty`	1500
`post_flip_shaping_weight`	2
`post_flip_shaping_gamma`	0.999
`post_flip_shaping_clip`	30
`post_flip_center_weight`	100
`post_flip_horizontal_speed_weight`	50
`post_flip_vertical_speed_weight`	100
`post_flip_angle_weight`	70
`post_flip_angular_speed_weight`	40
`post_flip_leg_contact_weight`	30
`landing_zone_half_width`	0.2
`post_flip_zone_excess_weight`	180
`post_flip_target_vx_gain`	0.6
`post_flip_max_target_vx`	0.4
`post_flip_horizontal_deadband`	0.08
`post_flip_target_vy_high`	-0.45
`post_flip_target_vy_near_ground`	-0.12
`near_ground_height`	0.6
`safe_touchdown_vertical_speed`	0.14
`near_ground_overspeed_weight`	120

Evaluation

Deterministic evaluation over 100 fixed-seed episodes:

Metric	Value
Mean shaped reward	-413.81
Mean original reward	-671.00
Full-rotation rate	15.0%
Recovery rate	9.0%
Recovery given a flip	60.0%
Safe-landing rate	0.0%
Flip-and-land rate	0.0%
Terminal in-zone rate	44.0%
In-zone crash rate	13.0%

Architecture

Algorithm: PPO
Policy: MLP actor-critic
Actor hidden layers: [128, 128]
Critic hidden layers: [128, 128]
Observation dimensions: 11
Discrete actions: 4

Training configuration

Parameter	Value
Phase timesteps	10000000
Parallel environments	32
Learning rate	5e-05
Rollout steps per environment	1024
Batch size	64
Optimisation epochs	4
Gamma	0.999
GAE lambda	0.98
Entropy coefficient	0.005
PPO clip range	0.1
Training seed	42

Replay

Seed: 20000
Original reward: -669.61
Shaped reward: -294.96
Rotations completed: 1.00
Flip completed: False
Recovery completed: False
Landed safely: False
Outcome: episode_ended_without_flip

Flip, recovery and landing replay

Open the full MP4 replay

Repository files

ppo-LunarLander-v3-flip-128x128.zip: selected PPO model
flip_landing_reward_wrapper.py: custom environment wrapper
training_config.json: PPO training settings
reward_config.json: reward configuration
episode_results.csv: fixed-seed evaluation episodes
results.json: machine-readable evaluation summary
config.json: compact model metadata
replay.gif: model-card preview
replay.mp4: full replay