Views
No views yet
krooz/pyre-ppo-agent⚠️ This is a raw PyTorch checkpoint, not atransformersmodel. The Hugging Face hosted Inference API cannot run it directly. Use the inference code below to load and run it locally.
pyre_ppo_hard_v2)artifacts/pyre_ppo_hard_v2.csv, pyre_ppo_hard_v2_eval.csv,
pyre_ppo_hard_v2.png (MA-20 curves match save_training_graph_png in train_torch_ppo.py),
and artifacts/pyre_ppo_hard_v2_training.log (HTTP trainer via train_torch_ppo_http.py, env at http://localhost:8000).| Metric | Value |
|---|---|
| Total episodes | 600 |
| Wall-clock training time | ~227 s (~2.6 eps/s) |
| Final success rate (MA-20, training graph title) | 55% |
| Final reward mean (MA-20) | +3.21 |
Final success rate (rolling last 30 ep, CSV s30 / log) | 47% |
| Overall evacuation rate (all 600 ep, CSV) | 52.7% |
| Per-difficulty evacuation (easy / medium / hard) | 67.7% / 59.5% / 10.5% |
| Curriculum | easy → medium → hard with patience gate (0.70 over 20 ep); hard-phase mix hard:0.4, medium:0.4, easy:0.2 |
| Eval cadence | Every 25 episodes, 5 deterministic rollouts |
| Eval difficulty | hard (pyre_ppo_hard_v2_eval.csv) |
1uv run python training/ppo/train_torch_ppo_http.py \
2 --episodes 600 \
3 --difficulty-schedule easy,medium,hard \
4 --patience-threshold 0.70 \
5 --patience-window 20 \
6 --hard-mix-dist hard:0.4,medium:0.4,easy:0.2 \
7 --update-every 8 \
8 --update-epochs 6 \
9 --eval-every 25 \
10 --eval-difficulty hard \
11 --eval-episodes 5 \
12 --checkpoint-every 50 \
13 --entropy-coef 0.05 \
14 --step-delay 0 \
15 --viz-after-ep 500 \
16 --output artifacts/pyre_ppo_hard_v2.pt \
17 --log-file artifacts/pyre_ppo_hard_v2_training.log| Property | Value |
|---|---|
| Total parameters | 12,065,650 |
| Input vector dim | 23,140 (encoder base_dim 5785 × 4 stacked frames) |
| Action dim | 41 (4 move + 4 look + 1 wait + 16 door open + 16 door close) |
| Hidden MLP | 512 → 256 → 128 |
| Param | Value |
|---|---|
| Learning rate | 3×10⁻⁴ (with LR decay toward 0.1× end factor unless disabled) |
| PPO clip ε | 0.2 |
| Entropy coeff | 0.05 |
| Value coeff | 0.5 |
| Gamma | 0.99 |
| GAE λ | 0.95 |
| PPO update every | 8 episodes |
| PPO epochs / minibatch | 6 / 256 |
| Max grad norm | 0.5 |
| Observation mode | visible (partial observability) |
| Device | cuda (train_torch_ppo.py default; set --device cpu if needed) |
pyre_ppo_hard_v2_eval.csv)| Episode | Difficulty | Success rate | Reward mean | Steps mean |
|---|---|---|---|---|
| 25 | hard | 0% | −10.124 | 58.0 |
| 50 | hard | 0% | −11.184 | 58.4 |
| 75 | hard | 0% | −11.468 | 35.6 |
| 100 | hard | 0% | −9.827 | 74.0 |
| 125 | hard | 20% | −7.792 | 25.0 |
| 150 | hard | 40% | −4.238 | 28.0 |
| 175 | hard | 20% | −6.674 | 35.2 |
| 200 | hard | 0% | −12.304 | 74.6 |
| 225 | hard | 0% | −11.080 | 100.0 |
| 250 | hard | 20% | −5.648 | 38.4 |
| 275 | hard | 0% | −10.368 | 76.2 |
| 300 | hard | 20% | −4.421 | 72.8 |
| 325 | hard | 0% | −11.180 | 48.2 |
| 350 | hard | 0% | −9.845 | 74.0 |
| 375 | hard | 0% | −11.320 | 26.4 |
| 400 | hard | 0% | −12.256 | 34.0 |
| 425 | hard | 20% | −7.024 | 36.4 |
| 450 | hard | 0% | −10.726 | 56.4 |
| 475 | hard | 0% | −9.072 | 88.6 |
| 500 | hard | 0% | −12.050 | 66.6 |
| 525 | hard | 20% | −5.528 | 41.6 |
| 550 | hard | 0% | −11.274 | 52.4 |
| 575 | hard | 0% | −10.578 | 58.4 |
| 600 | hard | 0% | −12.068 | 36.6 |
| File | Description |
|---|---|
model.pt | PyTorch checkpoint (network_state, optimizer_state, scheduler_state, args, episode) |
training_graph.png | Training curves (reward + success rate vs episode) |
episode_metrics.csv | Per-episode training metrics |
eval_metrics.csv | Periodic eval aggregates |
training.log | Full console transcript of the HTTP training run |
1import sys
2import torch
3from huggingface_hub import hf_hub_download
4
5# 1. Point Python at your local pyre_env checkout (or install the package)
6sys.path.insert(0, "pyre_env")
7
8from training.ppo.train_torch_ppo import (
9 ActorCritic,
10 ObservationEncoder,
11 action_index_to_env_action,
12 build_action_mask,
13)
14
15# 2. Download the checkpoint from this Hub repo
16ckpt_path = hf_hub_download(repo_id="krooz/pyre-ppo-agent", filename="model.pt")
17ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)
18
19# 3. Rebuild the policy from saved training args
20saved_args = ckpt["args"]
21encoder = ObservationEncoder(mode=saved_args.get("observation_mode", "visible"))
22hidden_sizes = tuple(int(x) for x in saved_args.get("hidden_sizes", "512,256,128").split(","))
23history_length = saved_args.get("history_length", 4)
24input_dim = encoder.base_dim * history_length
25network = ActorCritic(input_dim, 41, hidden_sizes)
26network.load_state_dict(ckpt["network_state"])
27network.eval()
28print(f"Loaded checkpoint from episode {ckpt.get('episode', '?')}")
29
30# 4. Roll out one episode (in-process env — swap for HTTP client if you prefer)
31from openenv_pyre import PyreEnvironment
32from collections import deque
33import numpy as np
34
35env = PyreEnvironment()
36obs = env.reset(difficulty="medium")
37frames = deque([np.zeros(encoder.base_dim, dtype=np.float32)] * history_length, maxlen=history_length)
38frames.append(encoder.encode(obs))
39
40total_reward = 0.0
41with torch.no_grad():
42 while True:
43 state_vec = np.concatenate(list(frames), dtype=np.float32)
44 obs_t = torch.tensor(state_vec, dtype=torch.float32).unsqueeze(0)
45 mask_t = torch.tensor(build_action_mask(obs, exclude_look=True), dtype=torch.float32).unsqueeze(0)
46 action_t, _, _ = network.act(obs_t, mask_t, deterministic=True)
47 obs = env.step(action_index_to_env_action(int(action_t.item())))
48 total_reward += float(obs.reward or 0.0)
49 frames.append(encoder.encode(obs))
50 if obs.done:
51 break
52
53print(f"Episode finished — evacuated={obs.agent_evacuated} reward={total_reward:.3f}")training/ppo/train_torch_ppo_http.pytraining/ppo/train_torch_ppo.pytraining/ppo/pyre_ppo_training.ipynb