longnav-objectnav-flow-nopose-cotrain-2p5hz-rl-a09-held128-ck791
RL fine-tune of
longnav-objectnav-flow-nopose-cotrain-2p5hz,
by flow-SDE policy gradient on continuous-control ObjectNav. Checkpoint 791 of the
flow_sde_a09_held128 run — the first run in this family with a
genuinely held-out
in-training eval (26 pinned episodes excluded from the training stream), and the
checkpoint was
selected by that held-out series (peak band, cycles ~740–800), not by
training loss or end-of-run convention.
The base is the MERGED repo, not raw Qwen. This adapter holds the RL delta alone
(~2% of the SFT adapter's magnitude). Both turn_vector_head_config.json (model.model_id
— the field the flow_rollout backend actually reads) and adapter/adapter_config.json
point at Aasdfip/longnav-objectnav-flow-nopose-cotrain-2p5hz-merged, so --ckpt <this repo>
works with the same code that ran the SFT model, no changes. Applying it to raw
Qwen/Qwen3-VL-2B-Instruct instead would evaluate the pretrained VLM plus a 2%
perturbation — near-zero scores, and no error anywhere.
Results (sample400, 397 scored, paired)
Same 400 HM3D val episodes, same packing and seeds, --no-pose-injection on both arms.
Unlike the earlier -rl-a09-ck303 release (whose table came from a rank-stacked
composition over the raw base), these numbers were measured on exactly the composition
this repo ships: merged base + this adapter, probe-verified bit-equal to the training base.
| metric | this checkpoint | SFT baseline | delta |
|---|
| oracle success | 0.733 | 0.650 | +0.083 |
| oracle SPL | 0.483 | 0.401 | +0.082 |
| success | 0.589 | 0.554 | +0.035 |
| SPL | 0.272 | 0.254 | +0.018 |
| soft SPL | 0.311 | 0.291 | +0.021 |
McNemar on oracle success: 57 episodes fixed, 24 broken, z = 3.67, p < 0.001.
Against the prior RL release (-rl-a09-ck303, trained on 24 episodes / 4 scenes): this
checkpoint is equal or ahead in oracle success in every goal category (paired on the
same 397: oracle +0.030 vs that run's later ck391, p = 0.126 — statistical parity with a
uniform nominal edge), from a training pool with 20x the scene diversity and a held-out
selection protocol. As with every checkpoint in this family (no stop head, no success
bonus), the gains concentrate in reaching (oracle metrics) more than stopping (plain
success).
What the RL run was
| |
|---|
| algorithm | flow-SDE policy gradient (PPO-style clipped surrogate) on the denoising chain, N=3 of K=10 steps stochastic |
| exploration | a = 0.9 |
| action head | FROZEN (action_head_learning_rate = 0); gradients flow through it into the backbone LoRA |
| base | SFT LoRA merged into the weights; the trainable LoRA starts as a zero delta |
| advantage | REINFORCE++ with time-kernel baseline, gamma 0.95, whitened |
| training pool | 128 episodes over 80 HM3D train scenes, stratified by category x geodesic tercile, excluding plant |
| held-out eval | 26 pinned episodes (train scenes) excluded from the training stream; one ODE pass per 4 cycles |
| duration | ~950 cycles at lr 2e-6; ck791 selected at the held-out peak band (the run's own objective had converged by ~cycle 550) |
The training pool is scene-disjoint from HM3D val, so the table above is generalisation.
The held-out in-training gains (+0.042 success, +0.070 oSPL on never-trained episodes in
training scenes) transferred to val scenes essentially in full.
Files
| file | what |
|---|
adapter/ | LoRA (r=128, alpha=256) — the RL delta only, over the merged base |
turn_vector_head.pt | flow-matching action head; frozen during RL |
turn_vector_head_config.json | head config |
| tokenizer / preprocessor files | as trained |
Running it
Identical to the SFT model's command with --ckpt pointed here:
1python scripts/eval_objectnav_policy.py \
2 --episodes <hm3d objectnav val split> \
3 --scene-root <scene_datasets> \
4 --output-dir <out> \
5 --episode-ids "$(cat sample400_ids.txt)" \
6 --ckpt <this repo> \
7 --policy-python <env with transformers> \
8 --policy-sys-path <longnav src> \
9 --policy-backend flow_rollout --policy-seed 1000 \
10 --max-steps 175 --dt 0.04 --gap 10 \
11 --success-distance 1.0 --distance-to VIEW_POINTS \
12 --seed 0 --navmesh dataset --fresh-sim-per-episode \
13 --auto-stop --auto-stop-delay 15 \
14 --no-pose-injection \
15 --pid-preset baseline
--no-pose-injection matters: this policy family scores better with the pose feed off, and
the RL run's conversations were the nopose format.
Per-episode seeds are positional (seed + position in the run), so a differently-sized or
differently-ordered episode set reseeds every episode — roughly 40% of episodes flip
outcome between identical passes of this policy family. Reproduce the table with the full
400-episode list in the given order, not a subset.
