dexgarmentlab-folding-lifting-state-est-gps-flow-chamfer
GPSStateEstModel — graph-based (GNN + Transformer) state estimation for
variable-vertex cloth meshes. Reconstructs the full cloth mesh state from a
partial point cloud observation via flow matching, conditioned on each
cloth's own rest state and topology (no global template).
- Task data: DexGarmentLab mixed-garment fold + lift-place demos (
dexgarmentlab_folding_lifting_meshes.h5, Cloth-splatters/dexgarmentlab-folding-lifting-meshes)
- Formulation: flow matching + Chamfer auxiliary loss (variant C)
- Rest-position centering:
self
- Cross-attention mode: parallel
- Max vertices per mesh: 2048
- Best validation loss: 0.0008144395669104961 (checkpoint in
model/ is checkpoint-best)
- Training run:
dexgarment_flow_chamferC_2026-08-03_10-50-22_7138061 (full config in config.yml)
⚠️ Superseded — not recommended (note added 2026-08-24)
The Chamfer auxiliary loss (variant C) won the fold-unfold-lift ablation but does
not transfer to DexGarmentLab: at weight 0.5 it diverges on this dataset for both
schedulers, and the no-chamfer baselines were already 3–5× better on validation
before that happened. This checkpoint is kept for provenance; it should not be used
as a state estimator.
Use instead, both trained with the current recipe (EMA + heavy occlusion/outlier
augmentation, and — for the flow model — a ×3 coordinate scale):
For reference, on held-out demonstrations the current DDPM standard reaches
2.4e-4 m² validation MSE and identifies 7/8 test garments end-to-end through the
particle filter; ramp the Chamfer weight in or lower the learning rate if you want
to revisit this variant on a new dataset.
Chamfer auxiliary loss (variant C)
Trained with an auxiliary Chamfer distance loss between the denoised mesh
prediction and the observed point cloud (weight 0.5, applied only at low noise
levels — variant C from the fold-unfold-lift chamfer ablation). The extra loss
term is why this checkpoint's recorded best validation loss is not directly
comparable to the no-chamfer variants.
Rest-position centering: self
This checkpoint was trained with the self rest-position centering convention:
the per-cloth rest (template) positions fed to the model are centered by
subtracting the rest mesh's own (valid-vertex) mean, independent of the
observed point cloud.
Implications:
- The model is translation-invariant in rest space, which makes it suitable for
real-world inference: the canonical template can live in any coordinate
frame relative to the camera point cloud.
- At inference, the rest positions must be self-centered the same way. Pipelines
in the training repo read
model.config.rest_pos_centering (recorded as
"self" in this checkpoint's config.json) and do this automatically.
- Not interchangeable with checkpoints trained with the legacy
pcd convention
(all state-estimation checkpoints in this org trained on or before 2026-05-10).
