Cross-Person Generalization in Egocentric Video + IMU Activity Recognition

TL;DR

Research Questions

1. Cross-person gap: How much does accuracy degrade from within-person → cross-person evaluation?
2. Modality robustness: Which modality (video vs. IMU vs. fused) degrades less under person shift?
3. Distillation transfer: Does video→IMU distillation (COMODO) improve cross-person robustness?
4. Hard case prediction: Can IMU statistics predict which workers will cause generalization failure?
5. Self-supervised pre-training: Does EVI-MAE pre-training on all workers close the cross-person gap?
6. Scaling laws (new at 500K): How does cross-person accuracy scale from 13K → 50K → 100K → 250K → 500K clips?

Dataset

Baselines

Compute Budget

Timeline

• Weeks 1-4: Pilot validation on 13K clips (fast iteration, pipeline debug)
• Weeks 5-14: Full-scale experiments on 500K clips
• Weeks 15-16: Synthesis and paper write-up

Key References

• COMODO — Video→IMU distillation
• EVI-MAE — Multimodal masked autoencoder
• Generalizable HAR Survey — Cross-person/device/position taxonomy
• IMU-Video OOD HAR — Cross-modal SSL for OOD transfer
• MoBind — Hierarchical IMU-pose alignment

License