A Video Swin Transformer Base backbone with an I3D head trained to recognize 5 sequential actions in a clothes-folding task from top-down (cam_high) video. This is the best-performing checkpoint from the project's experiment series.

Model family: Video Swin Transformer (Base) + I3D Head
Task: 5-class video action recognition
Framework: mmaction2 (v0.15.0 fork)
Best checkpoint: epoch_50.pth → Test Top-1 0.9471

Model Overview

The model classifies a short video clip into one of 5 clothes-folding actions. It follows the standard two-stage video architecture: a 3D-tokenized Swin Transformer backbone produces spatio-temporal features, and an I3D-style head pools them into a per-class score.

Model Details

Component	Configuration
Backbone	Video Swin Transformer Base (`depths=[2,2,18,2]`, `embed_dim=128`, `num_heads=[4,8,16,32]`)
Patch size	`(2, 4, 4)` (2 temporal × 4×4 spatial)
Window size	7×7
Drop path	0.3
Backbone init	ImageNet-22K pretrained (`swin_base_patch4_window7_224_22k.pth`)
Head	I3DHead, `in_channels=1024`, `num_classes=5`
Input clip	48 frames × 224×224 (`clip_len=48`, `frame_interval=1`)

Class Labels

Label ID	Name
0	`flattening`
1	`frist_fold` *
2	`second_fold`
3	`third_fold`
4	`retraction`

* frist_fold is kept as-is to match the original dataset annotation spelling.

Intended Use & Limitations

Intended use: recognizing the 5 sequential stages of a clothes-folding action from top-down monocular video (e.g. a robot or human folding a garment in front of a high-mounted camera).

Limitations:

Trained only on cam_high (top-down) viewpoints of a single cloth-folding setup.
Not tested on other camera angles, backgrounds, garment types, or unseen folding styles.
Data is class-balanced per split but comes from a single recording environment.

Training Data

Dataset: clothes_fold_5 — 742 episodes / 3,707 clips, split train:val:test = 12:1:1.

Split	Episodes	Clips
train	636	3,177
val	53	265
test	53	265
Total	742	3,707

Labels are nearly class-balanced (≈636 train / 53 val / 53 test per class). Source data: top-down (cam_high) RGB video segmented per action.

Training Procedure

Configuration: 005_clip48_lr8e4_wd005_drop03.py

Hyperparameter	Value
Optimizer	AdamW, lr = 8e-4, betas (0.9, 0.999), weight_decay = 0.05
LR schedule	CosineAnnealing (min_lr 0), linear warmup 3 epochs
Backbone LR	×0.1 of base LR
Total epochs	50
Batch	8 clips / GPU
FP16	disabled
Augmentation (train)	RandomResizedCrop, Flip
Augmentation (val/test)	Resize(−1,224) + Center/ThreeCrop(224)

Implementation note: the mmaction2 DistOptimizerHook (gradient accumulation) is used with native fp32; mmdet fp16 is intentionally disabled for this Swin config.

Evaluation

Test protocol: num_clips=4 + ThreeCrop, average_clips=prob, max_testing_views=2.

Best Checkpoint (`epoch_50.pth`)

Metric	Value
Test Top-1 Accuracy	0.9471
Test Mean Class Accuracy	0.9560
Test Top-5 Accuracy	1.0

Comparison with Baselines (test Top-1)

Experiment	Best Test Top-1	Test Mean Acc
001 single-GPU baseline	0.9231	0.9367
002 8-GPU baseline	0.9231	0.9352
003 8-GPU medium batch	0.9183	0.9313
005 (this model)	0.9471	0.9560

Key findings from training:

Longer clips (clip_len 48, frame_interval 1) + lower LR (8e-4) + weight_decay 0.05 + drop_path 0.3 give ~+2.4% over the baselines.
Test accuracy exceeds val for all runs, so val should not be the sole model-selection signal.

How to Get Started

Below is an example using the mmaction2 test script. Place the downloaded checkpoint at the path given in --checkpoint (e.g. work_dirs/005_clip48_lr8e4_wd005_drop03/epoch_50.pth).

bash

# Single-GPU inference / evaluation with a config + checkpoint
python tools/test.py \
  configs/recognition/swin/005_clip48_lr8e4_wd005_drop03.py \
  work_dirs/005_clip48_lr8e4_wd005_drop03/epoch_50.pth \
  --out result/epoch_50.pkl \
  --eval top_k_accuracy mean_class_accuracy

Data preprocessing / reproduction is handled by tools/prepare_clothes_fold_dataset.py; the label map lives in demo/label_map_clothes_fold.txt.

Files in This Repository

File	Description	Recommended
`epoch_50.pth`	Best checkpoint (0.9471)	upload
`epoch_45.pth`	0.9375	optional
`epoch_40.pth`	0.9327	optional
baseline `epoch_30.pth` (001/002/003)	0.9231 / 0.9231 / 0.9183	optional
`005_clip48_lr8e4_wd005_drop03.py`	Config for reproduction	upload

Checkpoints are full training states (~1.1 GB each) and are uploaded alongside this card.

Citation / Acknowledgements

If you use this model in research, please cite:

bibtex

@misc{clothesfold-videoswin,
  title  = {Video Swin-Base for 5-Class Clothes-Fold Action Recognition},
  author = {CaoJiahao2},
  year   = {2026},
  note   = {mmaction2 (v0.15.0 fork) fine-tune of ImageNet-22K Swin-Base},
  howpublished = {\url{<this HF repo URL>}}  % TODO
}

Backbone pretraining: Liu et al., "Video Swin Transformer" / Swin Transformer (ImageNet-22K). Frameworks: mmaction2, Swin Transformer.

Jiahao28/video-swin-base-clothes-fold-5class

Video Swin-Base for Clothes-Fold 5-Class Action Recognition