You can deploy this model and start earning money today!

ThuongBuiRVC/
Traffic-JEPA

safetensors

traffic-safety

video-question-answering

Qwen/Qwen3-VL-8B-Instruct

finetune

mit

Views

No views yet

Traffic-JEPA — checkpoints

Checkpoints for Traffic-JEPA, the 1st place solution to the 10th AI City Challenge (2026), Track 2: Transportation Safety Understanding and Captioning (team Latent Painter – UTE, S2 60.0853, VQA accuracy 87.09 %).

Code: https://github.com/ThuongBuiRVC/Traffic-JEPA

Files

File	Stage	What
`model_best.pt`	VQA	the trained predictor head (5.83 M params) on top of frozen V-JEPA 2.1, Llama-3.2-1B and EmbeddingGemma
`caption_lora/`	Captioning, default	LoRA (rank 16, alpha 32) for `Qwen/Qwen3-VL-8B-Instruct`, trained text-only on SynWTS: facts -> caption. This is the configuration the paper reports.
`caption_lora_mm/`	Captioning, variant	the same LoRA shape trained multimodally — SynWTS facts and simulation frames -> caption. Used by the grounded variant, which also shows real frames at inference.

Usage

bash

git clone https://github.com/ThuongBuiRVC/Traffic-JEPA
cd Traffic-JEPA
hf download ThuongBuiRVC/Traffic-JEPA --local-dir checkpoints/

bash scripts/05_submit_test.sh     # VQA     -> submissions/submission_final.json
bash scripts/06_caption.sh         # caption -> submissions/caption_submission.json
bash scripts/06_caption.sh mm      # grounded variant -> submissions/caption_submission_mm.json

The VQA model is rebuilt from run_args.json, which scripts/05_submit_test.sh writes next to model_best.pt from configs/train_args.json. A mismatched config changes the scores without raising anything, so do not swap it by hand.

Training data

The VQA predictor is trained on SynWTS simulation questions; the decoder tables are estimated from simulation counts. The caption LoRA is trained on SynWTS captions only. Neither uses test ground truth.

Citation

bibtex

@inproceedings{bui2026trafficjepa,
  author    = {Nguyen Hoai Thuong Bui and Thanh Nguyen Vo and Trinh Tra Giang Nguyen and Ha Duc Bui},
  title     = {Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA},
  booktitle = {AI City Challenge Workshop},
  year      = {2026}
}