Intro to AI Assignment 2 — Image-to-Video Semantic Retrieval via Object Detection
(I used AI to help format summary tables listed below)
video_detections.parquet
Has one row per object detection from the provided corpus video sampled at 1 frame per 5 seconds from the 18:39–24:43 segment (exterior car part).
| Column | Type | Description |
|---|
| frame_index | int | Frame number (0-indexed) |
| timestamp | int | Timestamp in original video (seconds) |
| class_label | str | Detected car part (23 possible classes) |
| bounding_box_xmin | float | Left edge of bounding box |
| bounding_box_ymin | float | Top edge of bounding box |
| bounding_box_xmax | float | Right edge of bounding box |
| bounding_box_ymax | float | Bottom edge of bounding box |
| confidence_score | float | Detector confidence (0–1) |
retrieval_results.parquet
One row per query image that maps each query to the best matching video clip.
| Column | Type | Description |
|---|
| query_index | int | Index in aegean-ai/rav4-exterior-images dataset |
| start | int | Clip start timestamp in corpus video (seconds) |
| end | int | Clip end timestamp in corpus video (seconds) |
| class_label | list | Car part classes matched during retrieval |
| number_of_supporting_detections | int | Frames supporting the match |
| embed_url | str | YouTube embed URL with start/end params |
Detector Config/Set Up
- Model: YOLO26n-seg fine-tuned on Ultralytics carparts-seg dataset
- Training: 100 epochs, imgsz=640, T4 GPU (Google Colab)
- Confidence threshold: 0.4
- Frame sampling: 1 frame per 5 seconds
- Video segment: 18:39–24:43 (Exterior car review part)
