This repository hosts the pretrained UrbanOmniDetect checkpoints. The full
training and inference code (single-image BEV, real-time video, sliced
high-resolution tracking, and evaluation) lives in the
GitHub repository.
Highlights
Calibration-free 3D detection. A single model predicts 3D bounding box keypoints from a raw RGB image without camera intrinsics, depth estimation, or ground-plane priors.
View-agnostic. One unified architecture works across ego-vehicle, infrastructure, and aerial drone viewpoints.
State-of-the-art on monocular KITTI. AP3D = 30.71 and APBEV = 35.19 on the Moderate split at IoU >= 0.7, outperforming calibration-dependent baselines on Moderate and Hard.
Real-time. Under 11 ms inference on an A100 GPU with TensorRT at 640x640.
Robust. Calibration-dependent methods lose over 80% accuracy with a 5% focal-length error. Our method is invariant by construction.
KITTI Benchmark
Method
AP3D Easy
AP3D Mod.
AP3D Hard
APBEV Easy
APBEV Mod.
APBEV Hard
MonoDGP
30.76
22.34
19.02
39.40
28.20
24.42
MonoCon
26.33
19.01
15.98
34.65
25.39
21.93
MonoLSS
25.91
18.29
15.94
34.70
25.36
21.84
DEVIANT
24.63
16.54
14.52
32.60
23.04
19.99
Ours
29.61
30.71
27.76
33.86
35.19
31.38
Available Checkpoints
Checkpoints are named urbanomnidetect_<backbone><-head>_<imgsz>.pt and cover
YOLOv8, YOLOv9, YOLO11, and YOLO12 across all scales (n / s / m / l / x) and head
configurations (default, P1, P2, P6). The P2 feature-pyramid head is the most
accurate; the YOLO11x-P2 family is recommended.
Checkpoint
Backbone
Input
Use case
checkpoints/urbanomnidetect_yolo11x-p2_1920.pt
YOLO11x + P2
1920
Highest accuracy (recommended)
checkpoints/urbanomnidetect_yolo11x-p2_1280.pt
YOLO11x + P2
1280
Accuracy / speed balance
checkpoints/urbanomnidetect_yolo11x-p2_640.pt
YOLO11x + P2
640
Real-time video and BEV
checkpoints/urbanomnidetect_yolo11n-p2_640.pt
YOLO11n + P2
640
Edge / lowest latency
See the Files and versions tab for the complete checkpoint list.
The model follows the standard Ultralytics prediction API. Point it at any image,
from any viewpoint, and it will predict 3D bounding box keypoints without
requiring camera parameters.
Each detected object has 8 ordered keypoints representing the 2D projections of
its 3D bounding box corners. Indices 0 to 3 are the top corners and indices 4 to
7 are the ground-contact corners.
Bird's-Eye View and Real-Time Video
The bird's-eye view, real-time tracking, and sliced high-resolution pipelines
(draw_bev.py, bev_realtime.py, sahi_tracker.py) are provided in the
GitHub repository. The BEV head
maps ground-contact keypoints to a top-down plane through an
orthogonality-constrained homography, so no camera calibration is needed.
The models are trained on UrbanOmniView,
a 37,446-frame dataset that combines ego-vehicle (KITTI), infrastructure
(DAIR-V2X), and synthetic (UE5) viewpoints. The synthetic UE5 portion is
released as part of this work.
Citation
If you use UrbanOmniDetect or the UrbanOmniView dataset in your research, please cite:
bibtex
1@inproceedings{turkcan2026urbanomnidetect,
2 title = {Calibration-Free View-Agnostic Monocular 3D Object Detection for Urban Scenes},
3 author = {Turkcan, Mehmet Kerem and Gumaste, Devika and Kostic, Zoran},
4 booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
5 year = {2026}
6}
Acknowledgements
This work was supported by the NSF Engineering Research Center for Smart Streetscapes under Award EEC-2133516, NSF Grants CNS-2450567 and CNS-2038984, and by computing resources from the NVIDIA Academic Grant Program and the Empire AI Consortium.
