ZoeDepth — Metric Monocular Depth (ONNX)
ONNX export of
Intel/zoedepth-nyu-kitti — Intel ISL's metric-depth follow-up to DPT-Large. Same DPT-Large backbone with calibrated metric-bin heads grafted on (one trained on NYU indoor depths, one on KITTI outdoor depths, combined via a domain-routing classifier). Outputs depth in
real-world meters, not just relative ordering.
Re-exported from upstream PyTorch weights — Intel publishes only safetensors at the source repo. Provenance trail: Bhat et al. → Intel/zoedepth-nyu-kitti → transformers.ZoeDepthForDepthEstimation + thin wrapper → torch.onnx.export → these files. fp16 sibling produced from the fp32 trace via onnxconverter-common.
Toolchain:
torch 2.4.x (CUDA 12.4),
torchvision 0.19 (matched ABI),
transformers 4.45.2,
optimum[onnxruntime] 1.24.0,
onnxconverter-common>=1.14, opset 17,
do_constant_folding=True. Full conversion script:
scripts/export-zoedepth.ps1 in the Heliosoph repo (run once for fp32, again with
-Fp16 for the half-precision sibling).
Credit: Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, Matthias Müller (Intel Intelligent Systems Lab). Paper: "ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth", 2023.
What this repo contains
| File | Variant | Size | Use |
|---|
model.onnx | fp32 | ~1.3 GB | Default — full precision, identical numerics to the PyTorch upstream. |
model_fp16.onnx | fp16 | ~660 MB | Half precision — same architecture, ~½ the disk footprint. Identical output on CPU runtimes that upcast fp16→fp32; modest speedup on GPU/NPU with native fp16. |
config.json | — | <3 KB | HuggingFace model config (preserved for re-instantiation if needed). |
preprocessor_config.json | — | <1 KB | ZoeDepthImageProcessor settings — image-size targets, normalization stats. |
The fp32 model.onnx is self-contained — it came in at 1.37 GB, just under the 2 GB protobuf limit, so no external-data .onnx.data sidecar. Same for the fp16 variant at 693 MB.
What "metric depth" means (vs the other depth models on Heliosoph)
| Repo | Output | When to use |
|---|
| Heliosoph/zoedepth-nyu-kitti-onnx (this repo) | Metric depth (meters) | 3D reconstruction at real-world scale, distance measurement, AR overlays, multi-image cloud fusion |
| Heliosoph/dpt-large-onnx | Relative depth | Visualization, single-image effects, "what's closer than what" without needing real units |
| Heliosoph/midas-small-onnx | Relative depth | Edge / CPU / mobile — fast relative depth |
| onnx-community/depth-anything-v2-small | Relative depth (SOTA) | Modern default for relative depth |
Pick this repo specifically when you need meters — most monocular depth models give you a number per pixel that's only meaningful relative to other pixels in the same image; this one gives you a number that's calibrated against real-world distance.
Input / output
| Spec |
|---|
| Input name | pixel_values |
| Input shape | [batch, 3, H, W] (NCHW) |
| Input dtype | float32 (fp32 variant) or float16 (fp16 variant) |
| Constraint | H and W must each be divisible by 32 |
| Preprocessing | Use ZoeDepthImageProcessor from the included preprocessor_config.json — resize + normalize with the ZoeDepth-specific image stats |
| Output name | predicted_depth |
| Output shape | [batch, H, W] (single-channel; no extra channel dim) |
| Output unit | meters (real-world distance from camera to surface) |
| Dynamic axes | batch, height, width |
How to use
1import onnxruntime as ort
2import numpy as np
3from PIL import Image
4from transformers import ZoeDepthImageProcessor
5
6# Use the included preprocessor — it handles the 32-alignment + normalize.
7proc = ZoeDepthImageProcessor.from_pretrained(".")
8sess = ort.InferenceSession("model.onnx") # or "model_fp16.onnx"
9
10img = Image.open("photo.jpg").convert("RGB")
11inputs = proc(images=img, return_tensors="np") # NCHW float32, 32-aligned
12
13depth_meters = sess.run(
14 None,
15 {"pixel_values": inputs["pixel_values"]},
16)[0][0] # [H, W], meters
17
18# depth_meters[y, x] = distance from camera to that surface point, in meters
For the fp16 model, the input also needs to be float16 — cast inputs["pixel_values"] to np.float16 before feeding it in.
Why two variants
- fp32 is the safe default — identical numerics to the upstream PyTorch reference, no surprises.
- fp16 halves disk footprint and model-load memory. On GPU / NPU with native fp16 you also get a modest speedup; on CPU runtimes that upcast fp16→fp32 internally the speed is identical to fp32 but you save the memory. Depth output is essentially identical (the fp16 quantization noise is below the model's own per-pixel error).
If you're not sure: pick fp32 for accuracy-sensitive scientific work, fp16 for shipping / deployment / edge.
License
MIT — same as upstream
Intel/zoedepth-nyu-kitti.
LICENSE file included. The ONNX-export step (and the fp16 numerical conversion) doesn't change licensing — same model, different serialization.
