Strongly recommended when you know the sensor intrinsics.
--output_resolution_mode
Output behavior for inference_depth.py.
--render_size
Video resolution for inference_gs.py.
--output_ply_dir
Custom output directory for gaussian export.
4. Multi-View / Video Depth + Global Point Cloud (inference_multi_view_depth.py)
Use this when you want sequence-level depth inference from an RGB image folder or video, plus per-frame aligned point clouds and one merged global point cloud. By default the script runs DA3 once on the whole sequence, then aligns each InfiniDepth depth map to the corresponding DA3 depth map before export. When you already know the camera intrinsics and extrinsics, you can instead provide them directly and skip DA3 entirely.
Required inputs
RGB image directory, single RGB image, or video
Sparse depth directory / single file / depth video when --model_type=InfiniDepth_DepthSensor
Required checkpoints / dependencies
checkpoints/depth/infinidepth.ckpt for RGB-only inference
checkpoints/depth/infinidepth_depthsensor.ckpt for RGB + depth sensor inference
checkpoints/moge-2-vitl-normal/model.pt recover metric scale for RGB-only frame inference
depth-anything-3 installed in the current environment when using the default DA3-based sequence mode; default DA3 model is depth-anything/DA3-LARGE-1.1
For video input, replace --input_path with a video file. When --model_type=InfiniDepth_DepthSensor, --input_depth_path can also be a depth video and must contain the same number of frames as the RGB input.
The explicit camera mode expects Waymo-style text files under intrinsics/ and extrinsics/. Files are sorted lexicographically and matched one-to-one against the sorted RGB image list, so the number of camera files must exactly match the number of images. In this mode the script skips DA3 loading, DA3 cache export, DA3 RANSAC conditioning, and DA3 post scale alignment. This mode currently supports image inputs only, not video.
For the RGB-only example above, outputs are written to
example_data/multi-view/waymo/pred_sequence/image/frames/depth/ for aligned raw depth maps
example_data/multi-view/waymo/pred_sequence/image/frames/depth_vis/ for colorized depth maps
example_data/multi-view/waymo/pred_sequence/image/frames/pcd/ for per-frame aligned point clouds
example_data/multi-view/waymo/pred_sequence/image/frames/meta/ for per-frame camera and alignment metadata
example_data/multi-view/waymo/pred_sequence/image/da3/sequence_pose.npz for cached DA3 predictions
example_data/multi-view/waymo/pred_sequence/image/merged/sequence_merged.ply for the merged global point cloud
Depth directory, single depth file, or depth video; required for InfiniDepth_DepthSensor.
--camera_intrinsics_dir --camera_extrinsics_dir
Enable explicit camera mode from sorted Waymo-style txt directories. Image inputs only; file counts must match the RGB frame count.
--input_mode
Force images or video instead of auto detection.
--align_to_da3_depth
Align each InfiniDepth depth map to the corresponding DA3 depth map before export. Ignored in explicit camera mode.
--save_frame_pcd
Save one aligned point cloud per frame.
--save_merged_pcd
Save the merged global point cloud across the whole sequence.
--da3_scale_align_conf_threshold
Minimum DA3 confidence used during per-frame scale estimation.
--output_root
Override the default pred_sequence/<sequence_name>/ output directory.
5. Common Argument Conventions
Argument
Used in
Description
--input_image_path
depth + gs
Path to the input RGB image.
--input_path
multi-view
Path to an RGB image directory, single image, or video.
--input_depth_path
depth + gs + multi-view
Optional metric depth prompt; required for InfiniDepth_DepthSensor. In multi-view mode this can be a depth directory, single depth file, or depth video.
--camera_intrinsics_dir --camera_extrinsics_dir
multi-view
Optional sequence camera parameter directories. When both are set, multi-view inference skips DA3 and uses the provided sorted txt files directly.
--model_type
depth + gs + multi-view
InfiniDepth for RGB-only, InfiniDepth_DepthSensor for RGB + sparse depth.
--depth_model_path
depth + gs
Path to the depth checkpoint.
--gs_model_path
gs only
Path to the gaussian predictor checkpoint.
--moge2_pretrained
depth + gs
MoGe-2 checkpoint used when --input_depth_path is missing.
--fx_org --fy_org --cx_org --cy_org
depth + gs
Camera intrinsics in original image resolution. Missing values fall back to MoGe-2 estimates or image-size defaults.
--input_size
depth + gs
Network input size (H,W) used during inference.
--enable_skyseg_model
depth + gs + multi-view
Enable sky masking before depth or gaussian sampling.
workspace stores experiment outputs under outputs/<task>/<exp_name>/
commonspace stores datasets and pretrained weights shared across experiments
Quick Command Index
If you want ...
Recommended command
Fine-tune from an existing checkpoint
Add ckpt_path=... to the training command
Train from scratch
Omit ckpt_path and use a fresh exp_name
Validate on the mixed real-data benchmark
Run main.py with --entry val
1. Fine-Tuning from an Existing Checkpoint (main.py, default train_net entry)
Use this when you want to initialize training from an existing InfiniDepth checkpoint. The training config referenced below uses Hypersim as the training set and runs validation on a mixed real-data benchmark at the end of each epoch.
${workspace}/outputs/${task}/${exp_name}/checkpoints/ for saved checkpoints
${workspace}/outputs/${task}/${exp_name}/tb/ for TensorBoard logs
Notes
ckpt_path must point to an existing checkpoint.
The training data config is training/exp_configs/components/data/train/infinidepth_train_hypersim.yaml.
If the same exp_name already has checkpoints in ${workspace}/outputs/${task}/${exp_name}/checkpoints/, training will resume from the latest saved checkpoint in that directory.
2. Train from Scratch (main.py, no ckpt_path)
Use this when you want to start training without loading an InfiniDepth .ckpt. In this mode, do not pass ckpt_path. The model will still load the DINOv3 backbone from ${commonspace}/pretrained_models/dinov3/. You need to download the DINOv3 weights yourself and place them there before running.
If you intentionally want to reuse an old exp_name, set resume_training=False to prevent automatic resume. Be careful: when resume_training=False, the code will delete the old output directory before training.
3. Validation from a Checkpoint (main.py with --entry val)
Use this when you want to run validation metrics on the mixed real-data benchmark defined in training/exp_configs/components/data/test/infinidepth_mix_data.yaml.
These datasets are read from ${commonspace}/datasets/ using the meta files referenced in training/exp_configs/components/data/test/infinidepth_mix_data.yaml.
Outputs
${workspace}/outputs/${task}/${exp_name}/val_metrics/ for validation logs
${workspace}/outputs/${task}/${exp_name}/default/metrics/metrics.json for aggregated validation metrics
${workspace}/outputs/${task}/${exp_name}/default/metrics/all_scenes.csv when model.save_metrics=True
4. Common Overrides
Argument
What it controls
ckpt_path
Initialization or evaluation checkpoint path. Omit it for training from scratch.
exp_name
Experiment name used to build ${workspace}/outputs/${task}/${exp_name}.
pl_trainer.devices
Number of GPUs used by PyTorch Lightning.
model.compute_abs_metric
Enable absolute-metric evaluation during training or validation.
model.save_orig_pred
Save original prediction outputs alongside logs and metrics.
model.save_metrics
Save metric files for later inspection.
--entry val
Switch main.py from the default training entry to the validation entry.
--include
Merge an extra data config, such as training/exp_configs/components/data/test/infinidepth_mix_data.yaml.
