DA-2 WebGPU Port

🔗 Original Work

• Repository: EnVision-Research/DA-2
• Paper: arXiv:2509.26618
• Project Page: depth-any-in-any-dir.github.io

🚀 WebGPU Demo

Prerequisites

• Python 3.10+ (for model export)
• Web Browser with WebGPU support (Chrome 113+, Edge 113+, or Firefox Nightly).

Installation

1. Clone the repository:
   bash

   1git clone <your-repo-url> 2cd DA-2-Web
2. Set up Python environment:
   bash

   1python3 -m venv venv 2source venv/bin/activate # On Windows: venv\Scripts\activate 3pip install -r requirements.txt

Model Preparation

1. Download the model weights: Download model.safetensors from the HuggingFace repository and place it in the root directory of this project.
2. Export to ONNX: Run the export script. This script handles the conversion to FP16 and applies necessary fixes for WebGPU compatibility (e.g., replacing clamp with max/min).
   python export_onnx.py
   This will generate da2_model.onnx.
3. Merge ONNX files: The export process might generate external data files. Use the merge script to create a single .onnx file for easier web loading.
   python merge_onnx.py
   This will generate da2_model_single.onnx.

Running the Demo

1. Start a local web server: You need to serve the files over HTTP(S) for the browser to load the model and WebGPU context.
   python3 -m http.server 8000
2. Open in Browser: Navigate to http://localhost:8000/web/ in your WebGPU-compatible browser.
3. Usage:
   • Click "Choose File" to upload a panoramic image.
   • Click "Run Inference" to generate the depth map.
   • The process runs entirely locally on your GPU.

🛠️ Technical Details of the Port

• Precision: The model was converted to FP16 (Half Precision) to reduce file size (~1.4GB -> ~700MB) and improve performance on consumer GPUs.
• Opset: Exported using ONNX Opset 17.
• Modifications:
  • The SphereViT and ViT_w_Esphere modules were modified to ensure strict FP16 compatibility.
  • torch.clamp operations were replaced with torch.max and torch.min combinations to avoid Clip operator issues in onnxruntime-web when handling mixed scalar/tensor inputs.
  • Sphere embeddings are pre-calculated and cast to FP16 within the model graph.

📄 License