You can deploy this model and start earning money today!

ilessio-aiflowlab/
project_heimdall

Name: project_heimdall
Availability: InStock

Views

No views yet

HEIMDALL — PROFusion Pose Regression

Paper: PROFusion: Robust and Accurate Dense Reconstruction via Camera Pose Regression and Optimization arXiv: 2509.24236 Authors: Dong, Wang, Cai, Ma, Yang (University of Hong Kong)

Model Description

Dual-stream pose regression network (RGB + Depth) for robust camera pose estimation. Part of the ANIMA robotics perception stack (Wave 6).

Architecture

Backbone: ResNet-18 (dual-stream: RGB encoder + Depth encoder)
Attention: Depth-conditioned channel attention
Decoder: FC layers → quaternion (4D) + translation (3D)
Input: RGB-D frame pairs (224x224)
Output: Relative SE(3) pose matrix

Usage

python

import torch
from anima_heimdall.models.pose_net import PoseRegressionNet

model = PoseRegressionNet(backbone="resnet18")
model.load_state_dict(torch.load("pytorch/heimdall_pose_v1.pth")["model_state_dict"])
model.eval()

rgb_a = torch.randn(1, 3, 224, 224)
depth_a = torch.randn(1, 1, 224, 224)
rgb_b = torch.randn(1, 3, 224, 224)
depth_b = torch.randn(1, 1, 224, 224)

quat, trans = model(rgb_a, depth_a, rgb_b, depth_b)

Training

Trained on TUM-VI sequences with ANIMA Training Standard v1.0. Checkpoint: project_heimdall_cuda_v1_epoch34_val0.0093.pth

Citation

bibtex

@article{dong2025profusion,
  title={PROFusion: Robust and Accurate Dense Reconstruction via Camera Pose Regression and Optimization},
  author={Dong, Siyan and Wang, Zijun and Cai, Lulu and Ma, Yi and Yang, Yanchao},
  journal={arXiv preprint arXiv:2509.24236},
  year={2025}
}

ilessio-aiflowlab/project_heimdall