You can deploy this model and start earning money today!

tue-mps/
videomt-dinov2-large-vipseg

Name: videomt-dinov2-large-vipseg
Availability: InStock

video-panoptic-segmentation

Views

No views yet

VidEoMT-L on VIPSeg

This repository contains the Hugging Face Transformers conversion of the official VidEoMT checkpoint vipseg_vit_large_55.2.pth from tue-mps/VidEoMT.

Model details

Architecture: VidEoMT with a DINOv2 ViT-L/14 with 4 register tokens backbone
Task: video panoptic segmentation
Dataset: VIPSeg
Input resolution: 1280 x 1280
Number of frames: 2
Paper: Your ViT is Secretly Also a Video Segmentation Model

Reported metrics

Metric	Value
VPQ	55.2
STQ	48.9
FPS	75

The metrics above are the numbers reported by the authors in the official model zoo.

Usage

python

from transformers import AutoModelForUniversalSegmentation, AutoVideoProcessor

model_id = "tue-mps/videomt-dinov2-large-vipseg"
processor = AutoVideoProcessor.from_pretrained(model_id)
model = AutoModelForUniversalSegmentation.from_pretrained(model_id)

Use processor.post_process_instance_segmentation, processor.post_process_panoptic_segmentation, or processor.post_process_semantic_segmentation depending on the target task.