You can deploy this model and start earning money today!

JayCao99/
dit-diffusion-dinov3-xarm-blue-mug-v0

Name: dit-diffusion-dinov3-xarm-blue-mug-v0
Availability: InStock

Views

No views yet

Multi-Task DiT (diffusion) + DINOv3 ViT-B/16

LeRobot Multi-Task DiT policy checkpoints with a DINOv3 ViT-B/16 vision encoder (text encoder stays CLIP), uploaded by goal_gen/upload_hf_checkpoints_dinov3.sh. Each checkpoint-*/ subfolder is the deployment-ready pretrained_model/ payload (model.safetensors + config.json + pre/postprocessor + train_config.json).

Subfolder	Train step	Final train loss
`checkpoint-100000`	100,000	0.006

⚠️ Loading requires the bundled patch

These checkpoints set vision_encoder_name="facebook/dinov3-vitb16-pretrain-lvd1689m" in config.json, which stock lerobot rejects. This repo ships the runtime monkey-patch multitask_dit_dinov3_patch/ at its root; import it once before from_pretrained and loading works with no gated HuggingFace access (the DINOv3 architecture is built locally and the fine-tuned weights come from the checkpoint).

python

import sys
from huggingface_hub import snapshot_download

# Download the patch package + one checkpoint.
local = snapshot_download(
    "JayCao99/dit-diffusion-dinov3-xarm-blue-mug-v0",
    allow_patterns=["multitask_dit_dinov3_patch/*", "checkpoint-100000/*"],
)

sys.path.insert(0, local)          # make the patch package importable
import multitask_dit_dinov3_patch            # noqa: F401 — applies the patch

from lerobot.policies.multi_task_dit.modeling_multi_task_dit import MultiTaskDiTPolicy
policy = MultiTaskDiTPolicy.from_pretrained(f"{local}/checkpoint-100000")
policy.eval()

For lerobot CLI tools (lerobot-train, lerobot-eval, ...), instead pass --policy.discover_packages_path=multitask_dit_dinov3_patch (with the package on PYTHONPATH).

See multitask_dit_dinov3_patch/README.md in this repo for details.