Technical Report · Project Website · Code · OpenNeoData

Tactile hardware is fragmented, so a model built on one signal format is bound to the device that produced it. NeoForce instead describes every tactile observation as a dense three-axis force field over the sensing surface, capturing shear and pressure in a form that is physically grounded and shared across sensors, and learns a temporally structured representation on top of it.

The model is a joint visual-tactile ViT: RGB frames and dual-sensor force fields are patched onto one shared token grid and run through a single trunk, so the modalities attend to each other directly instead of being fused after the fact. A temporal transformer over per-frame CLS pairs supplies context, and a progressive decoder reconstructs the force field and contact mask at the full input resolution, supervised at every scale it passes through.

This repository holds the weights. The code, together with the data preparation, training and evaluation guides, lives in the code repository.

The trained NeoForce model at step 100,000, kept as a full training state rather than inference weights alone:

The frozen tactile conversion model, which turns raw tactile camera frames into the force field NeoForce consumes:

gel image -> bird-view correction -> optical flow based on the reference frame
          -> conversion network -> masked, scaled force field

Each file has a fixed path the code looks in:

The Installation Guide covers the rest of the setup.

The six force channels are [left fx, fy, fz, right fx, fy, fz], one triplet per tactile sensor.

Trained on OpenNeoData, the 5,000-hour open-source subset of NeoData.