Tactus is the tactile sensor pack for Eximius Labs' fusion-embedding stack. It maps a
short window of pressure-array frames (a 32x32 taxel grid, the signal class produced by
resistive/FSR gloves, e-skins, and instrumented robot hands) into the
Qwen3-VL-Embedding-2B text embedding
space, so touch becomes searchable in plain language: recognition is a text query, not a
trained classifier head.
Tactus reads low-dimensional pressure arrays, not optical tactile images. Optical
sensors (GelSight, DIGIT) already have strong models (TVL, UniTouch, Sparsh); the cheap,
widely-shipped resistive arrays did not. To our knowledge Tactus is the first open model
to put this sensor class in a text-aligned, cross-modal embedding space.
Tactus is part of the fusion-embedding family alongside
Tactus Mat
(the same pack trained for a 64x32 body pressure mat),
Tremor (motion) and
Ember (thermal). Its
embeddings target the canonical readout of
fusion-embedding-2, so
tactile windows are directly comparable to that model's text, image, video, and audio in
one 2048-d space, and drop into the Engram
memory layer (pip install engram-robomem) as a first-class sense.
Tactus architecture: calibrated pressure windows pass through an MAE-pretrained per-frame trunk, a learned frame fusion, and a trained projector into the fusion-embedding shared space, where touch becomes searchable in natural language alongside every other modality
Tactus is a trained CNN trunk plus projector over pressure windows. Each 32x32 frame
passes through a ResNet-18-width trunk (3x3 stem, four stages; 32x32 -> 4x4 spatial map);
the K frames of a grasp window are fused by a learned 1x1 convolution over their
concatenated feature maps, pooled, and projected into the frozen base's 2048-d text space.
The trunk is initialized by masked-autoencoder pretraining (mask 0.6, per-patch normalized
targets) on 144k unlabeled STAG-family pressure frames, then fine-tuned contrastively
against canonical text embeddings of natural grasp phrases.
The design choice that matters is the data path: pressure is normalized with the sensor's
own calibration affine (clip((raw - 500) / 150, 0, 1), the STAG reference preprocessing),
and pretraining stays same-sensor. In our ablations, correct normalization and same-sensor
MAE were worth more than every architecture change combined, while cross-sensor pretraining
pooling gave nothing, consistent with published findings (HTT, TacVerse).
2048 (canonical whitened readout; directly comparable across modalities)
Pooling strategy
last-token pooling (text side)
Base model
Qwen/Qwen3-VL-Embedding-2B via fusion-embedding-2-2b-preview (frozen)
Pretraining
same-sensor MAE, 144k frames incl. unlabeled; supervised test frames excluded
Trained components
trunk + conv frame-fusion + projector, 16.2M; shipped as model.safetensors
Distribution
~65 MB trained head; the frozen base downloads from its own repository
See it in action
Real held-out grasps, recognized from pressure alone. Each panel is a genuine STAG test
frame (the most active frame of that class in the held-out split, by total pressure) with
the text query the model matches it against: no camera, no trained classifier head. Across
the full test split the model averages 0.77 top-1 and 0.94 top-3 over 27 such queries.
Four real held-out STAG test pressure maps with their text queries: a mug, scissors, a full can, safety glasses, each recognized from the 32x32 pressure pattern alone
Training and Evaluation
Tactus trains in two stages on the STAG datasets
(Sundaram et al., Nature 2019): a
masked-autoencoder pretrain over every STAG-family pressure frame (classification +
blindfolded + weights + handposes, 144k frames including unlabeled ones, supervised test
frames excluded), then contrastive fine-tuning of the whole head against canonical text
embeddings of grasp phrases, with STAG-style cluster sampling (each training window draws
diverse frames from across a recording rather than consecutive near-duplicates).
Evaluation is 27-way object recognition on fully held-out test recordings, scored as
cosine ranking against text queries (open-vocabulary; the model never trains a classifier
head).
top-1 (27-way)
top-3
recording-level top-1
This checkpoint
0.817
0.951
0.741
Recipe mean (4 independent runs)
0.771 +/- 0.062
0.935
0.722
Training from scratch (no MAE), mean of 3
0.705
0.905
0.691
STAG 2019 supervised closed-set CNN
0.76
-
-
chance
0.037
0.111
0.037
Interpreting these numbers: the recipe's mean exceeds the original paper's supervised CNN
while performing a harder task (open-vocabulary text queries against a frozen language
space, versus a 27-way trained classifier), though by less than one standard error; we
describe the result as matching to exceeding the original baseline, with best runs at
0.83, rather than claiming a definitive margin. Top-3 accuracy is stable across every
run. Our evaluation mirrors STAG's cluster-sampling test protocol but is not their
byte-identical harness. Same-sensor MAE pretraining accounts for about +7 points over
training from scratch. Full recipe, ablations, and negative results: results.json and
the GitHub repository.
Usage
Requirements
torch (CUDA recommended), numpy, safetensors
pip install fusion-embedding[hf] for the text side (the canonical whitened readout
Tactus was trained against; embedding text any other way will misrank)
The frozen base downloads from EximiusLabs/fusion-embedding-2-2b-preview.
via inference.py (this repository)
python
1import numpy as np
2from inference import TactusEmbedder
34ta = TactusEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-tactus",5 revision="v0.1-preview")67# a grasp window: [F, 32, 32] pressure frames (uint8 0-255 or floats in [0, 1]);8# for raw sensor counts pass raw="stag" to apply the calibration affine9window = np.load("grasp.npy")1011for text, score in ta.rank(window,["a mug","scissors","a full soda can","an empty hand"]):12print(f"{score:+.3f}{text}")1314# or embed both sides into the shared space directly15p = ta.embed_pressure(window)# 2048-d, L2-normalized16t = ta.embed_text(["holding a mug"])# canonical text embedding, same space
Pressure embeddings land in the same space as fusion-embedding-2's text, image, video, and
audio, and as Tremor's motion, so cross-modal search over a robot session works out of the
box through Engram. Match text against pressure
through this API rather than embedding text with the raw base model; Tactus was trained
against the canonical whitened readout, and other text paths will misrank.
Related models
Tactus joins the fusion-embedding sense packs, all built on
fusion-embedding-2:
All packs embed into one 2048-d space, so a query can match across senses. The
Engram memory layer (pip install engram-robomem)
wires them into a searchable robot session memory with temporal reasoning.
License
The trained weights in this repository are released under
CC-BY-NC-4.0 (non-commercial).
This reflects the training data's lineage: Tactus is trained on the
STAG datasets, which are released for non-commercial
research use. A commercially-clean retrain (on permissively licensed pressure corpora) is
future work; a commercial license may follow.
Limitations
Run-to-run variance. The training recipe's top-1 varies +/-0.06 across seeds
(0.70-0.83 over four runs). The released checkpoint is a strong draw, and the mean is
reported alongside it. Seed stabilization is active work.
One sensor family. Trained on one glove (STAG's 32x32 grid, 548 taxels). Our
cross-sensor experiments show transfer to other taxel geometries needs fine-tuning, not
zero-shot use; the input path accepts any [F,32,32] window, and other resolutions must
be resampled.
27-object vocabulary at eval. Open-vocabulary means text queries, not tested
open-set generalization to arbitrary unseen object categories; treat novel-category
recognition as unvalidated.
Research preview. Not a production classifier. The intended use is
language-addressable touch inside a multimodal memory, not high-stakes recognition.
English text only, through the canonical readout (fusion-embedding package); do
not embed text with the raw base model.
Citation
If you use Tactus, please cite this repository and the dataset it builds on:
bibtex
1@article{tactus2026,
2 title = {Tactus: Open-Vocabulary Object Recognition from Low-Cost
3 Pressure Arrays},
4 author = {Tonmoy, Abdul Basit},
5 journal = {arXiv preprint arXiv:2608.04043},
6 year = {2026}
7}
Tactus trains on STAG (Sundaram et al., Learning the signatures of the human grasp
using a scalable tactile glove, Nature 2019); please cite that work when using the
benchmark numbers. The text space is Qwen3-VL-Embedding-2B.
