Tremor is the inertial (IMU) sensor pack for Eximius Labs' fusion-embedding stack.
It maps a short window of body-worn motion (a 3-axis accelerometer stream) into the
Qwen3-VL-Embedding-2B text
embedding space, so a robot's or wearable's motion history becomes searchable in plain
language: "find when it was walking", "find when it picked something up".
Tremor is part of the fusion-embedding family and plugs into the same shared space as
fusion-embedding-2. It
targets the frozen Qwen3-VL-Embedding-2B text embeddings that the family is built on, and
because fusion-embedding-2's text, image, and video paths are byte-identical to that base,
Tremor's motion embeddings are directly comparable to that model's audio, text, image, and
video in one 2048-d space. Motion becomes a first-class, language-addressable modality
alongside the rest.
The family. Each sense is a separately loadable pack over the same frozen base: Tactus reads touch from a 32x32 pressure glove, Tactus Mat reads a 64x32 body pressure mat, Ember reads heat, and Tremor reads motion, with a Unitree-G1 head. Because the base is never modified, adding a sense costs a small trained head and an afternoon of compute rather than a new foundation model.
Model Overview
Tremor architecture: a frozen UniMTS encoder feeds a trained projector into the fusion-embedding shared space, where motion becomes searchable in natural language alongside every other modality
Tremor is a frozen UniMTS motion encoder plus a
small trained projector. A single accelerometer window is placed at one joint of
UniMTS's orientation-invariant skeleton-graph encoder (ST-GCN), which produces a 512-d
motion feature; a projector of roughly 2.6M parameters maps that into the frozen Qwen
base's 2048-d text space. Nothing is trained except the projector. Motion and text
embeddings are L2-normalized and compared by cosine similarity.
The design choice that matters is the encoder: rather than train an IMU encoder from
scratch, Tremor reuses a frozen, orientation-invariant pretrained one and learns only
the projection into the language space. The pretraining prior, not our training, is
what carries cross-subject and cross-mounting generalization.
Feature
Value
Parameters
~2.06B frozen Qwen base + frozen UniMTS encoder; 2.6M trained
~10 MB trained projector; the frozen encoder and base download from their own repos
See it in action
A full activity sequence, read live. A Unitree G1 performs each activity in simulation — sitting, walking, running, climbing — while a window slides across a real, held-out body-worn accelerometer stream (RealWorld, out of training) and Tremor's scores update in real time, tracking the motion as it changes. The robot shows each activity full-body; the signal and the predictions are real, frame-synced data.
A Unitree G1 performs sitting, walking, running and climbing full-body while Tremor's live scores track the activity on a real held-out accelerometer stream
On the robot's own IMU and camera. On a real Unitree G1 humanoid (Humanoid-Everyday), Tremor reads the robot's own IMU as language, live and frame-synced to its egocentric camera — zero simulation. As it switches from adjusting a phone stand to walking to a door, its IMU signature changes from flat to dynamic and Tremor's read tracks the activity in real time.
Live demo on a real Unitree G1: the robot's own egocentric camera with its frame-synced IMU (zero simulation); Tremor's read tracks the activity, from adjusting a phone stand to walking to a door
Any body-worn sensor. No robot is required — the same works on a wristband or wearable. Here the raw accelerometer stream slides by directly and Tremor's scores track sitting → walking → running → climbing on the held-out RealWorld set.
Live classification on a held-out human-activity stream: a window slides across the accelerometer signal and Tremor's activity scores update in real time
Text → motion retrieval. Query an activity in plain language and Tremor returns the nearest motion windows. Below, on the RealWorld set held out of training entirely (zero-shot), each thumbnail is a retrieved window's 3-axis accelerometer trace; correct retrievals (the window's true activity matches the query) are outlined in green.
Text to motion retrieval on held-out RealWorld: each query returns its top-5 nearest accelerometer windows, correct retrievals outlined green
Training and Evaluation
Tremor is trained with InfoNCE against the frozen base's native chat-template text
embeddings, on a diverse 16-source pool of real inertial data: 13 standard
human-activity-recognition datasets, Ego4D (egocentric
human), Humanoid-Everyday (Unitree robot), and
Capture24
(151 subjects of free-living wrist accelerometry). Only the projector learns.
Tremor method: diverse real IMU sources are aligned by InfoNCE to the frozen Qwen language space with only the projector trained, giving zero-shot generalization to unseen datasets
The single most important lesson of the project is the evaluation metric. Instance
retrieval (finding the exact paired window) is misleading for this task and makes a
working activity recognizer look dead. The right metric is k-way activity
classification on held-out subjects and held-out datasets, and all numbers below use
it (chance = 1/k).
Diversity is what makes the base general
Trained on one source (Ego4D alone), an IMU encoder generalizes poorly to sensor
streams that look nothing like it. Tremor's headline result is zero-shot transfer to
datasets held out of training entirely. On five held-out datasets, a single-source
baseline (the identical architecture trained on Ego4D only) sits at or below chance,
while Tremor generalizes.
Trained on 16 IMU sources, Tremor generalizes zero-shot to datasets held out of training entirely, far exceeding a single-source baseline that sits at or below chance
Held-out dataset (5-way, zero-shot)
Single-source baseline
Tremor v0.1
RealWorld
0.16
0.65
USC-HAD
0.13
0.58
DSADS
0.29
0.51
UTD-MHAD
0.21
0.27
held-out mean
0.20 (chance)
0.502
These are the single-sensor numbers the shipped inference.py produces: one accelerometer
window in m/s^2, resampled from the sensor's rate to 20 Hz and placed at one fixed skeleton
joint. That default joint is selected on in-domain held-out data disjoint from the four
zero-shot sets above, so these numbers are not tuned on the benchmark. Feeding the model the
sensor's true sample rate and unit convention is what recovers this accuracy; a rate-agnostic
window scores well below it. (A research configuration that uses each dataset's own
multi-sensor body placement reaches 0.545 mean, but that requires per-dataset sensor-mount
knowledge and is not available to a general single-IMU input.)
On the two in-domain holds (held-out subjects/episodes of sources seen in training),
Tremor reaches 0.40 5-way on both Ego4D and the Humanoid-Everyday robot set. The
held-out datasets above are excluded from training entirely, so those numbers are
genuine cross-dataset generalization.
Point UNIMTS_REPO (or unimts_repo=) at the clone. The UniMTS weights download
automatically from the hub.
The frozen Qwen3-VL-Embedding-2B base downloads from its original repository.
via inference.py (this repository)
python
1import numpy as np
2from inference import TremorEmbedder
34tr = TremorEmbedder.from_pretrained(5"EximiusLabs/fusion-embedding-2-tremor",6 revision="v0.1-preview",7 unimts_repo="UniMTS",# path to your git clone of xiyuanzh/UniMTS8)910# accel: a 3-axis accelerometer window in m/s^2, shape [3, T] (any length)11accel = np.load("window.npy")# e.g. [3, 300]1213# rank candidate activities; pass your sensor's true sample rate (Hz) so the window is14# resampled to the model's 20 Hz. Pass unit="g" if your sensor reports g rather than m/s^2.15for text, score in tr.rank(accel,["walking","sitting","running","climbing stairs"],16 sample_rate_hz=50):17print(f"{score:+.3f}{text}")1819# or embed motion and text into the shared space directly20m = tr.embed_motion(accel, sample_rate_hz=50)# 2048-d, L2-normalized21t = tr.embed_text(["walking","sitting"])# 2048-d each
Give the model the sensor's true sample rate and an accelerometer in m/s^2 (gravity ~9.8);
these are what the held-out numbers above are measured with. Embedding quality is also
sensitive to the base's chat-template formatting; inference.py applies it for you, so match
candidate activities against motion through this API rather than embedding text yourself.
motion is a 3-axis accelerometer window shaped [3, T]. Use text instead to
embed an activity query. Returns 2048-d vectors, so motion and text land in one
space. The Unitree G1 head deploys separately at
the G1 hub page.
Start with this general base — it is the tier that generalizes to sensors it has never seen. Move
to the per-fleet Tremor-G1 head only when you deploy on that platform and want maximum in-domain
accuracy. Specializing a head raises in-domain accuracy but does not improve cross-dataset transfer;
the two do different jobs. Deployment guidance (sensor contract, streaming, on-device inference,
per-fleet fine-tuning) is in the Tremor-G1 card.
The same shared space also has a touch sense:
fusion-embedding-2-tactus embeds
32x32 pressure/taxel arrays (FSR gloves, e-skins, robot hands) and matches to exceeds the STAG
(Nature 2019) supervised baseline while remaining open-vocabulary.
Engram
This pack is one of the modalities Engram searches. Engram is
the open cross-modal memory layer for physical AI: it indexes a robot's video, audio, and motion into
one embedding space and answers questions about it in plain language, including temporal reasoning that
retrieval alone cannot.
The trained projector weights in this repository are released under
CC-BY-NC-4.0 (non-commercial).
This posture is deliberate and reflects the model's lineage. Tremor's frozen encoder is
UniMTS, which was pretrained on synthetic IMU
derived from the AMASS / HumanML3D motion corpora. AMASS
is released under a non-commercial research license, so we ship Tremor non-commercial to
respect that lineage. A commercially-clean encoder (re-pretrained on permissively
licensed motion) is future work; a commercial license may follow.
Limitations
Accelerometer only. The released UniMTS encoder is accelerometer-only; gyroscope
is not used. Rotational-rate information is not currently modeled.
Cross-dataset ceiling. Zero-shot cross-dataset accuracy plateaus around 0.50
(5-way). This is a research preview, not a production activity classifier. The right
use is language-addressable motion search, not high-stakes classification.
Single-sensor, single-joint. One IMU mapped to one skeleton joint, matching a
fixed-mount deployment (a robot torso, a wristband). Multi-sensor fusion is not
modeled.
English text only, and the base's chat-template formatting must be used.
Specialization vs. generalization. Fine-tuning on a specific platform's data
raises accuracy on that platform substantially but does not improve cross-dataset
transfer. Tremor is the general base; per-fleet fine-tuning is a separate recipe.
Citation
If you use Tremor, please cite this repository and acknowledge the frozen components it
builds on:
bibtex
1@misc{tremor2026,
2 title = {Tremor: an inertial-motion sensor pack for the fusion-embedding space},
3 author = {Tonmoy, Abdul Basit},
4 year = {2026},
5 note = {Eximius Labs. Model weights CC-BY-NC-4.0.},
6 url = {https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor}
7}
Tremor's frozen motion encoder is UniMTS (Zhang et al., UniMTS: Unified Pre-training
for Motion Time Series, NeurIPS 2024), pretrained on synthetic IMU derived from
AMASS (Mahmood et al., ICCV 2019). The text space is Qwen3-VL-Embedding-2B.
Please cite those works as well.
