You can deploy this model and start earning money today!

LabProc/
tacit

transformers

video-classification

self-supervised-learning

v-jepa

laboratory-procedures

domain-adaptation

cc-by-4.0

endpoints_compatible

Views

No views yet

Tacit

Tacit is a domain-adapted V-JEPA-2.1 video encoder for laboratory procedure understanding. It is the trained companion to the LabProc benchmark, released as part of our NeurIPS 2026 Evaluations & Datasets Track submission.

Benchmark dataset: Labproc/labproc
Code & evaluation harness: tacit-anon/labproc
License: CC BY 4.0

Model Details

Architecture: Vision transformer with 24 layers, 16 attention heads, hidden size 1024, MLP ratio 4. Patch size 16, image size 384×384. Uses RoPE positional encoding with interpolation, supporting variable frame counts at inference (we use 16 frames per clip at evaluation time).
Total parameters: ~300M (full encoder); 37.8M trainable during adaptation (12.4%, last 3 of 24 transformer blocks).
Output: 1024-dimensional clip-level features after mean-pooling across spatiotemporal patch tokens.
Base model: V-JEPA-2.1 ViT-L distilled from ViT-G at 384×384, released by Meta FAIR.
Adaptation: EMA target encoder (τ=0.996) + motion-conditioned masking (ratio 0.75).
Released checkpoint: Epoch 4 of a 7-epoch adaptation run; training loss 0.70.

Intended Use

Primary intended uses: Tacit is intended as a calibration target for laboratory video understanding research — producing frozen visual features for laboratory procedure clips, to be used as input to downstream linear or shallow probes. It is also intended for comparison against future video encoders, larger adaptation runs, parameter-matched open-weight VLMs, and v2 PCR/Western-blot benchmark instantiations.

Out-of-scope uses:

Production deployment in laboratory safety, quality assurance, or regulatory compliance settings without substantial additional validation. The Strict Hard accuracy of 66.7% is a research-grade signal, not a deployment-grade reliability target.
Use on non-laboratory video content. Adaptation on laboratory video specifically reshapes the representation toward this domain.
Use as a foundation for behavioral or biometric inference from laboratory operator footage.
Same-State CCR evaluation (within-state temporal ordering). The released checkpoint's adaptation pipeline attenuates within-state temporal coherence.

Training Details

Training data: laboratory procedure videos collected via the three-stage filtering pipeline described in the LabProc paper, spanning organic purification, polymerase chain reaction, and Western blot procedures. v1 LabProc benchmark evaluates only on the organic purification subset; the adaptation set spans all three branches.
Optimizer: AdamW
Learning rate: 5×10⁻⁶, cosine schedule
Weight decay: 0.01
Batch size: 4
Frames per clip (training): 64
Frames per clip (inference): 16
Mask ratio: 0.75
EMA momentum: 0.996
Epochs: 7 (released: epoch 4)
Precision: FP16 mixed-precision
Trainable parameters: 37.8M (last 3 of 24 transformer blocks)
Compute: 28 minutes single-H100-80GB wall-clock; ~$1.30 in rented compute.

Evaluation

Headline LabProc v1 benchmark results (full table in the paper):

Task	Random	Base V-JEPA-2.1	Tacit (ep4)	Claude Opus
PSC-10 (10-class state)	10.0	16.2	31.2	72.2
TED visual+text (4-MCQ)	25.0	75.3	76.1	82.4
CCR pairwise	50.0	43.9	58.7	67.0
VSD aggregate	50.0	50.2	57.8	73.9
TED-V Hard	50.0	60.9	69.6	67.4
TED-V Strict Hard	50.0	60.6	66.7	57.6

Tacit leads Claude Opus on the two motion-discrimination subsets (TED-V Hard, Strict Hard) where vision-language models are structurally insufficient.

How to Use

Install the evaluation harness and use the encoder:

bash

git clone https://github.com/tacit-anon/labproc
cd labproc
pip install -e .

python

import torch
from labproc_tacit.encoder import build_encoder, load_checkpoint

# Load Tacit checkpoint
encoder = build_encoder(model_name="vit_large", patch_size=16, image_size=384)
load_checkpoint(encoder, "tacit_ep4.pt")  # downloaded from this HF repo
encoder.eval().cuda()

# Encode a clip
with torch.no_grad():
    features = encoder(clip)  # clip: (B, T=16, C=3, H=384, W=384)
    pooled = features.mean(dim=1)  # (B, 1024) clip-level embedding

See the GitHub repo for full evaluation scripts and benchmark reproduction.

Limitations

Domain bias: heavily skewed toward English-language YouTube laboratory content. Likely systematic features that may not transfer to industrial laboratories, foreign-language workflows, or atypical equipment.
Operator bias: although adaptation downweights operator-specific signal via motion masking, operators are visible in nearly every frame.
Adaptation-induced trade-off: Tacit's adaptation attenuates within-state temporal coherence by ~0.14 in τ vs the V-JEPA-2.1 base. Users requiring both within-state ordering and cross-state recognition will need a different adaptation strategy.
Single-annotator ground truth for PSC, CCR, and VSD-aggregate evaluation labels.
Modest adaptation scale relative to general video pretraining (1M+ hours for V-JEPA-2.1).

See the paper Section 8 ("Limitations") for the complete discussion.

Citation

bibtex

@inproceedings{labproc2026,
  title  = {LabProc and Tacit: A Benchmark and Domain-Adapted
            Video Encoder for Laboratory Procedure Understanding},
  author = {Anonymous},
  booktitle = {NeurIPS 2026 Track on Datasets and Benchmarks},
  year   = {2026}
}

LabProc/tacit