astertech-cvd-coach-lora-r0
LoRA adapter for Qwen/Qwen2.5-VL-7B-Instruct that performs streaming step-completion
detection on egocentric video of CVD (chemical vapour deposition) tube-furnace
experiments. At each tick the model is shown the last 30 s of footage plus the current
instruction and answers a Yes/No question about whether that step is now complete, so a
coach can acknowledge the step and advance.
This is round 0 of the AsterTech-vLLM project (milestone M3). Completion detection
only — mistake detection is deliberately out of scope.
Provenance and access
Trained on local-only laboratory footage that is not distributed and never leaves the
project's own machines. The weights here are derived from that footage, which is why this
repository is private. Treat any change of visibility as a deliberate decision about
derived data, not a routine toggle.
Training recipe
Reproduced verbatim from results/m3/pooled_provenance.json (seed 0, deterministic
inputs: hash-pinned annotations + frozen benchmark manifest benchmarks/v1.json).
| |
|---|
| Base model | Qwen/Qwen2.5-VL-7B-Instruct (bf16) |
| LoRA | r=16, α=32, dropout=0.05 |
| Optimiser | lr 1e-5, batch 1 × grad-accum 8, 1 epoch, seed 0 |
| Frames | 2 fps, ≤16 frames/example, 30 s clip window, max_pixels 151200 |
| Pair generation | completion_pairs_from_gt(short_yes=True), neg_per_step=4, neg_margin_s=8, min_step_s=12 |
| Dataset | 672 examples — 448 Yes / 224 No (56 neg-eligible steps ×5 + 392 yes-only + 3 zero-length skips) |
| Train videos | the 13 pool videos of benchmark v1 (the 3 frozen holdout videos are never trained on) |
Training data is same-step contrastive pairs: for each annotated step one "Yes" window
ending at the completion time and N "No" windows ending mid-step, with identical
instruction text and window length, so only the pixels differ. Every example comes from
one code path over the project's own ground truth — no external corpora, no mixed sources.
Results (M3, benchmark v1)
Evaluated at the frozen v1 eval config (tick 5 s / window 30 s / 2 fps / ≤16 frames /
confirm 1; match window 30 s full width).
| split | zero-shot IC-Acc | SFT IC-Acc | delta | missed | overtalk |
|---|
| pool (LOVO, 13 folds) | 14.63% (66/451) | 48.78% (220/451) | +34.15 | 0.854 → 0.512 | 0.283 → 0.315 |
| frozen holdout (this pooled adapter) | 20.48% (17/83) | 54.22% (45/83) | +33.74 | 0.795 → 0.458 | 0.292 → 0.262 |
The holdout videos were never trained on in any run, and gained as much as the LOVO folds
— which is what rules out "it memorised the training videos".
Slice movement on the pool (zero-shot → SFT): short steps 8/80 → 49/80, pressure waits
1/64 → 18/64, other 57/311 → 154/311.
About this copy — a reproduction, not the original artifact
The original M3 pooled adapter was written to instance-store disk and lost when the
training box was stopped. These weights are a re-run of the identical recipe on
2026-08-04: same committed annotations, same hash-pinned manifest, same 13 train ids,
seed 0, same hyperparameters. The reproduction is exact on every input we can check —
the dataset came out at 672 examples / 448 Yes / 224 No, matching the original pin
byte-for-byte, and training took 66.9 min against the original's 67.0 min, ending at
train_loss 0.4088.
GPU nondeterminism means the weights are not bit-identical to the originals, so the table
above is the original run's measurement. See "Verification" below for this copy's own
measured holdout score.
Caveats — read these before trusting it
- Overtalk rose on the pool (0.283 → 0.315). The 2:1 Yes bias buys detections partly
through more emissions. Holdout overtalk improved (0.292 → 0.262), so this is not a
degenerate always-yes model, but it is a trade. The next lever is harder mid-step
negative windows.
- The pressure-wait gain is not gauge reading. The DigiVac LCD is provably illegible
at the model's 360×420 effective input, so the model is learning the operator's
reaction cue (stillness, then reaching for the next valve), not the displayed value.
Do not present it as instrument reading.
- Small dataset. 16 videos / 534 completions total. Single-video scores (n = 21–49)
are noise; only the aggregates above carry signal.
Verification
These exact weights were re-scored on the 3 frozen holdout videos (2026-08-04), loaded
from this repository by id — so the number below also verifies the download path, not just
the local files.
| metric | original M3 adapter | this copy |
|---|
| IC-Acc | 54.22% (45/83) | 57.83% (48/83) |
| missed | 0.458 | 0.422 |
| overtalk | 0.262 | 0.238 |
| mean signed offset | — | +1.02 s (abs 4.19 s) |
+3.61 pts against a ±5 pt reproduction gate — pass. The difference is GPU nondeterminism
at n=83 (3 completions), not a better model; both are the same recipe and the original
table is the one to quote. Zero-shot on the same 3 videos is 20.48% (17/83).
Usage
1from peft import PeftModel
2from transformers import AutoModelForVision2Seq
3
4base = AutoModelForVision2Seq.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", dtype="bfloat16")
5model = PeftModel.from_pretrained(base, "Rnoooo/astertech-cvd-coach-lora-r0")
Requires authentication (HF_TOKEN) while the repository is private. The adapter expects
the exact Yes/No completion prompt shape it was trained on — the project's
astertech.prompts.completion_messages. Using a different prompt string collapses recall;
that was the central bug of an earlier round.
