You can deploy this model and start earning money today!

AbstractPhil/
qwen3.5-0.8b-relay-caption

Name: qwen3.5-0.8b-relay-caption
Availability: InStock

Views

No views yet

RelayPatchwork Caption Adapter for Qwen3.5-0.8B

A 6.26M-parameter caption adapter for the frozen Qwen/Qwen3.5-0.8B vision-language trunk: one RelayPatchwork block per LLM-tower decoder layer (24 × ~261k params), each reading the block's hidden states through a 16-slot aleph address — a closed-form sinh/cosh soft read over a 64-atom S³ codebook — and adding a gated squared-ReLU patch. The trunk is untouched: remove the wrappers and the base model is restored exactly (verified bit-exact in the parent research series).

Numbers (all held-out, greedy decoding)

config	token-F1	terminate	nonempty
frozen trunk	0.408	1.0	1.0
+ adapter (seed 0)	0.706	1.0	1.0
+ adapter (seed 1)	0.704	1.0	1.0

Seed stability: |s0 − s1| = 0.0019 token-F1 — the +0.30 delta replicates across independently trained seeds.
Artifact validation: the checkpoint in this repo was re-downloaded fresh from the hub and re-judged (gate PASS, F1 reproduced exactly); the standalone loader below was separately fidelity-tested against the research stack on the same held-out set.

Known trade-off (measured, e004e): the adapter is always-on, and with it loaded the trunk's other structured-JSON vision behaviors degrade — most sharply, depth-comparison JSON validity drops from 1.0 (bare gauge) to 0.0 (n=12, same items/process). Use this adapter for captioning; detach it (drop the wrappers) for structured vision tasks. This is the documented format-anchor trade-off of the parent research series, not an incidental bug.

Trained on ~4k fashion image–caption pairs (AbstractPhil/qwen-deepfashion-fused); the eval slice is held out from training. Token-F1 is bag-of-words F1 against the ground-truth caption — a distribution-match measure, not a general captioning benchmark. The base instruct model already captions fluently; what the adapter buys is the target caption style and coverage (garments, colors, materials, pose, setting).

Usage

python

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from relay_caption import attach, caption

model = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3.5-0.8B", torch_dtype=torch.float32).cuda()
proc = AutoProcessor.from_pretrained("Qwen/Qwen3.5-0.8B")
attach(model, "v35e4_caption_full_s0.pt")
print(caption(model, proc, "photo.jpg"))

relay_caption.py (in this repo) is self-contained: torch + transformers only. attach wraps the 24 decoder layers and loads the adapter; caption runs the trained prompt format greedily to <|im_end|>.

Files

relay_caption.py — standalone loader (fidelity-tested).
v35e4_caption_full_s0.pt — the primary adapter (seed 0).
v35e4_caption_full_s1.pt — the replication seed.

Provenance

Produced by the geolip-aleph research series on this trunk; the full experimental record (training bed, judges, cross-seed and held-out validation, amendments) is public at AbstractPhil/geolip-aleph-qwen-3.5-0.8b-instruct (exp004_caption/, plus the v35_e004b/c/d validation rows). Training riders: fp32, TF32 off, pure Adam (no weight decay), frozen trunk, loss on caption tokens only.

AbstractPhil/qwen3.5-0.8b-relay-caption