Laguna-S-2.1 with sight. A vision-language model that attaches the Qwen3-VL vision
tower to poolside's Laguna-S-2.1 through a trained projector.
Laguna-S-2.1 is a strong code and reasoning model with no image input. This
checkpoint adds it without touching a single Laguna weight. Both the text backbone
and the vision tower are frozen and byte-identical to their upstream releases. The
only newly trained parameters are the 35.4M projector that maps Qwen3-VL's
1152-dim patch embeddings into Laguna's 3072-dim token space.
Component
Detail
Text backbone
Laguna-S-2.1, 117B total / 5.3B active, MoE, frozen
Vision tower
Qwen3-VL, 27 layers, 1152-dim, patch 16, frozen
Projector
norm -> 2x2 concat -> linear_fc1 -> GELU -> linear_fc2, 1152 to 4608 to 3072, trained
Every benchmark ran twice: once normally, once with the image features replaced by
Gaussian noise of the same shape and scale. The gap between the two is the
measurement. 300 questions per benchmark, per arm.
benchmark
sighted
blind
gap
chance
MMMU-Pro
38.33%
13.00%
25.33
25%
MMMU_DEV_VAL
45.33%
20.00%
25.33
25%
MMBench_DEV_EN
73.67%
25.67%
48.00
25%
SEEDBench_IMG
68.00%
19.00%
49.00
25%
HallusionBench
51.67%
4.33%
47.34
50%
TextVQA_VAL
74.00%
2.67%
71.33
n/a
OCRVQA
43.67%
6.00%
37.67
n/a
Read the chance column before the sighted column.
HallusionBench is yes/no, so its floor is 50%. A sighted score of 51.67% is at
chance. The 47-point gap says the model is using the image; the absolute score says
it is not yet resisting the trap questions. See next steps.
TextVQA and OCRVQA are open-ended and scored here by normalised containment,
which is looser than official VQA accuracy. Treat those two rows as internal
measurements, not leaderboard-comparable numbers.
Blind scores below chance are expected. Noise in the image slots is worse than
no image at all, because the model still attends to the slots.
An earlier checkpoint scored higher. Step 900, one epoch, measured 41.0%
sighted and 13.67% blind on MMMU-Pro, gap 27.33. It was deleted by a retention
rule before assembly, so these weights are step 2070. See limitations.
The blind arm is the point of the table. Sighted accuracy on its own cannot separate
a model reading images from a language model answering off the question wording. In
our own experiments a different graft scored 41.67% sighted on MMMU-Pro and 40.0%
blind, a gap of 1.67 points. It was not using the images at all, and its headline
number was indistinguishable from this model's.
Limitations
This is not a natively trained VLM. The language model never saw an image
during its own training. It receives features translated into its embedding space
by a small MLP. Models trained end to end on multimodal data score considerably
higher on the same benchmarks.
The released checkpoint is supervised finetuning only. No reinforcement
learning stage is included in these weights.
Trained on short answers. The training mixture is short-form visual QA with
captions excluded, because long targets are off-policy for a frozen backbone and
carry high loss regardless of how well the features are aligned. Long-form
description and free-form generation about images were never trained and should be
expected to be weak.
Partial warm start. The Qwen3-VL merger outputs 4096 and Laguna's hidden size
is 3072, so linear_fc2 was randomly initialised. Only 4 of 6 projector tensors
transferred.
Hallucination is the clearest weakness. At-chance HallusionBench means the
model reads the image but does not reliably contradict a false premise in the
question.
Fine text is limited by image token budget. Patches are merged 2x2 before the
projector, so dense documents and small print lose resolution before the language
model ever sees them.
These are the second-epoch weights, and the first epoch was better. At step
900 (57,600 samples) training loss was 1.30 and MMMU-Pro was 41.0% sighted, gap
27.33. At step 2070 (132,480 samples) loss was 0.61 and MMMU-Pro was 38.33%,
gap 25.33. A keep-last-3 retention rule deleted step 900 before the artifact was
assembled, so the released projector is step 2070. Every number in the table
above was measured on these exact weights.
That 2.67-point difference is not statistically established. At p near 0.40
and n=300, one arm carries a standard error of 2.83 points and an unpaired
difference of two arms carries 4.0 points, so the observed gap sits inside one
standard error of zero. Both arms ran the same 300 questions and a paired test
would have smaller variance, but the per-item outputs and every checkpoint were
deleted with the training volume, so it cannot be run. Treat "one epoch was
better" as unmeasured rather than as a small measured effect. What the run does
show is that training loss halved while held-out accuracy did not improve, so
loss is not tracking the thing you care about.
Next steps
Reinforcement learning, at a scale that can be measured. GRPO on the projector
with verifiable rewards is the obvious next stage, and it is cheap because only 35.4M
parameters move. Our own 10-step probe is not evidence either way: step-to-step
reward variance was std 0.024 on a mean of 0.122, wider than anything 10 steps could
produce. A real run needs hundreds of steps before the benchmark suite can settle it.
Unfreeze the language model. Everything here holds Laguna byte-identical, which
is what makes the text behaviour guarantee below true. Relaxing that, even to LoRA on
attention projections only, is the largest available gain and the point at which the
model stops being a drop-in replacement for the base weights.
Raise the image token budget. Dropping the 2x2 merge quadruples image tokens and
should move TextVQA and OCRVQA most. It costs sequence length, so it trades against
throughput rather than against quality.
Train against hallucination directly. HallusionBench sits at its 50% floor. That
needs contrastive data where the question's premise is false, which the current
short-answer mixture contains almost none of.
More alignment data. 57,600 samples is small. The gap was still moving between
checkpoints when we stopped, which means the curve had not flattened.
What this would not fix. No amount of projector training makes a frozen text
model reason about images the way an end-to-end multimodal model does. The projector
translates; it does not teach. The ceiling here is set by what Laguna can already do
once the content arrives in its embedding space.
Text behaviour is unchanged, structurally
With no image tokens in the prompt, the forward pass is bit-identical to stock
Laguna-S-2.1. Every backbone weight is frozen, the projector is not in the text
path, and the added placeholder id never appears. This is a property of the
construction, not a benchmark result.
What is in this repository
mm_projector.safetensors 35.40 M the only trained part
vision_tower.safetensors 576.39 M Qwen3-VL tower, frozen
config.json wires backbone, tower, projector
preprocessor_config.json Qwen preprocessing, patch 16
tokenizer.json, tokenizer_config.json, generation_config.json
The text shards are not included. They are byte-identical to
poolside/Laguna-S-2.1, so point --model-path at the upstream checkpoint and
load these files alongside it. Copying 219 GB would buy nothing.
Full inference code, including the projector module and the blind-control
evaluation harness, is in the linked GitHub repository.
License
MIT, following both parents. The projector weights are released under MIT.
Redistributed upstream weights keep their original terms.
Acknowledgements
Built on poolside's Laguna-S-2.1 and Alibaba's Qwen3-VL. Neither team was involved
in this work. Please do not send issues about this checkpoint to them.
