A LoRA adapter for Qwen/Qwen2-VL-2B-Instruct that is part of Datasent's polygen visual-token compression recipe. It teaches the LLM to interpret a compressed visual-token sequence produced by the polygen splice between Qwen2-VL's vision encoder and its language model. Paired with the splice, it recovers baseline accuracy under aggressive compression; on a 7B backbone the same recipe exceeds the un-spliced baseline.

Interactive side-by-side demo: datas3nt/qwen2vl-polygen

What this adapter is (and isn't)

This is not a general-purpose Qwen2-VL fine-tune. It is one half of a two-part configuration:

The polygen splice sits between Qwen2-VL's merger output and its LLM input and replaces the per-image visual-token sequence with a much shorter structured representation (or, in the hook splice path, an in-place reconstruction that preserves the original token count while compressing the information).
This LoRA adapter teaches the LLM to interpret that compressed representation. Without the adapter, the LLM treats the compressed sequence as out-of-distribution noise; with the adapter, accuracy returns to baseline (or above it on larger backbones).

The adapter alone (no polygen splice) gives a small accuracy lift on ScienceQA-distribution-shifted tasks via cross-benchmark transfer, but its purpose is the joint configuration with the splice.

Recipe

Parameter	Value
Base model	`Qwen/Qwen2-VL-2B-Instruct`
Adapter type	LoRA (PEFT)
Rank `r`	16
Alpha	32
Dropout	0.0
Bias	none
Target modules	`q_proj`, `v_proj`
Task type	`CAUSAL_LM`
Trainable parameters	2,179,072 (~0.099% of base)
Training data	ScienceQA train split, n=300 examples
Training steps	1000
Optimizer	AdamW
Visual encoder	Frozen during training (`requires_grad=False` on `visual.*`)

Results

The adapter is evaluated as the LoRA half of Datasent's hybrid LOSSLESS / COARSE polygen splice, which exposes five operating tiers (T=0 most aggressive compression through T=4 lossless storage). Below are cross-benchmark numbers at T=4 + LoRA (the conservative, lossless-storage tier), evaluated against an un-spliced un-LoRA baseline.

Benchmark	Backbone	Hardware	Splice	Accuracy delta
ScienceQA	2B	Apple MPS	bypass	+3.08pp
ScienceQA	7B	NVIDIA H200	bypass	+6.9pp at T=4, +8.4pp at T=3
MMMU	2B	Apple MPS	bypass	+1.57pp
VQAv2 (n=2000)	2B	NVIDIA H200	bypass	+0.15pp
VQAv2 (n=2000)	7B	NVIDIA H200	bypass	+2.50pp
DocVQA (n=500)	2B	NVIDIA H200	hook	+0.21pp (within Wilson 95% CI)

Compute and storage are reported as attention-FLOP reduction (a FLOP count, not wall-clock) and storage compression, both a function of the operating tier. On the 7B backbone: ~71% fewer attention FLOPs at T=2 (accuracy +7.4pp) and 5.1x storage at T=3 (accuracy +8.4pp). The 7B card and the live demo carry the full per-tier curve.

The LoRA was trained only on ScienceQA. The lifts on MMMU, VQAv2, and DocVQA are pure cross-benchmark transfer; the adapter never saw any of those distributions during training. The DocVQA hook-path result uses a length-preserving reconstruction that preserves Qwen2-VL's 2-D positional encoding for high-token-count document images; the GPU-resident reconstruction in the SDK removes the hook path's earlier per-forward compute overhead.

How to use

This adapter is intended to be loaded by Datasent's polygen SDK as part of a two-step setup:

python

from polygen.integrations.qwen2vl import PolygenQwen2VL

model = PolygenQwen2VL.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    lora_adapter="datas3nt/qwen2vl-polygen-lora-r16-1000",
    operating_tier="T=4",
    splice_mode="auto",          # bypass for moderate L_v, hook for high L_v
    torch_dtype="float16",
    device_map="cuda",
)

For evaluation without the polygen splice (rare; the adapter's value is in the joint configuration), it can be loaded as a standard PEFT adapter:

python

from peft import PeftModel
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor

base = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    torch_dtype="float16",
    device_map="cuda",
)
model = PeftModel.from_pretrained(base, "datas3nt/qwen2vl-polygen-lora-r16-1000")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")

Intended use

Primary: evaluation and demonstration of Datasent's polygen visual-token compression on Qwen2-VL-2B, paired with the splice.
Secondary: cross-benchmark transfer studies on ScienceQA-adjacent vision-language tasks (the +1.90pp transfer effect on VQAv2 is the cleanest case).
Not intended for: general-purpose Qwen2-VL fine-tuning, production deployment without the polygen SDK, or stand-alone use on tasks that diverge significantly from ScienceQA's distribution.

Limitations

2B-only. This adapter targets Qwen2-VL-2B-Instruct. The same recipe on a 7B backbone (with separately-trained 7B LoRA) is what produces the +8.4pp accuracy / ~71% fewer-attention-FLOPs headline; that artifact lives under a separate repository.
Recipe assumes the polygen splice. Without the visual-token compression the adapter underperforms a full fine-tune; its value is in the joint configuration.
English training data. ScienceQA is an English-language benchmark; behaviour on non-English captions or non-Latin scripts is undocumented.
n=300 train sample. The adapter is intentionally small and cheap to train (~30 minutes on a single H200). Performance on out-of-distribution tasks beyond the four benchmarks above has not been characterized.

Citation

bibtex

@misc{datasent-polygen-vlm-2026,
  title  = {Polygen: visual-token compression for vision-language models},
  author = {Datasent, Inc.},
  year   = {2026},
  url    = {https://huggingface.co/spaces/datas3nt/qwen2vl-polygen},
}

License

Distributed under a Datasent evaluation license. Contact greg@datasent.com for production licensing.