A LoRA fine-tune of google/gemma-4-e4b-it trained with cross-entropy plus top-k logit distillation from the teacher. The teacher is the sparse mixture-of-experts google/gemma-4-26b-a4b-it.

The adapter has been folded into the base weights, so this is a standalone checkpoint: load it exactly like the base model, with no PEFT and no CEED code.

This is Group B2 of the CEED study (Causal Expert–Evidence Distillation), a research artifact published for reproducibility. It is not a product.

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("gnitoahc/ceed-b2", dtype="float16")
processor = AutoProcessor.from_pretrained("gnitoahc/ceed-b2")

The model was trained and scored with a short-answer instruction in the prompt. Without it an instruction-tuned model answers "The total written in the image is **28**." against gold "28" and scores zero on every metric here.

Training


Corpus	chartqa 2,500, docvqa 5,349, gqa 10,000 (17,849 examples, 80/10/10 split by example id)
Passes over the training split	2.69
Adapter	LoRA rank 4
Final cross-entropy	0.9816
Final KD term	2.3852
Seed	0
Run identity	`575ea8aeee645401277edca0c6555326879f72291a6ac8330f8170dd80b922ee`

Evaluation

Dataset	Metric	Score	n
docvqa	ANLS	0.8506	565
gqa	exact match	0.6191	1016
chartqa	relaxed accuracy	0.5783	249

Scored by CEED's own harness (harness_version: ceed-direct-1) with greedy decoding, on CEED's own 10% validation split.

These numbers are not comparable to published DocVQA / GQA / ChartQA leaderboard results. Different splits, different prompt, different decoding. They are meaningful only against the other CEED Groups, which were scored identically.

Limitations

This is a LoRA result. Merging folds the adapter into the weights; it does not turn a rank-4 adapter into a full fine-tune. CEED's own ADR-0005 bars LoRA numbers from the study's headline table, because a null result under a small adapter cannot be attributed between "the signal does not transfer" and "the adapter lacked the capacity to hold it". Read any comparison involving this checkpoint with that in mind.
The distillation gain is not established. The no-teacher control (CEED B1), trained identically but with kd_weight: 0, scored above this checkpoint on every dataset (docvqa 0.8506 vs 0.8798; gqa 0.6191 vs 0.6959; chartqa 0.5783 vs 0.7871). Whatever this checkpoint's objective contributes, it is not visible as an advantage over supervised fine-tuning here.
Trained on document, natural-image and chart VQA in English only. Behaviour outside that is untested.
Inherits the base model's limitations and the Gemma licence.

ceed_provenance.json beside the weights carries the source run's identity, parameter-efficiency mode, and metrics.