You can deploy this model and start earning money today!

Siluni/
gemma3-4b-cpt

Name: gemma3-4b-cpt
Availability: InStock

peft

safetensors

sinhala

continued-pretraining

gemma

qlora

low-resource

Siluni/sinhala-vqa-dataset

google/gemma-3-4b-it

adapter

Views

No views yet

gemma3-4b-cpt

CPT-only (Continued Pre-Training) adapter for Gemma-3-4B-IT on the MADLAD-400 Sinhala corpus. This adapter does not perform VQA on its own. It is intended to be used as the first stage of the sequential CPT → VQA pipeline together with Siluni/gemma3-4b-cpt-vqa-33k.

Base model: google/gemma-3-4b-it
Experiment: Group 3 — Sequential CPT stage
CPT corpus: MADLAD-400 Sinhala (~293M words)
Method: QLoRA (4-bit NF4, LoRA rank 16, alpha 32)

⚠️ Sequential Loading Required

This adapter must be loaded together with the VQA adapter and combined before inference. See Siluni/gemma3-4b-cpt-vqa-33k for the full loading instructions.

Citation

bibtex

@misc{keerthiratne2025sinhalavqa,
  title        = {Benchmarking and Adapting Compact Multimodal Models for Sinhala Visual Question Answering},
  author       = {Keerthiratne, Siluni and Weerasinghe, Ruvan and Sumanathilaka, Deshan},
  year         = {2025},
  institution  = {Informatics Institute of Technology / Robert Gordon University},
}