You can deploy this model and start earning money today!

Sandy-sys/
osint-argus-gemma4-qat-w4a16

Name: osint-argus-gemma4-qat-w4a16
Rating: 2.5 (7 reviews)

Views

No views yet

OSINT ARGUS — Gemma 4 12B QAT · Indian-language OSINT · W4A16 tri-modal

Google's QAT (quantisation-aware-trained) Gemma 4 12B, full-parameter fine-tuned for Indian-context open-source-intelligence analysis over text, images and speech, and packed to int4 / group-32 / symmetric — the exact numerical grid it was trained on.

One self-contained repository. No adapter, no LoRA, no merge step, no second model to download. Pull this repo, point a server at it, and it runs.


Base	`google/gemma-4-12B-it-qat-q4_0-unquantized`
Architecture	`Gemma4UnifiedForConditionalGeneration`, 48-layer early-fusion trunk (10.90 B params)
Modalities	text · image (280 soft tokens/image) · audio
Quantisation	int4, group size 32, symmetric, `pack-quantized` (compressed-tensors)
Weights	8.25 GB, single `model.safetensors`
Trained parameters	all 10.90 B trunk weights + 52.4 M modality embedders
Output contract	strict 11-key OSINT JSON record

1 · Running it

bash

huggingface-cli download Sandy-sys/osint-argus-gemma4-qat-w4a16 --local-dir argus

python -m sglang.launch_server --model-path ./argus \
  --quantization compressed-tensors \
  --mem-fraction-static 0.45 \
  --context-length 8192

That is the entire procedure. The repository already contains the packed weights, tokenizer, multimodal processor, chat template, and the config keys the server needs. Nothing to patch, merge, or fetch separately.

Verified end-to-end: pulled fresh from this repo into an empty directory, loaded with no local files, and generated correctly at 8.25 GB resident.

VRAM — read this before deploying

runtime	behaviour	VRAM
SGLang / vLLM	keeps weights packed at int4	8.25 GB ✅
Transformers	decompresses to bf16 at first inference	~25.1 GB ⚠️

The 10 GB budget holds only under a runtime with native compressed-tensors int4 kernels. Transformers is fine for a desktop sanity check and wrong for an edge box.

Under SGLang this leaves ~1.75 GB of a 10 GB allocation free, so the model co-resides with a second model on a 30 GB card (validated target: MSI Edge Expert alongside Nemotron 3 Super NVFP4).

Transformers, for a quick check on a large card

python

from transformers import AutoProcessor, AutoModelForCausalLM
import torch

repo = "Sandy-sys/osint-argus-gemma4-qat-w4a16"
proc = AutoProcessor.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, device_map="cuda:0")

msgs = [{"role": "user", "content": [
    {"type": "image", "url": "poster.jpg"},
    {"type": "text", "text": "Analyse this image as an OSINT record."}]}]
inp = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                               return_dict=True, return_tensors="pt").to(0)
print(proc.decode(model.generate(**inp, max_new_tokens=512)[0][inp["input_ids"].shape[1]:],
                  skip_special_tokens=True))

Config note. vision_config.model_patch_size=48, vision_config.num_soft_tokens=280 and audio_config.output_proj_dims=640 are already backfilled in config.json. They are absent from Google's upstream QAT config and SGLang exits without them. Listed so the difference from upstream is documented — no action needed.

2 · The output contract

Every response is a single JSON object with exactly these 11 keys:

json

{
  "language": "hi", "script": "Deva", "code_mixed": false,
  "entities": [{"text": "...", "type": "PER|ORG|LOC|MISC"}],
  "claims": [{"text": "...", "verdict": "supported|refuted|unverified"}],
  "stance": "favor|against|neutral",
  "sentiment": "positive|negative|neutral",
  "misinformation": "verified|false|unverified",
  "hostility": {"label": "none|offensive|hateful"},
  "events": [{"type": "...", "text": "..."}],
  "summary_en": "English summary, <= 280 chars"
}

The schema_lock gate below measures how reliably the model emits a valid record under bare decoding — no constrained grammar, no JSON mode, no retries.

3 · Evaluation

All figures measured on the exact weights in this repository. Evaluation items are held out by construction: every media path used in training is excluded, article texts are matched by normalised content hash so the same story behind a different image is still excluded, and audio clips are excluded by raw-byte SHA-256.

3.1 Text — 200 items per task, bare decoding

gate	stock Gemma-4 QAT	this model
`schema_lock` — valid 11-key OSINT record	0.0	0.985
`ner` — micro-F1, 11 Indian languages	0.6163	0.7046
`sentiment` — accuracy, 13 languages	0.76	0.785
`claim` — refuted-class F1	0.8125	0.8685
`halluc_detect` — hallucination-detection F1	0.3981	0.4783
`faithfulness` — grounded-answer schema rate	0.9536	0.9669
`general` — MCQ accuracy %	59.0	53.0

3.2 Multimodal — 106–120 items per task

gate	this model	majority baseline	n
`vision_ocr` — CER ↓	0.5482	—	120	—
`vision_misinfo` — accuracy	0.5083	0.5083	120	at baseline
`vision_hostility` — accuracy	0.434	0.6321	106	below baseline
`audio_asr` — CER ↓	0.6889	—	120	—
`audio_langid` — accuracy	0.1833	0.175	120	above baseline
`audio_abuse` — accuracy	0.425	0.5417	120	below baseline

3.3 Image text by script (CER, lower is better)

script	code	CER	n
Arabic (Urdu)	`Arab`	0.2271	38
Devanagari (hi, mr, sa, ne)	`Deva`	0.6059	31
Bengali (bn, as)	`Beng`	0.6831	38
Odia	`Orya`	0.9546	13

Trained on all 12 INDICSTR12 scripts; the four above are the ones with held-out measurement.

3.4 Speech transcription by language (CER, lower is better)

language	code	CER	n
Bhojpuri	`bho`	0.069	1
Hindi	`hi`	0.2832	7
Maithili	`mai`	0.3016	2
English	`en`	0.4189	2
Marathi	`mr`	0.4717	3
Sindhi	`sd`	0.5502	1
Dogri	`doi`	0.5673	2
Bengali	`bn`	0.8173	6
Kashmiri	`ks`	0.8588	3
Punjabi	`pa`	0.8615	1
Assamese	`as`	0.8778	2
Tamil	`ta`	0.9083	2
Urdu	`ur`	0.9211	1
Telugu	`te`	0.9278	1
Malayalam	`ml`	0.9289	2
Gujarati	`gu`	0.9335	2
Odia	`or`	0.9346	2
Kannada	`kn`	0.939	3
Bodo	`brx`	0.9545	2
Konkani	`kok`	0.9583	2
Manipuri	`mni`	1.0	2

Per-language sample counts are small (n = 1–7); treat these as indicative ranking, not precise measurement.

3.5 Spoken language identification (recall)

language	code	recall
English	`en`	1.0
Hindi	`hi`	0.9231
Sanskrit	`sa`	0.5
Bengali	`bn`	0.2381

Zero recall: Assamese, Bhojpuri, Gujarati, Kannada, Konkani, Maithili, Malayalam, Manipuri, Marathi, Odia, Punjabi, Sindhi, Tamil, Telugu, Urdu. Spoken language-ID is effectively usable for Hindi and English only.

4 · Language coverage

Text OSINT — trained across Assamese, Bengali, Bodo, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu and Urdu, plus Maithili, Sanskrit, Dravidian code-mixed content and English. The text gates in §3.1 are aggregates over this mix.

Practical summary. The Hindi belt is the strongest region across all three modalities (Hindi/Bhojpuri/Maithili/Marathi speech, Devanagari reading, full text analysis). Urdu is the best-read script from images. The Dravidian languages and the north-east are well served in text but weak in speech; Odia is the weakest script for image reading.

5 · Known limitations

Stated plainly, because they matter operationally:

Media classifiers are weak. vision_hostility and audio_abuse score below their majority baselines; vision_misinfo is only marginally above baseline with poor fake-class recall. Treat the misinformation and hostility fields as weak signals on media inputs, never as decisions. They are considerably more reliable on text.
Spoken language-ID is near chance outside Hindi and English (§3.5).
Odia image text is not usable (CER 0.95).
General MCQ knowledge dropped from the stock model's 59.0 to 53.0 — the cost of heavy specialisation toward the OSINT contract.
Small evaluation samples per language in §3.4/§3.5.

These axes received five dedicated training rounds. The finding: full-parameter QAT training at learning rates that preserve text quality shifts classifier priors without creating discrimination, and rates that create discrimination damage everything else. This is a genuine frontier of the method, documented rather than papered over.

6 · Method

Quantisation-aware fine-tuning with a straight-through estimator. The forward pass computes fake_quant(W) on the deployment grid (int4, group 32, scale = amax / 7.5); the backward pass updates the bf16 master weight unchanged. The weights are therefore optimised for the exact numerical grid they are served on, rather than trained in bf16 and quantised afterwards.

Full-parameter, single artifact. All 10.90 B trunk parameters plus the 52.4 M modality embedders were trained directly. Because Gemma 4 Unified is early-fusion — 664 of its 677 tensors are a shared trunk processing text, vision and audio alike — training the trunk improves all three modalities at once. There is no adapter to load and no merge step, which also eliminates a whole class of deployment defect (a merge that shifts every quantisation scale).

Other techniques used: stratified round-robin corpus interleaving so every training window sees the full capability mix; selective layer unfreezing; classifier-token loss weighting; source-conditional field masking; and a final weight average of two sibling checkpoints of this same model (36 of 48 trunk layers were bit-identical by construction).

Selection. Checkpoints were chosen by capability gates, never by language-model loss — loss and capability moved in opposite directions three separate times during this programme.

7 · Datasets

Referenced under their respective licences; none are redistributed here. Evaluation slices were reserved before the corresponding training data was built, in every case.

Factuality, claims and hallucination RAGTruth · LLM-AggreFact · HaluEval · ClaimBuster · AVeriTeC · CLEF CheckThat! · X-Fact · Climate-FEVER · ASQA · Indic-Hallucination (as, bn, gu, hi, kn, ml, ta, te, ur)

Indian-language understanding Naamapadam NER (11 languages) · IndicSentiment (13 languages) · Dravidian CodeMix · Indic-RAG (bn, gu, hi, ta) · MMLU · Indic-MMLU · AI2-ARC

Misinformation (text) IFND · BanFakeNews-2.0 · HinFakeNews · Urdu Fake News

Vision IFND-multimodal (fake-news imagery) · INDICSTR12 full + crops (12-script Indic scene text) · Memotion / Memotion 2 · MMSoc Hateful Memes · BHM (Bengali hateful memes) · MET-Meme · Bangla document OCR · Sanskrit/Devanagari OCR · Urdu (Arabic-script) OCR

Audio ADIMA (abusive speech, 8 Indic languages) · IndicSUPERB · IndicVoices · IndicTTS · FLEURS · Assamese / Bengali / Gujarati / Hindi / Nepali speech corpora · per-language language-ID pools (16 languages)

A machine-readable manifest of every source, pinned by commit SHA, is in the companion state repository as sources.json.

8 · Provenance and reproduction

Training code, all corpora builders, the complete set of 30+ gate reports, the per-round campaign history and the full session log live in the companion repository Sandy-sys/aditi-ps18-state.

Programme: OSINT ARGUS — AI-based OSINT analysis and monitoring, Indian context.

This card was generated on 2026-08-25 directly from this artifact's own gate reports. Every number above is measured, not estimated.