W4A16 (compressed-tensors) quantization of gnosis-lm/Gnosis-MedPolicy-12B at v0.4 (revision="v0.4"). This card mirrors the base model card for the full description, evaluation, lineage, intended use, and license; the base repo is canonical.

For inference stacks that consume compressed-tensors - vLLM and SGLang (~8GB VRAM). This is the quant served on our inference server as gnosis-medpolicy.

W4A16 here is group_size 32, symmetric int4 weights with Google-exact min/max scales. int4 is the same numeric scheme gemma-4's quantization-aware training (QAT) was trained against, so int4 inference stays close to the weights Google tuned for.

Lineage note: this quant is built from the SLERP'd deployment build (the fine-tune SLERP-merged with gemma-4-12B-it-qat-q4_0-unquantized to recover instruct/tool-use behavior) and is the exact model we serve. It is not a re-quantization of this repo's main weights, which are the pre-SLERP fine-tune base - the served quant and the main fine-tune base are intentionally distinct lineages.

gemma-4's heterogeneous head dims require the TRITON_ATTN backend; fp8 KV is recommended.

Gnosis is a family of custom models trained to operate in the RLM (Recursive Language Model) pattern inside the prehend harness: long context is offloaded into a variable in a REPL and the model writes programs that slice, search, and recursively query it instead of attending over it directly. Gnosis-MedPolicy is the medical-policy / billing decomposition specialty.

This repo's current release is v0.4 (tag). Address a specific version with revision="v0.4".

v0.4 targets confabulation on real-knowledge items specifically: it penalizes the confabulated document-id span (not the whole trajectory) via a span-masked DPO objective with a DPOP anchor, trained on counterfactual swap-id minimal pairs. Lineage:

google/gemma-4-12B-it -> (SFT) v13 -> (KTO, rendered-prompt fix) 0.2 -> (on-policy DPO + SLERP) 0.3
                                                                     \-> (span-masked confab DPO + SLERP) 0.4

v0.4 = 0.2 + a 156-pair span-masked DPO step. Each pair is a counterfactual: chosen = the grounded final assertion, rejected = the same assertion with the document-id swapped, with the grounding REPL output kept in the prompt so the objective rewards conditioning on visible evidence (genuine Context-DPO) rather than ungrounded id recall. A DPOP anchor prevents the likelihood displacement that vanilla DPO would cause on such high-overlap minimal pairs. The result is then SLERP-merged with google/gemma-4-12B-it-qat-q4_0-unquantized (variant-g) to recover instruct/tool-use behavior for the served quant (see lineage note below). This is the late candidate (final DPO adapter), chosen over an earlier-epoch candidate that over-abstained and degraded more.

Eval set: 200 fresh synthetic corpus-NIAH tasks (needle codes disjoint from training) + 40 real ICD-10 tasks. grounded_correct = grounded and exact-label-correct. Counts are correct/scored.

Honest read: the headline gain is on real ICD-10 grounded-correct (17.5% -> 35.9%), the item class where confabulation of plausible-but-wrong codes was the dominant failure - exactly what the span-masked confab objective was built to cut, and it did so without regressing synthetic grounding (+3.7pp), tool-misuse (down again), or output degradation (2.0%). ICD is small-n (n=39), so treat the magnitude as directional rather than individually significant; the synthetic gain is modest and confabulation remains the dominant failure mode in absolute terms. Eval used the baseline arm only.

This repo holds the full-precision (bf16) PRE-SLERP fine-tune base (Gemma4ForConditionalGeneration)

gemma-4's heterogeneous head dims require the TRITON_ATTN backend; fp8 KV is recommended.

Built to run as the inference/orchestrator inside the prehend RLM harness over offloaded long context; it is not a general chat model and is evaluated on retrieval-grounded decomposition. Absolute grounded-correct is still modest and confabulation remains the dominant failure mode. Eval used the baseline arm only (no retrieval-steering); arm ranking is known to invert between synthetic and real ICD tasks. Medical outputs are research artifacts, not clinical or billing advice.

apache-2.0, inherited from the google/gemma-4 lineage (the Gemma 4 family is released under Apache-2.0).

Trained with the rlm-trainer stack; evaluated with prehend. Family naming and packaging convention: org gnosis-lm, one repo per specialty+size, version as a git tag.