You can deploy this model and start earning money today!

mglynnh/
financial-advice-probe-gemma2-2b-l14

Name: financial-advice-probe-gemma2-2b-l14
Availability: InStock

Views

No views yet

Financial-advice probe (Gemma-2-2B, layer 14)

A small CovSeq classifier head trained on layer-14 hidden states of google/gemma-2-2b-it. Reads activations from a (prompt, completion) pair and decides whether the completion contains personalised financial recommendations (a policy violation), as opposed to general financial education (allowed).

This is the Phase A probe — laptop-scale shakedown trained on ~250 GPT-4.1 completions covering hand-written clear-cut cases, persona-conditioned generations, and a synthetic categorise-then-generate stream. Annotated by Claude Sonnet 4.5 via batch API.

Eval summary (token-level, full Phase A split)

metric	value
token-level ROC-AUC	0.904
n scored tokens	138,685
positive tokens	10,998
negative tokens	127,687

Phase A gate was token-level ROC-AUC ≥ 0.9 — cleared.

Architecture

Proxy encoder: google/gemma-2-2b-it, bf16. Forward pass only — does not generate. The probe reads layer-14 hidden states for completion tokens.
Probe head: CovSeq, window size 8, compressed size 64, output size 1. Trained with BCEWithLogitsLoss and per-sample sampler weights for class balance.
Training data: (prompt, GPT-4.1 completion) pairs, span-annotated. Train-time and serve-time activation distributions match because the completions in both come from GPT-4.1.

Files

probe_head.bin — trained CovSeq head weights (~1.2 MB).
config.json — full training config (architecture, optim, layer index).
probe_scores_all.json — eval summary (token-level AUC + counts).
training_metrics.json — per-step training metrics.

Use

This probe is a post-hoc filter: GPT-4.1 generates a completion, the probe scores it, the gateway decides what to do. A reference deployment lives in hartford/modal_app.py — a Modal app exposing /v1/chat/completions (OpenAI-compatible) that wraps the probe in front of GPT-4.1 via OpenRouter.

python

import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
gemma = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-2b-it",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    output_hidden_states=True,
)

probe_path = hf_hub_download(
    "mglynnh/financial-advice-probe-gemma2-2b-l14", "probe_head.bin"
)
probe_state = torch.load(probe_path, map_location="cuda")
# Wire into the CovSeq probe class — see hartford/scripts/probe_demo.py for
# the reference loader (`load_bundle`) and end-to-end scoring (`score_pair`).

Limitations

Phase A only. Tiny training set (~250 rows). Not ship-ready. Phase B scales to ~3.5k rows + OOD eval against claude-sonnet-4.5 completions.
One policy. Financial advice only.
English only.
Encoder-locked. Swapping the proxy encoder or upstream generator requires retraining — the activation distribution shifts.