A LoRA fine-tune of cyber-analyst-4B that adds a strict finding-verifier behavior. Given the evidence for a suspected web vulnerability, it decides whether the evidence actually proves the bug, and rejects the ones that only look real.
[!WARNING]
This is a pilot. It was trained on a small (~100 example) balanced set to check the
recipe before scaling. It lives in its own repo and does not replace the production
cyber-analyst-4B.
What it does
You give it the evidence for a suspected vulnerability (tool, URL, payload, response
snippet, PoC). It returns a JSON verdict:
It judges the evidence, not the scary-looking payload. That is the difference between
a verifier and a scanner. It is trained to reject the look-alikes that fool most tools:
Looks like a finding
Why it is a false positive
SQLi, got a 500
a generic 500 with no DB error and no extracted data proves nothing
blind boolean SQLi
the true and false conditions returned identical responses
reflected XSS
the payload came back HTML-encoded or JS-escaped, so it cannot run
SSRF to a metadata IP
ECONNREFUSED. the request was blocked, no internal data returned
IDOR by enumerating IDs
every non-owned ID returned 403; authorization is enforced
path traversal with a null byte
404, no /etc/passwd contents; the bypass read no sensitive file
JWT with alg=none
the server rejected it with 401
When something is genuinely suspicious but not confirmed, it answers inconclusive and
says what evidence is still missing (a differential test, a decoded token, an extracted
row). It also knows who it is (Cyber-Analyst 4B, by BrainboxAI), in Hebrew and English.
Lineage
Google Gemma-4 E4B
-> cyber-analyst-4B (1.27M security examples, BrainboxAI)
-> this pilot (verifier + identity LoRA delta)
Training
Base
BrainboxAI/cyber-analyst-4B (weights kept; this only adds a delta)
~100 balanced verifier examples (true / false / inconclusive, with adversarial hard negatives across 19 vuln classes) plus a small share of bilingual identity
Eval
held-out split, eval_loss around 1.13
Hardware
one RTX 4090, ~16GB VRAM
Benchmark: does the delta actually reject false positives?
The whole point of the verifier delta is to stop the model from confirming
vulnerabilities that only look real. So the benchmark measures exactly that.
17 held-out examples (never seen in training): 8 true positives, 6 false
positives, 3 inconclusive. Each is a suspected finding with its evidence. The
model returns a verdict. Thinking mode is disabled with an assistant prefill so
the model answers directly. Same test, same prompt, run against the base and the
pilot.
Base vs pilot on 17 held-out examples
The base model confirms everything. It says true_positive to all 8 real bugs,
but also to all 6 look-alikes: a 500 with no DB error, an SSRF that got
ECONNREFUSED, an IDOR where every non-owned ID returned 403. It cannot tell a
finding from a decoy. That is the scanner bias a verifier is supposed to remove.
After 100 verifier examples the pilot rejects 4 of those 6 false positives, and
overall accuracy goes from 47% to 65%. The cost was one over-correction: it
flagged a genuine Twig SSTI as a false positive. A fair trade for a delta this
small, and the direction is the one we wanted.
What it still gets wrong: neither model abstains. Both answer 0 of 3 inconclusive
cases, because the pilot inherited the base model's habit of always committing to
a verdict, and 100 examples was not enough to install "say inconclusive when the
confirming evidence is missing." That is the first thing the next round fixes,
with more inconclusive training data.
Read this as what it is: a pilot proving the recipe works. A 100-example delta
moved false-positive rejection from 0/6 to 4/6 without breaking true-positive
detection. Scale the data and the abstention gap should close too.
Note: Gemma-4 is a new architecture. Loading the GGUF needs a recent llama.cpp, so
update LM Studio or Ollama to the latest. In LM Studio, make sure the format selector
is set to GGUF.
Quick start (LM Studio)
Download gguf/cyber-analyst-4B-safetensors.Q4_K_M.gguf and load it. For the verifier
task, use a system prompt like:
You are a strict security-finding verifier. Given the evidence for a suspected
vulnerability, decide true_positive (the evidence proves exploitation),
false_positive (disproven or not exploitable), or inconclusive (suspicious but
the confirming evidence is missing; say what is missing). Judge from the evidence,
not the title. Respond with only JSON: {"verdict":"…","reason":"…"}.
Continue training from here
Load the base cyber-analyst-4B and attach the adapter from
cyber-analyst-4B-verifier-pilot-lora
to keep training. The cleaner option is to train from the original cyber-analyst-4B
with the full accumulated dataset, which avoids stacking one delta on top of another.
A merged 16-bit model can be regenerated from the base plus the adapter in one step
when you need it.
Limitations
Pilot scale, about 100 examples. It checks the format and basic discrimination. It is
not a production verifier yet. Scale the dataset (600+) before relying on it.
It inherits the base model's limits (public-data bias, temporal cutoff).
Not for autonomous security decisions without human review.
-orange){kind=icon}
