WardenSight-1B is a 1-billion parameter terminal agent specialized for vulnerability
file localization in WordPress plugins. It is built on
Antares-1B (Cisco Foundation AI), itself
built on IBM Granite 4.0 1B, and adapts that general-purpose localizer to a single
ecosystem through supervised fine-tuning on vulnerability-localization trajectories
from security fixes in WordPress plugins.
Given a plugin repository and a CWE class, WardenSight explores the code through a
read-only terminal interface — issuing grep, find, cat and similar commands — and
submits a ranked list of files it believes contain that weakness. It runs entirely
on-premises at 1B parameters, so client code never leaves the machine.
On 1,646 held-out WordPress plugin tasks, WardenSight-1B achieves a File-F1 of 0.3263
against the base model's 0.2671 — a +22.2% relative improvement, paired
bootstrap 95% CI [+0.0378, +0.0797], P(better) = 1.00. It reaches that by being
more selective: it names 1.31 files per task where the base model names 2.55.
WardenSight is a triage instrument. It tells an analyst which files to read first.
It does not produce line numbers, exploits, patches, or proof that code is vulnerable.
WP-VLoc File-F1 comparison
To our knowledge, WardenSight-1B is the first publicly released open-weight model
specialized and evaluated for vulnerability file localization in WordPress plugins.
WordPress plugin security review by authorized personnel, where a
vulnerability class is already known or suspected and the task is locating it. Typical
uses are triaging a disclosure, narrowing a code review, or prioritizing files for
manual audit.
Downstream Use
The model emits standard tool calls and integrates into any agentic loop providing a
sandboxed read-only terminal. Reports are available in human-readable, JSON and SARIF
formats for ingestion by code-scanning platforms.
Out-of-Scope Use
Determining that code is safe. The model has no reliable ability to report the
absence of a vulnerability — see Limitations. Point it at clean code and it will
name a file anyway.
Autonomous remediation or any workflow acting on output without human review.
Compliance evidence or attestation of code security.
WordPress themes. Every training and evaluation task is a plugin — no theme
appears anywhere in the 1,807 tasks used for this work. Themes are structurally
similar PHP and the model will run against one, but its accuracy there is entirely
unmeasured. Treat theme results as unvalidated.
Non-WordPress codebases. Specialization was measured on WordPress plugin PHP
only; on other ecosystems use the general-purpose base model instead.
Offensive use. The model localizes weaknesses for defensive review; it does not
generate exploits and must not be used to attack systems without authorization.
How to Get Started with the Model
WardenSight is an agent model, not a one-shot text classifier. It needs three pieces:
the weights, a raw-completions model server, and the Antares CLI agent loop with the
WardenSight prompt adapter.
For a version-pinned WordPress.org plugin, install tooling/ws-scan on your PATH
and run wardensight plugin-slug@1.2.3 CWE-79. It downloads and extracts the requested
plugin release before invoking the same agent loop. See
tooling/README.md
for installation details and the exact division between Cisco's CLI and the
WardenSight wrapper.
The adapter matters. WardenSight is fine-tuned on a three-tool contract
(terminal, submit_vulnerable_files, submit_no_vulnerability_found) with a
specific system message; the CLI's default adapter presents a fourth tool the model
has never seen, and under it the model degrades into repeated commands. The adapter,
profile and a convenience wrapper are in
wardensight-1b/tooling.
Sampling settings used for every published number: temperature 0.3, frequency
penalty 0.3, 15 terminal calls, 2,048 max tokens, stop tokens
["<|end_of_text|>", "<|start_of_role|>"].
Training and Evaluation
Training Data
584 agent trajectories over WordPress plugin repositories with known, fixed
vulnerabilities, drawn from Wordfence Intelligence
advisories (CC BY 4.0). Each trajectory explores a plugin snapshot taken at the last
vulnerable release and terminates in a file submission.
Ground truth is derived from security-relevant diff hunks, not whole release
diffs. This distinction is load-bearing: 58% of files in a typical WordPress
vulnerable→fix release diff contain no security change, and under whole-diff labels a
constant guess of <slug>.php scores 0.3428 and outranks every trained model. Labels
were rebuilt around CWE-aware hunks, which drops that control to 0.1041 and makes the
benchmark able to rank localizers at all.
Training plugins are disjoint at the plugin level from every evaluation set.
Training Setup
Full fine-tune of fdtn-ai/antares-1b, no adapters or quantization.
Training setting
Value
Method
Supervised fine-tuning on 584 localization trajectories
Training length
2 epochs
Learning rate
8e-6
Maximum sequence length
5,120 tokens
Numeric precision
bfloat16
Training hardware
1× ASUS Ascent GX10 (NVIDIA GB10, 128 GB unified memory)
Evaluation
Evaluated on WP-VLoc, a WordPress-specific benchmark built for this work: plugin
snapshots reconstructed at the pre-fix release, paired with security-hunk ground truth
files. Held-out plugins never appear in training. Scored by exact path-set File-F1
with a paired bootstrap (20,000 resamples) against the base model on identical tasks.
WardenSight-1B has not been evaluated on Cisco's separate 500-task, six-ecosystem
VLoc Bench, so its WP-VLoc score should not be inserted into that benchmark's
cross-model table.
model
File-F1
precision
recall
files named
n
WardenSight-1B
0.3263
0.397
0.315
1.31
1,646
Antares-1B (base)
0.2671
0.289
0.330
2.55
1,646
constant guess <slug>.php
0.1041
0.132
0.094
1.00
1,646
constant guess slug + index.php
0.0816
0.074
0.104
2.00
1,646
Paired improvement +0.0592 File-F1 (+22.2%), 95% CI [+0.0378, +0.0797],
P(better) = 1.00. Task level: 453 improved, 340 regressed, 853 unchanged; 5% trimmed
mean +0.0621, so the gain is not driven by outliers.
Safety Alignment
This model operates as a component in a sandboxed agentic loop, not as a
conversational assistant, and no standalone safety alignment was performed on it.
Safety should be addressed at the system level:
Run the agent inside an isolated sandbox (Docker with network=none)
Restrict deployment to authorized security personnel
Require human review of output before any remediation action
Audit agent trajectories in high-risk workflows
Do not expose this model as a general-purpose assistant endpoint.
Limitations
It cannot report a clean repository. On 500 already-patched snapshots it
declared a vulnerability present in every one (TNR 0.000), and its abstention rate
across all held-out evaluation is 0.000 — it has never declined to name a file.
This is a property of the model family rather than this checkpoint alone; the base
model scored TNR 0.006 on the same test. Use it only when a vulnerability is known
or strongly suspected to exist.
Coverage is partial. It places a correct file in its ranked output on ~41% of
held-out tasks. It is a filter that reorders analyst attention, not a detector that
finds everything — and it solves 76 tasks the base model misses while missing 90 the
base model solves.
Terminal budget. Performance degrades on large plugins where 15 commands cannot
cover the codebase, and on vulnerabilities spanning many files.
Pattern dependence. Strongest on weaknesses with grep-able signatures (XSS,
missing authorization); weaker where the flaw requires semantic reasoning about
behavior (unsafe deserialization, injection through indirect data flow).
Ecosystem scope. WordPress plugins only — 1,807 plugin tasks, no themes and
no non-WordPress code. Specialization was not measured beyond that and should not be
assumed to transfer.
Knowledge cutoff. Inherits the base model's April 2025 cutoff; training data
covers WordPress advisories through July 2026.
Recommendations
Give it a plugin you already suspect and the correct CWE class — supplying the wrong
class produces unhelpful output rather than an admission that nothing was found. Pin
the plugin version when investigating a known advisory; scanning the current release
often re-finds files whose vulnerability shipped a fix. Read the ranked output as a
reading order, and confirm every finding in the source before acting on it.
Provenance
The 584 training trajectories for this checkpoint were generated with Claude Code.
WardenSight-1B is a full fine-tune of fdtn-ai/antares-1b; Claude was the teacher
used to produce the WordPress localization demonstrations, not the student or base
model.
WardenSight-1B is not affiliated with or endorsed by Anthropic. The raw Claude
trajectories are not distributed with this release. Users are responsible for
evaluating any provider terms that may apply to downstream use of model-generated
training data.
Cisco vulnerability-localization-benchmark, Apache-2.0, with local corrections to the command validator, path normalization and the true-negative scorer
