Every number is the same benchmark run twice: once against the frozen base model, once against this checkpoint, in the same container with the same prompts, seeds and sampled rows.

Public security benchmarks

Suite	Base	This model	Delta
`cti_mcq`	67.2%	66.1%	-1.2%
`cti_rcm`	61.5%	62.5%	+1.0%
`mmlu_computer_security`	82.0%	84.0%	+2.0%
`mmlu_security_studies`	77.1%	75.5%	-1.6%
`secqa_v1`	98.2%	99.1%	+0.9%
`secqa_v2`	97.0%	97.0%	+0.0%

MCQ suites are scored by the log-probability of the option letter, not by parsing generated text. That isolates knowledge from answer formatting -- the fine-tune was trained on formatting, so parsing would credit it twice -- but it means these numbers are not comparable to published leaderboard scores, which parse decoded answers.

mmlu_* rows are forgetting probes, not targets. A flat result there is the intended outcome.

Held-out split

Metric	Base	This model	Delta
`exact_match`	0.0033	0.2433	+0.2400
`median_token_f1`	0.2203	0.5049	+0.2846
`refusal_rate`	0.0233	0.0000	-0.0233
`token_f1`	0.2798	0.5796	+0.2998

Deltas are raw, not direction-normalized: refusal_rate and any *_failure_rate improve by going down, so a negative delta on those rows is the good direction.

Per-example: 261 of 300 holdout examples improved, 14 unchanged, 25 regressed. A mean delta can hide a few large wins masking many small regressions, which is what this split is for.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "HRB25/qwen35-9b-pentest-v5"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Explain what `nmap -sS` does."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    # Training rendered a closed, empty <think> block. Leaving this at its
    # default opens a thinking block the fine-tune never saw after its own
    # examples, and the model will answer inside reasoning it was not trained
    # to produce. Match training unless you are deliberately experimenting.
    enable_thinking=False,
)

Merged BF16 weights are in merged/; the PEFT adapter alone is in adapter/; GGUF builds for llama.cpp and LM Studio are in gguf/. The complete metric table, including diagnostics omitted above, is in TECHNICAL_REPORT.md and TECHNICAL_REPORT.json.

Intended use

Authorized penetration testing, security education, and defensive research: explaining tooling and techniques, reading vulnerability descriptions, and supporting engagements you have written permission to perform.

Out of scope: unauthorized access, targeting systems you do not own or have permission to test, and any use where a wrong answer causes harm without a human verifying it first. The model was not evaluated for refusal behaviour on abuse requests beyond the refusal rate reported above.

Limitations

The held-out split is drawn from the training distribution, so on its own it cannot separate learned security knowledge from learned phrasing. The public suites are what carry that part of the claim.
Token-F1 is lexical. It does not establish that an exploit works or a patch is correct.
Benchmark rows were not checked for overlap with the training corpus.
GGUF quantization can shift results relative to the BF16 weights.

Licensing

These weights derive from Qwen/Qwen3.5-9B and inherit its licence terms -- check the base model card before redistributing or using commercially. Some evaluation datasets used to produce the numbers above are CC-BY-NC-SA; that constrains redistribution of those datasets, not of this model.