You can deploy this model and start earning money today!

xlr8harder/
synthid-qwen3-4b-instruct-2507-detectors

Name: synthid-qwen3-4b-instruct-2507-detectors
Availability: InStock

Views

No views yet

Qwen3-4B SynthID detector family

This repository contains independently loadable Bayesian SynthID detector bundles for Qwen/Qwen3-4B-Instruct-2507. The generated response corpus is xlr8harder/synthid-qwen3-4b-instruct-2507-wildchat; training, generation, vLLM integration, and evaluation code is in xlr8harder/synthid.

The Nature paper proposes several g-value scoring functions and uses the Bayesian scorer for its default SynthID experiments. A post-release amendment now compares these learned models with Google's official training-free weighted mean and the simple mean on the identical 200-token transfer grid. The reports and per-sample raw scores are under analysis/; methodology and interpretation are in DETECTOR_SCORE_FAMILY_AMENDMENT.md.

Generation profile: `synthid-paper-t0.7-k100-p1.0`

Model revision: cdbee75f17c01a7cc42f958dc650907174af0554
Native dtype: bfloat16
Maximum generated tokens: 4096
Temperature: 0.7
Top-k: 100
Top-p: 1.0
Min-p: 0.0
Sampling-table SHA-256: cb6af87dfb8414177e71cbc9c2790a771a8018ce1925916b160ab1b8f327dbad

Sampler settings are part of the detector provenance: these classifiers were fit on this profile and should not be described as sampler-independent merely because the watermark key and sampling table are unchanged.

Fixed operating-point transfer grid

Every detector selected one threshold on its own training-negative validation split at a target false-positive rate of 1%. The same threshold was then applied unchanged to all held-out test cohorts at 200 visible tokens. Matching-key Qwen rows report true detection rate and a dash for FPR. All other rows—including unwatermarked Qwen, the other watermark key, human ELI5, and original WildChat responses—report FPR and a dash for true detection rate.

Detector bundle	Trained negative	Trained positive	Evaluation dataset	N	True detection rate	False positive rate
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`eli5-human-lfqa`	2103	-	1.379%
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-key-a-matched`	8171	74.691%	-
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-key-a-unmatched`	8267	73.582%	-
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-key-b-matched`	8205	-	2.438%
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-key-b-unmatched`	8247	-	2.340%
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-unwatermarked-matched`	8173	-	2.386%
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`qwen-unwatermarked-unmatched`	8205	-	2.779%
`human-eli5-lfqa-test/key-a`	`eli5-human-lfqa`	`qwen-key-a-matched`	`wildchat-original`	6415	-	1.574%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`eli5-human-lfqa`	2103	-	0.951%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-key-a-matched`	8171	-	1.811%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-key-a-unmatched`	8267	-	1.621%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-key-b-matched`	8205	70.518%	-
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-key-b-unmatched`	8247	70.062%	-
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-unwatermarked-matched`	8173	-	1.823%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`qwen-unwatermarked-unmatched`	8205	-	1.621%
`human-eli5-lfqa-test/key-b`	`eli5-human-lfqa`	`qwen-key-b-matched`	`wildchat-original`	6415	-	1.652%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`eli5-human-lfqa`	2103	-	0.571%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-key-a-matched`	8171	67.984%	-
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-key-a-unmatched`	8267	68.332%	-
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-key-b-matched`	8205	-	1.304%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-key-b-unmatched`	8247	-	1.297%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-unwatermarked-matched`	8173	-	1.505%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`qwen-unwatermarked-unmatched`	8205	-	1.438%
`other-model-wildchat-original/key-a`	`wildchat-original`	`qwen-key-a-matched`	`wildchat-original`	6415	-	0.857%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`eli5-human-lfqa`	2103	-	0.903%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-key-a-matched`	8171	-	1.163%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-key-a-unmatched`	8267	-	1.028%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-key-b-matched`	8205	69.336%	-
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-key-b-unmatched`	8247	69.237%	-
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-unwatermarked-matched`	8173	-	1.272%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`qwen-unwatermarked-unmatched`	8205	-	1.219%
`other-model-wildchat-original/key-b`	`wildchat-original`	`qwen-key-b-matched`	`wildchat-original`	6415	-	1.091%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`eli5-human-lfqa`	2103	-	0.856%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-key-a-matched`	8171	71.093%	-
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-key-a-unmatched`	8267	70.727%	-
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-key-b-matched`	8205	-	1.036%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-key-b-unmatched`	8247	-	1.116%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-unwatermarked-matched`	8173	-	1.236%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`qwen-unwatermarked-unmatched`	8205	-	1.280%
`same-model-matched/key-a`	`qwen-unwatermarked-matched`	`qwen-key-a-matched`	`wildchat-original`	6415	-	1.091%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`eli5-human-lfqa`	2103	-	0.808%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-key-a-matched`	8171	-	0.955%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-key-a-unmatched`	8267	-	0.726%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-key-b-matched`	8205	67.508%	-
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-key-b-unmatched`	8247	66.933%	-
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-unwatermarked-matched`	8173	-	0.783%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`qwen-unwatermarked-unmatched`	8205	-	0.670%
`same-model-matched/key-b`	`qwen-unwatermarked-matched`	`qwen-key-b-matched`	`wildchat-original`	6415	-	0.701%

The human-ELI5 and original-WildChat variants are not pure watermark-only classifiers: their positive and negative classes also differ by model and domain. Use same-model-matched/* for the clean incremental watermark comparison. detector-grid-report.json contains the other evaluated lengths and operating points; operating-point-table.csv is the machine-readable version of the table above.

Loading

python

from transformers import BayesianDetectorModel

detector = BayesianDetectorModel.from_pretrained(
    "xlr8harder/synthid-qwen3-4b-instruct-2507-detectors",
    subfolder="same-model-matched/key-a/model",
)

The model weights and sampling table do not contain the private watermark key. Scoring token IDs requires the matching experiment key to recompute SynthID g-values. The two experiment-only keys are intentionally public for exact reproduction and must not be reused for deployment:

Their original pre-generation commitments are included in every bundle.

These artifacts are for research evaluation, not production provenance or adversarially robust attribution.

xlr8harder/synthid-qwen3-4b-instruct-2507-detectors

Qwen3-4B SynthID detector family

Generation profile: synthid-paper-t0.7-k100-p1.0

Fixed operating-point transfer grid

Loading

xlr8harder/
synthid-qwen3-4b-instruct-2507-detectors

Generation profile: `synthid-paper-t0.7-k100-p1.0`