English plus EU-core (Italian, German, French, Spanish, Portuguese); broader multilingual coverage is inherited from Qwen2.5 but not explicitly evaluated here
Task
Multi-label binary classification across 12 safety categories
Input
Up to 1024 tokens of plain text
Output
12 independent sigmoid scores, one per category; unsafe verdict derived from max(scores) >= threshold, where the threshold is per use case
Weights license
Apache-2.0 (inherited from upstream DuoGuard)
Calibration + benchmark license
Proprietary (Independently Platform)
Serving config
See picoguardian_v3_config.json in this repository
The shipped artefact in this repository is the torch-fp16 CUDA serving checkpoint. An earlier ONNX INT8 (MatMulNBits weight-only) export exists in git history; it is superseded and should not be used — constant-output behaviour was observed after the transformer_memcpy + MatMulNBits fusion on RTX 40-series hardware. The current artefact loads cleanly via vanilla transformers with no custom ops.
Intended use
Primary use case. Screening user input and LLM output in production AI applications across EU-language markets. The model returns per-category probabilities and a calibrated binary verdict that downstream systems can route on.
Suitable for:
Pre-LLM prompt-injection gating (cheap filter before an expensive generation call).
Post-LLM output moderation (catch unsafe completions before they reach the user).
UGC triage: comments, reviews, forum threads, support tickets.
EU AI Act Article 12 audit logging (per-category scores are stable and deterministic given identical inputs).
Real-time ranking and trust-and-safety workflows.
Crisis-keyword escalation into human review.
Unsuitable for:
Law-enforcement or autonomous decision-making without a human in the loop.
Validating medical, legal, or financial advice.
Users under 13 without parental oversight.
Any decision where a false negative causes physical harm without a downstream review layer.
Safety classification is a statistical process. Operators are responsible for the overall pipeline, including the fallback when the classifier is wrong.
The API exposes the identical per-category output as the open-weight model; the offline-to-API parity delta on the v2.1 reference corpus is |macro| = 0.0024.
Safety categories
The 12 categories are inherited from the DuoGuard taxonomy. Index order is load-bearing: it maps directly to the columns of the logits tensor returned by the sequence-classification head.
Index
Slug
Label
0
violent_crimes
Violent crimes
1
non_violent_crimes
Non-violent crimes
2
sex_related_crimes
Sex-related crimes
3
child_sexual_exploitation
Child sexual exploitation
4
specialized_advice
Specialized advice
5
privacy
Privacy
6
intellectual_property
Intellectual property
7
indiscriminate_weapons
Indiscriminate weapons
8
hate
Hate
9
suicide_self_harm
Suicide and self-harm
10
sexual_content
Sexual content
11
jailbreak_prompts
Jailbreak prompts
The hosted API exposes the same mapping at GET /v1/meta/categories.
Per-use-case calibration
This is the Picoguardian contribution on top of DuoGuard. The raw classifier returns 12 per-category sigmoid probabilities; turning those into a binary safe/unsafe verdict requires a threshold. A single global threshold is not defensible across use cases — an ugc_comments stream tolerates a very different precision/recall balance than llm_output_gate.
We evaluated DuoGuard's raw scores across ten real-world moderation use cases on an 84,440-row stratified corpus (v2.1). For each use case, we swept τ across [0.05, 0.95] and picked the τ that maximised accuracy on the held-out slice. The resulting presets ship inside picoguardian_v3_config.json in this repository.
Use case
Preset τ
Accuracy at preset τ
prompt_injection
0.20
91.67%
llm_output_gate
0.90
69.60%
ugc_comments
0.80
93.49%
forum_chat
0.90
95.82%
reviews
0.90
96.46%
support_tickets
0.70
95.63%
ranking
0.80
92.90%
ai_act_logs
0.80
93.38%
multilingual
0.75
81.83%
critical_escalation
0.40
85.11%
Global fallback τ = 0.75 (the macro-optimum across the same corpus).
Resolution order for the effective threshold on any request:
In other words: an explicit per-request threshold wins; otherwise the caller's identity-level default wins; otherwise the preset associated with the declared use_case wins; otherwise the global default. Do not treat these presets as universal — pick the use case that best matches your product surface, or sweep τ yourself on your own labelled data.
Benchmark
Methodology
Corpora: two independent evaluation sets.
The 84,440-row v2.1 reference corpus used for calibration (see build_bench_corpus in picoguardian_v3_config.json).
A 20,000-row 3-way refresh (2,000 per use case; one forum_chat row dropped at the API's 8,000-char cap → N = 19,999) run on 2026-04-18 to compare against two public baselines.
Comparators:
Picoguardian v3 at its per-use-case preset threshold.
Metric: binary verdict accuracy vs. the dataset's native ground-truth label. Macro-averaged across use cases (not across raw rows) so minority use cases are not drowned out.
Headline numbers (20k 3-way, 2026-04-18)
Model
Macro accuracy
Picoguardian v3
89.35%
OpenAI omni-moderation-latest
84.51%
Llama-Guard-4-12B
83.74%
Offline torch-fp16 reference macro on the same 20k slice: 89.59%. Offline-to-API delta: −0.24 pp (|macro delta| = 0.0024, well within the 0.003 parity gate).
Per-use-case (20k 3-way)
Use case
N
Preset τ
Picoguardian
LG4
OAI Mod
Winner
prompt_injection
2000
0.20
87.40%
85.25%
80.10%
Picoguardian
llm_output_gate
2000
0.90
76.35%
75.70%
71.49%
Picoguardian
ugc_comments
2000
0.80
93.75%
83.55%
85.36%
Picoguardian
forum_chat
1999
0.90
94.35%
91.70%
93.96%
Picoguardian
reviews
2000
0.90
96.60%
92.60%
94.49%
Picoguardian
support_tickets
2000
0.70
95.35%
94.50%
95.88%
OAI Mod (by 0.53 pp)
ranking
2000
0.80
91.75%
82.35%
84.22%
Picoguardian
ai_act_logs
2000
0.80
92.95%
82.50%
84.91%
Picoguardian
multilingual
2000
0.75
81.35%
69.00%
76.51%
Picoguardian
critical_escalation
2000
0.40
83.70%
80.30%
78.19%
Picoguardian
Picoguardian wins 9 of 10 use cases outright. The one loss (support_tickets) is 0.53 pp behind OpenAI Moderation and is kept in the table rather than filtered out of the published comparison.
Live API scoring on the same corpus produces a macro delta of ≤0.003 vs. the offline reference, i.e. API callers see effectively the same classifier as self-hosters of this checkpoint.
Evaluation corpus
Rows are drawn from public safety datasets and stratified by use case. No hand re-labelling: each row's true_label is extracted from the source's native schema.
The stratification scheme and per-use-case source mapping ship in the training repository (guard-model-training/) for reproducibility; the exact sampling logic is deterministic given the fixed random seed.
Limitations
Be honest about what this model is and is not.
1024-token context limit. Long documents are truncated. If you need to classify an article, chunk it and aggregate scores yourself (max works well; mean smears signal).
English is the strongest language. EU-6 (DE, FR, ES, IT, PT) is strong; other languages are inherited from Qwen2.5 but are out-of-distribution relative to the calibration corpus, and the multilingual preset accuracy (81.83%) reflects that harder setting.
Binary verdict hides per-category nuance. For adversarial or borderline content, consumers should read the full categories dict, not just the boolean verdict.
Text only. No image, audio, or video input. Multimodal inputs must be converted to text upstream (captions, transcripts) with the attendant losses.
Per-use-case calibration. A single threshold is not meaningful across categories with very different base rates. Stick to the preset for your use case, or calibrate your own τ on labelled data from your production distribution.
Base-rate sensitivity. The calibration corpus mixes safe and unsafe at ratios tuned per use case. Applying the model to a stream with a very different prior (e.g., 99% safe) will shift the precision/recall trade-off — expect to re-tune τ in that case.
Adversarial robustness. No specific defences against obfuscated attacks (leet-speak, zero-width characters, language mixing). The jailbreak_prompts category is trained on PolyGuardMix / AdvBench-style attacks; novel families will require retraining.
Probabilities are not calibrated as such. The raw sigmoids are usable for ranking and thresholding; do not interpret a raw 0.30 as "30% probability of being unsafe" in a Bayesian sense.
Serving
Recommended: hosted API
POST https://picoguardian.online/v1/guard
Free tier, paid tier, and enterprise self-host are all supported; see picoguardian.online for pricing and SLAs. The hosted API adds per-identity rate limiting, usage metering, replay history, audit logging, and the resolution-order logic for thresholds described above — none of which are part of the model itself.
Any HF-compatible inference server (TGI, vLLM, Triton with the Python backend, a FastAPI wrapper, etc.) will work. There are no custom ops.
Hardware. 494M parameters fit on any ≥4 GB consumer GPU at fp16. For reference, the production API sustains 2,400+ requests per second on a single RTX 4060 (8 GB VRAM) with p99 ≈ 261 ms at batch=64, seq=1024. The exact batching + IOBinding + opportunistic CUDA-Graph setup is documented in the serving repository.
Citation
If you use this model, please cite both the upstream DuoGuard paper and the Qwen2.5 base:
bibtex
1@misc{deng2025duoguard,
2 title = {DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails},
3 author = {Yihe Deng and Yu Yang and Junkai Zhang and Wei Wang and Bo Li},
4 year = {2025},
5 eprint = {2502.05163},
6 archivePrefix= {arXiv},
7 primaryClass = {cs.CL},
8 url = {https://arxiv.org/abs/2502.05163}
9}
1011@misc{qwen2_5,
12 title = {Qwen2.5 Technical Report},
13 author = {{Qwen Team}},
14 year = {2024},
15 url = {https://qwenlm.github.io/blog/qwen2.5/}
16}
1718@misc{picoguardian_v3,
19 title = {Picoguardian v3: Use-Case-Calibrated Multilingual Safety Classification},
20 author = {{Independently Platform}},
21 year = {2026},
22 howpublished = {\url{https://huggingface.co/independently-platform/picoguardian-v3}}
23}
Issue tracker (fallback): open an issue on this HuggingFace repository's discussion tab
Changelog
v3 — 2026-04-18. Shipping torch-fp16 CUDA serving checkpoint; full 12-category API exposure; EU-core multilingual evaluation; 9 of 10 use-case wins in the 20k 3-way bench vs. Llama-Guard-4-12B and OpenAI omni-moderation-latest. The previous ONNX INT8 artefact (commit e0c1d049) is superseded and should not be used.
