You can deploy this model and start earning money today!

ningpy/
intent-detection-V2.3

Name: intent-detection-V2.3
Availability: InStock

Views

No views yet

Intent Detection V2.3 (V12.9)

Health-app user intent classifier + safety router fine-tuned from Qwen2.5-7B-Instruct. Routes user messages into 5 states: PASS, REDFLAG, BLOCK, STEER, or COEXIST (emergency + personal-dose).

Performance (val set: 2518 samples)

Metric	Score
Safety F1 (REDFLAG ∪ BLOCK)	92.3%
Multi-label EM	88.0%
Joint acc (intent + all types)	81.5%
Single-label acc (v11-compatible projection)	90.1%
Intent acc	93.5%
Coexist recall (REDFLAG + BLOCK)	76.2%
Parse err	0.16% (of which ~half are eval-side bugs — model output is valid)
REDFLAG reason_code acc	98.3%

Improvements vs V11 (single-label):

Safety F1: 90.9% → 92.3% (+1.4pp)
REDFLAG F1: 90.0% → 94.1% (+4.1pp)
New multi-label capability: 76.2% coexist recall (v11 was 0%)

Output Schema (V12.9 flat)

6 fixed keys, deterministic order:

json

{
  "intent": "HA|HCN|MT|PI|OTHER",
  "redflag": true|false,
  "redflag_code": "RF-1_GENERAL|RF-2_PEDIATRIC|RF-3_OBSTETRIC|RF-4_ELDERLY|RF-5_IMMUNOCOMP|\"\"",
  "block": "BLOCK_HARD|STEER_SOFT|PASS|NONE",
  "block_code": "OOS_H_*|OOS_S_*|PASS_*|\"\"",
  "entities": {
    "drugs": ["<verbatim from user text>"],
    "diseases": ["..."],
    "symptoms": ["..."]
  }
}

Entities are verbatim substrings of the user message (no translation/normalization).

5 Valid State Combinations

State	redflag	block	Use case
PASS	false	PASS	Normal health knowledge Q
REDFLAG only	true	NONE	Emergency without personal-dose Q
BLOCK only	false	BLOCK_HARD	Personal action / off-topic
STEER only	false	STEER_SOFT	Personal clinical judgment
COEXIST	true	BLOCK_HARD	Emergency + personal-dose

Training data

54,404 samples (English + Chinese + Bruneian Malay)
Base 27K + STEER×3 + coexist×8 + patch13×3 (targeted narrow rules) + patch16×5 (anti-drift + narrow)
57.1% samples have non-empty entities (GPT-4o back-filled)

Usage with vLLM

bash

python -m vllm.entrypoints.openai.api_server \
    --model ningpy/intent-detection-V2.3 \
    --served-model-name intent \
    --dtype float16 \
    --port 8001

Client sample

See https://git.evyd.tech/peiyan.ning/intent-model-v12/tree/main/sample for a Python client with routing + response templates.

Recommended request settings:

temperature: 0.0
max_tokens: 200
stop: [] (rely on model's natural <|im_end|>)

Downstream routing

Model output	Action
`redflag=true, block="BLOCK_HARD"`	Emergency guidance + refuse dose advice
`redflag=true, block="NONE"`	Emergency guidance
`redflag=false, block="BLOCK_HARD"`	Refuse + refer to doctor
`redflag=false, block="STEER_SOFT"`	Soft guide to consultation
`redflag=false, block="PASS"`	Normal RAG/LLM answer