A 1.7B specialist that ranks a batch of deduplicated product findings by
impact, frequency, and effort, returning strict JSON with a reason per item.
One of three tools in the
distil-posthog-traffic-analyser
harness; trained on the Distil Labs platform.
Task contract
Input:
Findings (N):
[ {"id": "...", "kind": "bug|gap", "severity": 1-5,
"occurrences": <int>, "title": "...", "evidence": "..."}, ... ]
Return JSON now. Every id above must appear in "ranked" exactly once.
Output — valid JSON only, matching:
{"ranked":[{"id":"<input id>","rank":1,"reason":"<=2 sentences citing the numbers"}]}
Hard constraint: every input id appears exactly once — no dropped, invented,
or duplicated ids. Ranking judgment encoded in training: security issues
lead regardless of frequency; revenue blockers rank above equally severe
non-revenue issues; silent data corruption outranks loud errors; at equal
severity, reach wins; cosmetic items rank last.
Training
Base model: Qwen3-1.7B (Apache 2.0)
Teacher: openai.gpt-oss-120b (Apache 2.0)
Seed data: 22 hand-authored, schema-validated ranking batches
(single-item to 4-item, including equal-severity ties and
security/revenue trade-offs), committed at
examples/seeds/prioritizer.jsonl
(18 train / 4 held-out test); a coverage validator guarantees every seed
batch ranks each finding exactly once
Synthetic expansion: 10,004 examples generated and validated by the
Distil Labs platform from the seed set
A 0.6B variant was trained first; it ranked held-out batches correctly but
violated the exactly-once coverage constraint on a batch containing
near-duplicate findings. The 1.7B student holds exact coverage on that same
adversarial batch, which is why this size ships.
Evaluation
Held-out test set (n=4), scored by the platform's LLM judge (an answer fails
outright on any coverage violation, making this judge deliberately strict):
Untrained Qwen3-1.7B
Teacher (gpt-oss-120b)
This model
LLM-as-a-Judge
75.00% (3/4)
75.00% (3/4)
75.00% (3/4)
ROUGE
38.96%
47.17%
50.31%
The judge score saturates because both pass its format bar on most batches;
live behavior separates them. This model, on live batches (this GGUF via
Ollama, matching the hosted endpoint): exact id coverage on 6/6 —
4 held-out seed batches, one adversarial near-duplicate batch, and one live
pipeline batch fed by the other two tools — with rank-1 choices matching the
hand-written gold ranking on 3 of the 4 held-out batches and a defensible
judgment call on the fourth.
Usage (Ollama)
ollama create posthog-prioritizer -f Modelfile # FROM ./<this gguf>
