ScrubData planner — Qwen3-4B fine-tuned for tabular cleaning plans
A ≤4B planner for hands-off data cleaning: it reads an aggregated column profile
(per-value frequency counts) and emits a structured JSON cleaning plan that a
deterministic pandas executor applies. Built for the Build Small Hackathon
(🏡 Backyard AI · Tiny Titan · Well-Tuned).
Every training example is execution-verified: a candidate (dirty table, plan) pair is
kept only if running the executor on it provably recovers the known-clean table. Mix:
synthetic high-cardinality categorical tables (Zipf long-tail + realistic typos) + 20%
real-derived pairs from the Raha benchmarks (cell-aligned, learnable canonicalizations only).
Shipped composition (WS1 — verified union planner): in the product, every
model-proposed mapping is scored by a deterministic verifier (errors-are-rare frequency
gates, variant similarity, reference agreement; threshold SCRUBDATA_TAU, default 0.5)
and unioned with the grounded heuristic. Measured on hospital's 509 real errors:
0.905 precision @ 0.413 coverage (gated model plan alone: 0.993 @ 0.287 — 146/147
committed changes correct; seed-robust: 0.891 ± 0.012 @ 0.396 ± 0.025 over 3 training
seeds). Dropped merges become review flags, never silent skips.
Measured
Canonicalization micro-F1 0.90 (best single run; 0.80 ± 0.01 over 3 training seeds) (vs 0.45 for a much larger zero-shot generic model,
0.13 for a rule heuristic) on frozen held-out gold.
Real hospital typos (Raha, OOD): repair recall 0.00 → 0.42 from adding the
real-derived 20% (synthetic-only fails to transfer — documented honestly).
In production the model is wrapped with reference grounding + calibrated abstention
(it never free-generates a canonical for a grounded column type).
Post-freeze system (v2, June 2026) and the central finding
The shipped pipeline around this model gained four deterministic capabilities —
bounded suspect surfacing for high-cardinality columns, a generic entity
reference (exact-hit typing floor), cross-row majority voting with a
false-consensus guard, and convention-conservatism gates. Measured on the
WildClean benchmark:
unseen-source macro F1 0.363 @ damage 0.0219 (35 unseen-source pairs of the
42-pair benchmark; full-bench 0.343 @ 0.023), 0 silent edits
across 35 wild tables and a 239-table GitTables trust audit.
Honest scope of these weights: five further fine-tunes and a 3-arm GRPO pilot
(executor as verifiable reward) all failed to move held-out generalization — at this
scale the weights contribute format competence and in-distribution skill; the
deterministic machinery + plan-level verifier carry never-seen-table generalization.
Details in the paper. In the same verify+union harness, two of three zero-shot 24–31B
open-weights planners exceed this operating point (0.915 @ 0.485 vs 0.905 @ 0.413,
paper §scaling); the 4B remains the most precise gated planner at usable coverage and
the only locally-measured one — the architecture, not the fine-tune, is the portable
contribution.
How to run
Ollama / llama.cpp (recommended): use the non-thinking Modelfile from the Space repo
(notebooks/Modelfile). Q8_0 GGUF: ricalanis/scrubdata-qwen3-4b-v6-q8 (Q4_K_M corrupts
this model on Unsloth 2026.6.x exports — use Q8_0).
Transformers (bf16 + adapter): suppress the tool-call tokens at decode time or the
base model's tool-calling prior dominates:
English-centric; plans use a closed op vocabulary; canonicalization quality on entity
columns depends on the reference taxonomy's coverage; not a de-identification guarantee.
