Priority-Ordered Attribute Extraction from Industrial Search Queries
Fine-tuning Gemma 4 E4B with LoRA — Full Research Record
Status: Research Complete (V11 is final run) | Archived: 12 May 2026 Base Model:gemma-4-e4b-it Task: Extract product attributes from B2B search queries, ranked by importance
What This Project Does — In Plain English
When a buyer types a search query like Siemens 1 kW 3 phase servo motor, a smart search system needs to understand:
What attributes are in that query (brand, power, motor type, etc.)
Which attribute matters most — brand identity? product type? technical spec?
What are the right canonical names for each attribute (e.g., "brand" vs "manufacturer" vs "company" — which is preferred?)
Standard attribute extractors give you an unordered list. This project builds a model that outputs a ranked, structured list — most important attribute first, with preferred key names ordered too.
This is harder than regular extraction, and that difficulty is the research story.
The Output Format (What the Model Learns to Produce)
The project started by training Gemma 4 to extract attributes as a flat, unordered JSON — simpler than the final goal but necessary to establish that the model can learn canonical key names at all.
V3b: Open key names, good value recovery, bad key canonicalization
V5: Switched to IndiaMART-style canonical keys, added motor-domain data
V6: Strict canonical key list, cleaned training data, gold supervision → beat qmeans on exact flat match
Phase 2: Priority-Ordered Extraction (V7 → V11)
Harder: the model now has to emit a nested ranked structure (not flat), order attributes by importance, and order key synonyms within each attribute.
Run
Why it was tried
What happened
V7
Introduced nested priority schema
First useful result; 29% key F1, 15.5% key+value F1
V8
Larger model capacity + more aggressive loss
Recall dropped from 19% → 9%
V9
Even heavier gold weighting
Recall dropped to 2.5% — model became over-conservative
V10
Clean-only training, no noisy examples
0% recall — complete failure
V11
Rolled back to V7 recipe, balanced data mix
Recovered to 45.4% key F1, 20.3% key+value F1
The key lesson: More aggressive training made things worse. The V11 recovery came from going back to a simpler, more balanced approach.
Phase 3: Hybrid Serving
Rather than choosing between the fine-tuned model and qmeans, we combine them:
Parses the nested JSON output (handles malformed nesting)
Flattens prediction and gold to {key: value} sets
Computes precision, recall, F1 for key-only and key+value matches
Important: The evaluator was fixed mid-project when we discovered V7 was being scored near-zero due to extra nesting levels in outputs. The corrected evaluator recursively unwraps nested priority bodies. See priority_error_analysis_notes.md for full details.
