Phase 3 — Product Domain + Taxonomy + Product Grouping
Compliments Reference DB Pipeline
Version: 1.1.0
Date: 2026-07-30
Status: PASS
Input
| Property | Value |
|---|
| Source | phase2/outputs/phase2_output.parquet |
| HF Repo | saraNour/compliments-reference-phase2 |
| Row Count | 4,440 |
| Column Count | 38 |
What Phase 3 Does
A. Food / Non-Food Classification
Deterministic keyword-based rules. Creates product_domain column with values: food, non_food, unknown.
- Food: Dairy, meat, seafood, bakery, beverages, frozen, produce, pantry, snacks, confectionery
- Non-food: Pet food, medications, personal care, household cleaning, kitchenware, health remedies
- Unknown: Products that don't match any rule
B. Taxonomy Classification
Deterministic keyword-based taxonomy. Classification metadata only — NOT used as grouping key.
19 categories: DAIRY, MEAT_SEAFOOD, BEVERAGES, BAKERY, FROZEN, BREAKFAST, PASTA_RICE, CANNED_GOODS, CONDIMENTS_SAUCES, SNACKS, CONFECTIONERY, PRODUCE, PERSONAL_CARE, HOUSEHOLD_CLEANING, HEALTH_REMEDIES, PET_FOOD, HOUSEHOLD_SUPPLIES, BABY_CARE, GENERAL_GROCERY
C. Product Grouping
Identity-based deterministic grouping:
group_key = identity_hash + product_domain + normalized_core_title
identity_hash: 14-field hash from Phase 2 (9 flags + fat + product_line + flavour + formulation)product_domain: food/non_food/unknown — structurally prevents food/non-food mixingnormalized_core_title: Core title with generic packaging words removed, tokens sorted
D. Ambiguous Case Detection
Groups with >1 product are inspected for:
- Multiple core_titles
- Multiple flavours
- Multiple formulations
- Multiple fat levels
- Multiple product lines
E. Rule-Based Resolution
All grouping is deterministic. No LLM was required.
Output Files
A. reference_product_catalog.csv (Group-Level)
| Column | Type | Description |
|---|
| group_id | str | Deterministic UUID |
| group_name | str | Most frequent core_title |
| brand | str | Normalized brand |
| product_domain | str | food/non_food/unknown |
| taxonomy | str | Taxonomy category |
| identity_hash | str | 14-field identity hash |
| product_count | int | Products in group |
| unique_upcs | int | Unique UPCs in group |
B. product_group_mapping.csv (Product-Level)
| Column | Type | Description |
|---|
| upc | str | Barcode |
| external_id | str | Voila ID |
| group_id | str | Group UUID |
| group_name | str | Core title |
| core_title | str | Core title |
| original_title | str | Raw title |
| brand | str | Normalized brand |
| size | str | Raw size |
| variant_attributes | str | Parsed size JSON |
| identity_hash | str | Identity hash |
| product_domain | str | Food/non-food |
| taxonomy | str | Taxonomy |
| product_line | str | Product line |
| source | str | Data source |
| source_url | str | Source URL |
C. ambiguous_cases.csv
Groups with potential ambiguity and the reason.
D. unknown_products.csv
Products classified as "unknown" domain (1,243 products).
Deterministic Grouping Rules
| # | Rule | Implementation |
|---|
| 1 | Identity hash captures 14 fields | build_identity_hash() in Phase 2 |
| 2 | Product domain is part of grouping key | build_group_key() in Phase 3 |
| 3 | Core title normalized (generic words removed, sorted) | normalize_title_for_grouping() |
| 4 | Same normalized title + same identity = same group | Group key construction |
| 5 | Different identity = different group | Identity hash is first component |
| 6 | Food vs non_food = always different group | Product domain is second component |
Food/Non-Food Rules
100+ deterministic regex rules based on product title keywords.
- Pet food → non_food
- Medications (ibuprofen, acetaminophen, etc.) → non_food
- Personal care (tampons, shampoo, etc.) → non_food
- Household (detergent, garbage bags, etc.) → non_food
- Kitchenware (light bulbs, pans, etc.) → non_food
- Everything else matching food keywords → food
- No match → unknown
Regression Tests — ALL PASS (12/12)
| Test | Result |
|---|
| PB smooth sizes same group | PASS |
| PB crunchy sizes same group | PASS |
| PB organic smooth different | PASS |
| PB organic crunchy different | PASS |
| PB naturally simple different | PASS |
| PB light different | PASS |
| PB honey different | PASS |
| PB cookies different | PASS |
| PB ice cream different | PASS |
| PB dog treats different | PASS |
| CC 1% vs 2% different | PASS |
| Cottage cheese sizes same | PASS |
Validation Results
| Check | Result |
|---|
| Row count preserved | PASS |
| No duplicate mappings | PASS |
| No food/non-food mixing | PASS |
| No brand conflicts | PASS |
| Identity consistent | PASS |
| Taxonomy populated | PASS |
| No empty group names | PASS |
| One group per product | PASS |
| No orphans | PASS |
| Overall | PASS |
Files
phase3/
README.md
src/phase3.py
notebooks/phase3.ipynb
outputs/
reference_product_catalog.csv
product_group_mapping.csv
ambiguous_cases.csv
unknown_products.csv
phase3_executed.ipynb
validation/
phase3_validation.json
statistics/
phase3_statistics.json
audit/
phase3_1_unknown_audit.csv
phase3_1_unknown_audit.parquet
phase3_1_audit_summary.parquet
peanut_butter_grouping_example.parquet
notebooks/
phase3_1_audit.ipynb
How to Run
1cd /home/sara/gsoc/compliments-reference-db
2python -m phase3.src.phase3
LLM Usage
None. All grouping is deterministic.
Handoff
Phase 3 outputs are the frozen product-grouping layer.
Future phases will add:
- Phase 4: Variant assignment
- Phase 5: Nutrition merge
- Phase 6: Nutrition scoring (NutriScore, NOVA, EcoScore)
- GPC integration
- Agribalyse integration
