Baseline CNN RGB + Depth + Side-angle Carbohydrate Regression
This model predicts dish-level carbohydrate content from:
- overhead RGB image
- overhead depth image
- one side-angle RGB image at a time
All available side-angle images are used during training by expanding each dish into multiple samples.
Dataset
Based on Nutrition5K compact preprocessing.
Split counts (rows after side-view expansion)
- Train rows: 17348
- Validation rows: 2652
- Test rows: 3596
Unique dishes
- Train dishes: 2303
- Validation dishes: 334
- Test dishes: 474
Architecture
- RGB overhead CNN branch
- Depth overhead CNN branch
- Side-angle RGB CNN branch
- Feature fusion via concatenation
- Dense regression head
- Target:
total_carb
Training
- Loss: MSE
- Metrics: MAE, MSE
- Optimizer: Adam
- Best checkpoint selected by validation loss
Notes
Each available side-angle image becomes a separate sample while keeping the same overhead RGB and depth image for the corresponding dish.
