Celeb-FBI body-weight from a single image — best config A(ViT)+C (7.04 kg MAE)

Research artifact — do not use for medical, fitness, or individual assessment. Trained on celebrity images; under-predicts heavier subjects.

Download & run the best model — A(ViT)+C

Model Card — Human Weight Prediction from a Full-Body Image

1. Model details

• Architecture. ResNet-50 (ImageNet-pretrained) trunk + regression head Linear(2048→256) → ReLU → Dropout(0.3) → Linear(256→1).
• Hybrid variants.
  • Feature fusion (FusionRegressor): CNN 2048-d penultimate features concatenated with a small MLP over a 7-d anthropometric vector → shared head.
  • Late fusion: per-branch predictions (CNN, XGBoost-on-anthro, volume prior) combined by a linear meta-learner → A+B, A+C, A+B+C.
• Branch C — 3D body volume (best config). Estimates a body volume from the pose landmarks (elliptical-frustum torso, cylinder limbs, ellipsoid head), anchors it to metres with the subject's height, and calibrates `w = 300.6·V
  • 43.2(the practical form ofw ≈ ρV). The only branch supplying *absolute* scale; an exact SMPL-mesh volume backend is scaffolded. On a ResNet-50 backbone A+C reaches 7.82 kg; on the strongest backbone (ViT-B/16) the same volume fusion — **A(ViT)+C`** — is the best model overall at 7.04 kg (§4). Inference auto-detects the checkpoint's backbone and applies the matching A+C meta-learner.
• Anthropometric features (7-d). Six scale-invariant pose features — shoulder-width/torso, hip-width/torso, shoulder/hip ratio, leg-length/torso, arm-length/torso, bbox width/height — plus a pose_valid flag. Extracted with MediaPipe PoseLandmarker (Tasks API, 33-point BlazePose). Normalising by torso length makes them invariant to camera distance (a single image has no absolute scale).
• Training. Huber (Smooth-L1) loss; AdamW with split LRs (head 1e-4, backbone 1e-5, weight-decay 1e-4); cosine schedule; AMP on CUDA; early stopping on validation MAE (patience 7); batch 64; ≤40 epochs; input 224×224 with ImageNet normalisation. Targets standardised (mean 65.8, std 16.0 kg); metrics inverted to kg.
• Frameworks. PyTorch / torchvision, MediaPipe, scikit-learn, XGBoost.

2. Intended use

• Intended. Research and education — studying how far appearance + body-shape cues can estimate weight from one image; coursework / IEEE write-up.
• Out of scope (do not use for). Any medical, clinical, insurance, fitness, employment, or individual assessment. Outputs are point estimates with large per-person error (see §5) and a known directional bias against heavier bodies.

3. Data

• Source. Celeb-FBI (alecccdd/celeb-fbi) — 7,208 full-body celebrity photos.
• Splits. Provided test split (721) held out untouched. Training pool split 90/10 into train 5,032 / val 559, BMI-stratified and seeded (seed 42); all ablation models share this identical split.
• Labels. weight (kg, target), height (cm, for BMI), gender, age; missing values use a -1 sentinel and are dropped (units confirmed kg/cm in EDA).
• Pose coverage. Landmarks detected on 6,480 / 6,487 train (99.9%) and 721 / 721 test images, so the anthropometric branch is real signal, not imputation.
• Leakage. Per-image id checked across splits — no id shared. Residual risk remains: a celebrity may appear under multiple ids, so subject-level leakage cannot be fully excluded (documented limitation).

4. Metrics (test set, 721 images)

5. Error analysis & limitations

• Dominant failure: heavy subjects are under-predicted. The worst-20 CNN cases are large/muscular men (true 102–109 kg) predicted ~70 kg — 30+ kg under. The model is accurate (~6 kg) on the well-represented normal/underweight bands and collapses on the rare overweight/obese bands.
• The 3D volume branch is the fix — partially. A+C recovers the overweight band (13.9 → 12.3 kg) by injecting absolute size; but the obese band stays (14.7), because with only 66 obese images the volume map cannot calibrate the heavy tail either. So the bottleneck is both missing absolute scale (architectural, fixed by Branch C) and data coverage (which Branch C cannot fix).
• Why size matters. Weight fundamentally needs size, which a single image lacks; 2D scale-invariant ratios supply shape but not size (hence anthropometric-only is weakest, 9.7 kg). Branch C recovers absolute size by anchoring an estimated body volume to the subject's height — the only branch that does so.
• Other risks. Heavy clothing, unusual poses, occlusion, and extreme framing degrade both the CNN and pose features. Gender/age fairness beyond the gender/BMI breakdowns here was not exhaustively audited.

6. Ethical considerations

7. Reproduce the evaluation

1. pip install -r requirements.txt
2. Obtain the best checkpoint (*_best.pt, written to the Drive cache during training) and the cached pose-feature parquets.
3. Run notebooks/97_colab_eval.ipynb (or the equivalent src.eval calls) — it regenerates the ablation table, BMI confusion matrices, error-by-group charts, and the worst-k grid into reports/.