1pip install torch torchvision open_clip_torch peft pillow
2pip install huggingface_hub datasets transformers
1from huggingface_hub import hf_hub_download
2import torch
3
4# Download model checkpoint
5model_path = hf_hub_download(
6 repo_id="shawneil/Amazon-ml-Challenge-Model",
7 filename="best_model.pt"
8)
9
10# Load model (see GitHub repo for complete model definition)
11model = OptimizedCLIPPriceModel(clip_model)
12model.load_state_dict(torch.load(model_path, map_location='cpu'))
13model.eval()
1from PIL import Image
2import open_clip
3import torch
4
5# Load CLIP processor
6clip_model, _, preprocess = open_clip.create_model_and_transforms(
7 'ViT-L-14', pretrained='openai'
8)
9tokenizer = open_clip.get_tokenizer('ViT-L-14')
10
11# Prepare inputs
12image = Image.open("product_image.jpg")
13image_tensor = preprocess(image).unsqueeze(0)
14
15text = "Premium Organic Coffee Beans, 16 oz, Medium Roast"
16text_tokens = tokenizer([text])
17
18# Extract 40+ features (see feature engineering guide)
19features = extract_features(text) # Your feature extraction function
20features_tensor = torch.tensor(features).unsqueeze(0)
21
22# Predict price
23with torch.no_grad():
24 predicted_price = model(image_tensor, text_tokens, features_tensor)
25 print(f"Predicted Price: ${predicted_price.item():.2f}")
Product Image (512×512) ──┐
├──> CLIP Vision (ViT-L/14) ──┐
Product Text ─────────────┼──> CLIP Text Transformer ───┤
│ ├──> Feature Attention ──> Enhanced Head ──> Price
40+ Features ─────────────┘ │ (Self-Attn + Gate) (Dual-path +
(Quantities, Categories, │ Cross-Attn)
Brands, Quality, etc.) │
1{
2 "epochs": 3,
3 "batch_size": 32,
4 "gradient_accumulation": 2,
5 "effective_batch_size": 64,
6 "learning_rate": {
7 "vision": 1e-6,
8 "text": 1e-6,
9 "head": 1e-4
10 },
11 "optimizer": "AdamW (betas=(0.9, 0.999), weight_decay=0.01)",
12 "scheduler": "CosineAnnealingLR with warmup (500 steps)",
13 "gradient_clip": 0.5,
14 "mixed_precision": "fp16"
15}
Total Loss = 0.05×Huber + 0.05×MSE + 0.65×SMAPE +
0.15×PercentageError + 0.05×WeightedMAE + 0.05×QuantileLoss
Where:
- SMAPE: Primary competition metric (65% weight)
- Percentage Error: Relative error focus (15%)
- Huber: Robust regression (δ=0.8)
- Weighted MAE: Price-aware weighting (1/price)
- Quantile: Median regression (τ=0.5)
- MSE: Standard regression baseline
1@misc{rodrigues2025amazon,
2 title={Amazon Product Price Prediction using Multimodal Deep Learning},
3 author={Rodrigues, Shawneil},
4 year={2025},
5 publisher={Hugging Face},
6 howpublished={\url{https://huggingface.co/shawneil/Amazon-ml-Challenge-Model}},
7 note={SMAPE: 36.5\%}
8}
Built with ❤️ using PyTorch, CLIP, and smart feature engineering
From 52.3% to 36.5% SMAPE - Multimodal learning at its best
