This repository contains a powerful Level-3 stacked ensemble model designed for optimal fertilizer prediction. The model architecture uses over 80 diverse base learners, multiple advanced ensemble strategies, and robust feature engineering techniques to achieve state-of-the-art log loss performance.
Best LogLoss: 1.86257 (Ridge Ensemble) Hill Climbing LogLoss: 1.86554
📊 Evaluation Metric
LogLoss was used as the primary evaluation metric to assess model performance.
🧱 Architecture
The model follows a 3-tier ensemble stacking structure:
Level 1 - Diverse Base Models
Includes over 80+ models from the following families:
AutoGluon (27 models)
MLP (x2)
XGBoost (x20) – tuned with bagging and early stopping
LGBM GBDT (x4) and LGBM GOSS
TabTransformer (x3)
Neural Networks (NNx15) – including deep tabular variants
CatBoost (x2)
HistGradientBoost (HGBx2)
YDF (Yandex Decision Forest)
Feature Engineering Highlights:
✅ Binned Features – Numerical columns transformed into bins to capture non-linear effects.
✅ All Numerical → Categorical – Applied label encoding or one-hot encoding to convert features.
✅ Data Augmentation using Train+Orig blending.
Level 2 - Intermediate Ensembles
Various ensemble strategies were applied to Level-1 predictions:
🔁 Logistic Regression (LR)
🧱 Voting Classifier
🎯 Stacking Classifier
⚖️ Weighted Ensemble
📊 Cluster Averaging
🗳️ Weighted Voting Classifier
Level 3 - Final Meta-Ensemble
🔼 Hill Climbing
🧠 Ridge Ensemble (Best Performer)
🏆 Performance Summary
Level
Description
Score (LogLoss)
1
Base models
1.94 – 1.88
2
Intermediate ensembles
1.88 – 1.87
3
Ridge & Hill Climbing
1.86257 (best)
🖼️ Model Architecture
Model_Architecture (1).jpg
📁 Files Included
Swandip_optimal_fertilizer_model.joblib – Trained ensemble model
README.md – This file
architecture.jpg – Visual representation of the architecture
