Stacked GBM Ensemble for Income Classification (OpenML Task 7592)
Weighted ensemble of LightGBM, XGBoost, and CatBoost trained on the Adult Income dataset (UCI / OpenML task 7592). Hyperparameters optimised with Optuna (105 trials, TPE sampler). Evaluated under the standard 10-fold stratified CV protocol defined by OpenML.
Results outperform the best recorded run on the OpenML leaderboard (AdaBoost, 2017).
Model
AUC-ROC
Accuracy
This ensemble
0.9315
0.8760
OpenML best (AdaBoost, 2017)
0.9284
0.8740
LightGBM alone
0.9301
—
XGBoost alone
0.9302
—
CatBoost alone
0.9310
—
Method
Features (28 total). Six raw numeric features augmented with log-transformed capital variables, binary flags, age/hours bins, and two interaction terms (education-num × age, education-num × hours-per-week). Categorical columns encoded with OrdinalEncoder for LightGBM/XGBoost; CatBoost receives them natively.
Ensemble. Out-of-fold predictions from the three base learners are combined with fixed weights (LGB 0.1 / XGB 0.3 / CB 0.6). Decision threshold tuned on OOF predictions (0.512).
Tuning. Optuna TPE, 3-fold inner CV: 40 trials for LightGBM, 40 for XGBoost, 25 for CatBoost.
