Diabetes Health Indicators, Model Comparison Study
This repository contains two machine learning models trained on the BRFSS 2015 Diabetes Health Indicators Dataset (balanced 50/50 split of 70,692 rows).
Task
Binary classification: Predict whether an individual has diabetes or prediabetes (Diabetes_binary = 1) versus no diabetes (Diabetes_binary = 0) based on 21 health indicators.
Models Included
- Tuned Random Forest: Regularized tree ensemble (
n_estimators=200, max_depth=10, min_samples_leaf=2).
- Tuned MLP: Feedforward neural network with 1 hidden layer of 128 units,
dropout=0.4, and Adam optimizer.
Final Performance Comparison
| Model | Accuracy | Precision | Recall | F1-Score |
|---|
| Random Forest (Baseline) | 0.7331 | 0.7176 | 0.7687 | 0.7423 |
| Random Forest (Tuned) | 0.7488 | 0.7281 | 0.7942 | 0.7597 |
| MLP (Baseline) | 0.7435 | 0.7179 | 0.8024 | 0.7578 |
| MLP (Tuned) | 0.7512 | 0.7264 | 0.8059 | 0.7641 |
Feature Importance (Random Forest Gini Importance)
The top indicators identified by the Random Forest model:
- BMI (Body Mass Index)
- Age (Age category 1-13)
- GenHlth (Self-reported general health rating 1-5)
- Income (Household income scale 1-8)
- HighBP (High blood pressure indicator)
Repository Files
This repository contains the following files:
diabetes_mlp_tuned.keras: The tuned Multilayer Perceptron (MLP) model.diabetes_random_forest_tuned.joblib: The tuned Random Forest classifier.scaler.joblib: The fitted StandardScaler used to normalize the feature inputs.diabetes_case_study.ipynb: The Jupyter notebook detailing the data download, exploratory analysis, preprocessing, training, tuning, and evaluation.README.md: Model card and description.
How to Load and Use
To make predictions on new data, you must first preprocess and scale the features using the saved scaler.
Loading the Scaler:
1import joblib
2scaler = joblib.load('scaler.joblib')
Scikit-Learn Model (Random Forest):
1import joblib
2
3# Load model
4rf_model = joblib.load('diabetes_random_forest_tuned.joblib')
5
6# Preprocess and predict
7# X_scaled = scaler.transform(X_raw)
8# y_pred = rf_model.predict(X_scaled)
Keras MLP Model:
1from tensorflow import keras
2import joblib
3
4# Load model
5mlp_model = keras.models.load_model('diabetes_mlp_tuned.keras')
6
7# Preprocess and predict
8# X_scaled = scaler.transform(X_raw)
9# y_pred_prob = mlp_model.predict(X_scaled)
Conclusion
The tuned MLP marginally outperformed the tuned Random Forest in accuracy, recall, and F1 score. The neural network achieved the highest recall at 80.59%, which is critical in healthcare settings to limit false negatives. However, the MLP requires significantly more training time and architecture tuning than the ensemble model. Important risk factors identified by both methods include BMI, age, and general health.
Our finding that the tuned Random Forest and tuned MLP perform similarly aligns with broader research on tabular machine learning. Grinsztajn et al. (2022) benchmarked tree models against deep learning across 45 tabular datasets, showing that tree models remain superior or competitive on medium-sized tabular data. Their study found that neural networks struggle with the irregular target functions typical of tabular data and are more sensitive to uninformative features. This explains why hyperparameter tuning did not give the MLP a clear advantage over the Random Forest in this case study.
Limitations & Future Work
A main limitation is that the dataset relies on self-reported survey responses, which can introduce bias. The target column also groups prediabetes and diabetes into one class. Future research could explore gradient boosting models such as XGBoost or incorporate more complex feature engineering.
References
- L. Grinsztajn, E. Oyallon, and G. Varoquaux, "Why do tree-based models still outperform deep learning on typical tabular data?" Advances in Neural Information Processing Systems, vol. 35, pp. 507-520, 2022.
