The project uses the Credit Card Customers dataset from Kaggle (BankChurners.csv), containing 10,127 rows and 23 columns. Each row represents a single bank customer, mixing six categorical features (such as card category, income bracket, education level, marital status, and gender) with sixteen numeric features that capture demographics (age, dependents, months on book) and transaction behavior (total transaction amount and count, revolving balance, change ratios, and inactivity indicators).

The numeric target is Credit_Limit — the credit line in U.S. dollars assigned to each customer, ranging from $1,438 to roughly $34,516. The dataset is clean out of the box (no missing values, no duplicates) but does ship with two pre-computed Naive-Bayes columns and two columns that mathematically derive from the target (Avg_Open_To_Buy and Avg_Utilization_Ratio); all four are dropped before any modeling to prevent target leakage.

Can we predict a customer's Credit Limit from their demographics and banking behaviour, and can we classify customers into credit-tier categories?

The three plots below illustrate the cleaning and descriptive-statistics findings from the EDA above

The boxplot on the left flags 984 points beyond the IQR upper fence (≈ $14,800), and the histogram on the right shows where those points actually sit — they form a clean cluster at the $34,516 cap (the Platinum-tier ceiling) rather than scattering randomly across the high range. This is the textbook signature of a legitimate business cap, not noisy data, so I kept all rows in the dataset. The flagged "outliers" are exactly the premium customers we most want the model to predict accurately.

Target distribution. Credit_Limit is heavily right-skewed: most customers sit below $10K and a long tail stretches up to the $34.5K cap. This skew is why a log_revolving_bal transform is added later in feature engineering.

Correlation heatmap. Among numeric features, the strongest linear correlates with Credit_Limit are Total_Revolving_Bal, the card-tier ordinal, and the income ordinal — exactly what shows up later as the top feature importances in the tree models.

The full EDA is in the notebook; the six research questions and their headline findings are:

Selected visuals from the EDA:

Before any feature engineering, a baseline LinearRegression is fit on the raw numeric features only — no ordinal mapping, no log transforms, no clusters, no scaling. The split is 80/20 with random_state=42. This baseline gives a fair "before" number to measure feature engineering against.

The baseline reaches R² ≈ 0.47 with RMSE ≈ $6,640 and MAE ≈ $4,720 — meaning raw features alone explain about 47% of the variance in Credit_Limit, and the model is off by roughly $4.7K on average. The actual-vs-predicted scatter shows the points hugging the diagonal in the low-to-mid range but fanning out badly above $20K, where Platinum-tier customers live. The residual histogram is roughly centred on zero but visibly right-skewed, confirming the model systematically under-predicts very high limits.

This is the bar to beat — feature engineering, K-Means cluster features, and tree-based models in Part 5 push R² up to ~0.59 and meaningfully reduce both RMSE and the high-end fan-out.

Fifteen new features were engineered. The ratio features include avg_amt_per_trans, contacts_per_month, inactive_ratio, and engagement_score. Tenure is captured by years_on_book. Five binary flags were added (has_dependents, is_attrited, is_inactive_heavy, high_contact_flag, high_relationship). Three log transforms (log_total_trans_amt, log_total_trans_ct, log_revolving_bal) handle right-skewed financial columns. Three ordinal mappings replace ordered categoricals with sensible numeric midpoints: income_ord (income mid-points in $1000s), education_ord (years-of-education proxy), and card_tier_ord (Blue=1 → Platinum=4).

Behaviour features (everything except the target) are standard-scaled and K-Means is fit. The elbow method clearly suggests k = 4:

Projecting the four clusters into the first two principal components shows distinct customer segments — roughly: young low-tenure customers, premium mature customers, loyal mid-income customers, and an inactive churn-risk segment.

The cluster output adds 4 distance-to-centroid features + 4 cluster one-hot dummies to the modelling matrix, giving downstream models a non-linear summary of customer behaviour.

Three regressors are trained on the engineered, scaled feature matrix with an 80/20 split (random_state=42). The retrained Linear Regression is the engineered-features baseline. The Random Forest Regressor uses 200 trees with no depth limit. The Gradient Boosting Regressor uses 300 estimators with learning_rate=0.08 and max_depth=5.

The Linear Regression on engineered features reaches R² ≈ 0.51 with RMSE ≈ $6,340 — a clear lift over the raw-features baseline thanks to ordinal encoding, log transforms, and the cluster-derived columns. The Random Forest pushes R² to ≈ 0.59 with RMSE ≈ $5,800, capturing the non-linear jumps between card tiers that a linear model can't. The Gradient Boosting Regressor lands essentially tied at R² ≈ 0.59 and RMSE ≈ $5,830, with smoother residuals at the high end.

The three-panel comparison (MAE, RMSE, R²) makes the gap between the linear baseline and the two tree ensembles visible at a glance:

The feature-importance panels for Random Forest and Gradient Boosting agree closely: card_tier_ord, income_ord, Total_Revolving_Bal, and log_total_trans_amt dominate, fully consistent with the EDA story.

The actual-vs-predicted plot for the winning regressor shows points hugging the diagonal in the low-to-mid range, with the expected fan-out at the high end where Platinum-tier customers are sparse:

The continuous target is converted into three balanced classes — Low, Medium, High — using quantile binning at the 33rd and 66th percentiles. Critically, the thresholds q33 and q66 are computed on the training set only**, so no information from the test set leaks into the bin definitions. Class balance is roughly 33% per class, so accuracy is a valid primary metric and macro F1 is reported as a secondary check.

Three classifiers are trained on the same engineered feature matrix. Logistic Regression reaches an accuracy of about 0.61, Random Forest gets ~0.62, and Gradient Boosting comes in at ~0.62. All three are within statistical noise of each other. Random Forest has the highest test accuracy in this run and is exported as the classification winner.

The confusion matrices show the typical pattern for ordered classes — most of the residual confusion is between adjacent tiers (Low↔Medium, Medium↔High) rather than across the full range, which is economically tolerable: in business terms, predicting a Medium-tier customer as High costs much less than predicting them as Low.

For the High class, recall matters more than precision — missing a true premium customer (false negative) is more costly than over-promising a non-premium one. The winning Random Forest model achieves the cleanest diagonal on the High class.

The pipeline takes the BankChurners.csv dataset of 10,127 customers and predicts a customer's Credit_Limit end-to-end. After dropping five non-informative or target-leaking columns (the two pre-computed Naive-Bayes outputs, CLIENTNUM, Avg_Open_To_Buy, and Avg_Utilization_Ratio), full EDA across six research questions reveals that card tier, income, and revolving balance are the dominant signals — with a clear gender gap that is flagged as a fairness concern.

Fifteen engineered features (ratios, log transforms, ordinal mappings, binary flags) plus eight cluster-derived features from K-Means with k = 4 give downstream models a richer, non-linear summary of customer behaviour. On the regression task, the Linear Regression on engineered features reaches R² ≈ 0.51, while both the Random Forest and Gradient Boosting Regressors lift R² to ≈ 0.59 with RMSE around $5,800 — Gradient Boosting is exported as the winning regressor. Converting the target into three balanced classes via leakage-free quantile binning produces an accuracy of ≈ 0.62 for both Random Forest and Gradient Boosting, with confusion matrices that confuse only adjacent tiers — an economically tolerable error pattern. Random Forest is exported as the winning classifier.

The biggest lesson was target leakage: had Avg_Open_To_Buy and Avg_Utilization_Ratio been left in, the model would have hit a fake R² near 1.0. Catching that early was the single most important step. The second lesson was that proper feature engineering — ordinal mappings and log transforms — narrows the gap between linear and tree-based models, so most of the predictive power lives in the features rather than the model choice.

Below is a complete list of all files used throughout this project:

Author

Reichman University - Data Science Track

2026