📉 American Bankruptcy Prediction - Data Science Project
📌 Project Overview
This project applies end-to-end Machine Learning techniques to predict whether an American company will go bankrupt based on financial data.
The dataset contains financial attributes (X1-X18) for thousands of companies. The project follows a structured Data Science pipeline:
Exploratory Data Analysis (EDA)
Feature Engineering & Clustering
Regression Task: Predicting Net Income (X1).
Classification Task: Predicting Financial Stability (High/Low) based on median split.
We started by analyzing the raw data structure, distributions, and correlations.
1. Correlation Matrix
We analyzed the correlation between different financial features. Strong correlations helped us identify redundant features and potential interactions.
Correlation Matrix
2. Initial Target Distribution
The original dataset labels ('alive' vs 'failed') were highly imbalanced.
Target Distribution
🛠️ Part 2: Feature Engineering & Clustering
To improve model performance, we engineered new features:
Interactions: Multiplied strong features (e.g., X16 * X7).
Ratios: Created financial ratios (e.g., X2 / X3).
Clustering: Used K-Means to group companies into 3 clusters and added the Cluster_ID as a new feature for the supervised models.
K-Means Clustering Visualization (PCA)
Clusters
📈 Part 3: Regression Task
Goal: Predict the continuous variable X1 (Net Income) using the engineered features.
Model Comparison
We trained and compared three models: Linear Regression (Ridge), Random Forest Regressor, and Gradient Boosting.
Model
RMSE (Lower is better)
R² Score (Higher is better)
Ridge Regression 🏆
0.012
0.9999
Random Forest
888.67
0.934
Gradient Boosting
906.24
0.931
🏆 Winning Model: Ridge Regression
The Ridge Regression model achieved a near-perfect R2 score, indicating a strong linear relationship in the financial features.
Below is the Feature Importance showing the most influential predictors:
Feature Importance
🧩 Part 4: Classification Task
Goal: We converted the problem into a classification task by splitting the target X1 by its Median.
Class 0: Low Income (Below Median)
Class 1: High Income (Above Median)
Class Balance (After Split)
Using the median split ensured a perfectly balanced dataset for training:
New Class Balance
Model Comparison
We compared Logistic Regression, XGBoost, and Random Forest.
Model
F1-Score
Accuracy
Recall (Class 1)
Random Forest 🏆
0.9976
0.9976
0.997
XGBoost
0.9970
0.9970
0.996
Logistic Regression
0.9931
0.9931
0.992
🏆 Winning Model: Random Forest Classifier
The Random Forest model outperformed others.
Confusion Matrix: The model made very few errors on the test set.
Confusion Matrix
📂 Repository Contents
This repository contains all necessary files to reproduce the results:
intro_to_data_science_2.ipynb: The complete Python notebook with code, analysis, and visualizations.
american_bankruptcy.csv: The dataset used for training and testing.
best_regression_model.pkl: The trained Ridge Regression model.
best_classification_model.pkl: The trained Random Forest Classification model.
*.png: Visualization images generated during the process.
Submitted as part of the Data Science Course Assignment.
