This project applies end-to-end data science techniques to an e-commerce (Amazon-like) dataset in order to:
1.Predict the final order total (TotalAmount)** using regression models.
2.Segment orders/customers into spending tiers** (low / mid / high) using classification.
The work includes data cleaning, exploratory data analysis (EDA), feature engineering, clustering, regression modeling, classification modeling, and model deployment.
The main research question:
Can we accurately predict the final order total (TotalAmount) of an Amazon order using customer, product, pricing, and transaction-level features?**
Exploratory Data Analysis (EDA)
Key steps:
Data cleaning:
Parsed OrderDate as datetime.
Created time-based features such as Week (ISO week number).
Checked for missing values, duplicated rows, and basic consistency.
Descriptive statistics:
UnitPrice:
Mean ≈ 303, Std ≈ 172, Min = 5, Max ≈ 600.
Strong positive correlation with TotalAmount (~0.72).
Quantity:
Mostly between 1 and 5 units.
TotalAmount grows almost linearly with Quantity.
image
image
image
image
image
image
image
-Outlier analysis:
Some very high totals due to combination of high UnitPrice & Quantity.
Kept most outliers, since they represent genuine high-value orders, which are important for modeling.
Seasonality / Black Friday analysis:
Computed Week from OrderDate.
Found that Week 47 (Black Friday) and Week 48 (Cyber Monday) have slightly higher average TotalAmount than other weeks.
Week 47 shows the clearest spike relative to the global weekly average.
Feature Engineering
I added multiple engineered features to improve predictive power:
3.1 Time-based features
Week: ISO week number for each order.
IsHighSeason: 1 if Week in {47, 48} (Black Friday/Cyber Monday), else 0.
3.2 Price-related flags
IsExpensiveItem: 1 if UnitPrice above median unit price, else 0.
Note: To avoid data leakage, we did not include direct formulas that reconstruct TotalAmount (e.g., ValueBeforeTax = UnitPrice * Quantity * (1 − Discount)) in the final models.
3.3 Clustering-based features (Unsupervised Learning)
I applied K-Means clustering on standardized numerical features:
Features used for clustering:
UnitPrice, Quantity, Discount, Tax, ShippingCost
Steps:
Standardized the features with StandardScaler.
Trained KMeans(n_clusters=5, random_state=42).
Added:
ClusterID – the cluster assignment for each order.
DistanceToCentroid – Euclidean distance from each point to its cluster center.
PCA Visualization:
Reduced the clustering space to 2D using PCA (PCA1, PCA2) and plotted the clusters.
Clusters showed clear separation, corresponding roughly to:
High unit price & high quantity orders (high spenders).
Low unit price & low quantity orders (low spenders).
Orders with strong discounts.
Orders with higher taxes/shipping.
These cluster-derived features were then used as additional inputs to the regression and classification models.
Interpretation:
The baseline model already explains ≈91% of the variance in TotalAmount, mainly driven by Quantity, UnitPrice, and Tax. Category has only a minor effect.
4.2 Linear Regression with Feature Engineering & Clusters
This shows a modest improvement over the baseline, primarily due to cluster-based segmentation and simple time-based flags.
4.3 Tree-Based Regression Models
I then trained two advanced models on the engineered dataset:
Random Forest Regressor
Gradient Boosting Regressor
Results (approximate):
Linear Regression (engineered):
R² ≈ 0.92, RMSE ≈ 204
Random Forest Regressor:
R² ≈ 0.9999, RMSE ≈ 6.6
Gradient Boosting Regressor:
R² ≈ 0.996, RMSE ≈ 46.1
Because the underlying relationship between features and TotalAmount is nearly deterministic (a pricing formula applied consistently), tree-based models can almost perfectly reconstruct the function mapping inputs to target.
Winning Regression Model:
Random Forest Regressor — near-perfect performance, robust to non-linearities and feature interactions.
All models used a ColumnTransformer + Pipeline for preprocessing and training.
6.1 Logistic Regression (Multinomial)
Reasonable performance as a baseline.
Main issues:
Frequently confuses the middle class (1) with lower (0) and higher (2) tiers.
Limitation:
Struggles to capture non-linear relationships in the data.
6.2 Random Forest Classifier
Best overall performance:
Very high accuracy and macro F1.
Confusion matrix shows almost perfect classification.
Interpretation:
Effectively captures complex patterns and interactions.
Leverages clustering-based features (ClusterID, DistanceToCentroid) and raw numeric inputs.
6.3 Gradient Boosting Classifier
Strong performance:
High precision and recall across classes.
Slightly more errors than Random Forest, mainly between neighboring classes (1 ↔ 2).
Still a very competitive model, but not the top performer.
Critical mistakes considered:
In a business context, false negatives on the high-spending class (Class 2 → Class 0/1) are more costly than false positives, because they represent missed opportunities for targeting valuable customers.
Winning Classification Model:
Random Forest Classifier — highest accuracy, macro F1, and most balanced confusion matrix.
7. Exported Models
The following models were exported as .pkl files for deployment:
winning_model_random_forest.pkl
→ Random Forest regression model (pipeline including preprocessing).
rf_classifier_model.pkl
→ Random Forest classification model (pipeline including preprocessing).
