This project aims to analyze and predict flight prices using multiple machine learning approaches.
The assignment includes:
Exploratory Data Analysis (EDA)
Regression modeling (baseline → advanced)
Feature Engineering
Clustering (K-Means + PCA visualization)
Transforming regression into classification
Training & evaluating classification models
Project Goal
The goal is to understand what drives flight prices, build effective predictive models, and communicate insights clearly.
Part 1 - Dataset and Overview
The dataset contains detailed information about commercial flights, including airline, departure/arrival times, number of stops, duration, and price.
After cleaning (removing duplicates, dropping an unnamed column, and removing outliers in two columns using IQR), the final dataset shape is:
Conclusion: Ticket prices are highly right-skewed: most tickets are relatively cheap, with a small number of very expensive flights forming a long right tail of the distribution.
Price by Stops
image
Conclusion: Flights with one or more stops tend to be much more expensive than non-stop flights, and 1-stop itineraries are the priciest on average – showing that the number of stops is strongly related to ticket price.
Price by Duration
image
Conclusion: Longer flights generally have higher ticket prices, but the scatter is large, so duration alone cannot explain price; other features such as class, airline and number of stops also play an important role.
Research Questions
1. How does the average ticket price vary by month?
image
Conclusion: The average ticket price drops noticeably from February to March. This suggests a seasonal pattern where flights in March tend to be cheaper, possibly due to lower demand or post-holiday travel trends.
2. How do ticket prices vary across travel class and number of stops?
image
Conclusion: Business-class tickets are consistently more expensive than Economy across all stop categories. Prices also increase sharply with additional stops, especially for Business class. This indicates a strong interaction between travel class and number of stops in determining ticket price.
3. What is the relationship between flight duration and ticket price for non-stop flights?
image
Conclusion: There is a positive trend—longer non-stop flights tend to have higher ticket prices. However, the large scatter shows that duration alone cannot fully explain price variation, meaning additional factors like class, route, and airline also significantly influence ticket cost.
4. How is the price distribution different between Economy and Business class?
image
Conclusion: Economy tickets are heavily concentrated in the low-price range, while Business-class tickets appear at much higher prices with a wider spread. This clear separation shows that travel class is one of the strongest predictors of ticket price.
These insights were later connected to model performance and feature importance.
Part 3 - Regression Models
Baseline Model — Linear Regression
Using cleaned + encoded baseline features (11 features).
Results:
Train set:
MAE ≈ 4,493
RMSE ≈ 6,880
R² ≈ 0.91
Test set:
MAE ≈ 4,470
RMSE ≈ 6,880
R² ≈ 0.91
The baseline model performs strongly, showing that the engineered features are meaningful.
Baseline Regression Plot
image
Conclusion: The baseline linear regression captures the general upward trend between true and predicted prices, but many points deviate from the ideal diagonal line. This indicates that the model struggles with non-linear patterns and underfits in several price ranges, especially for very cheap and very expensive tickets.
Part 4 - Feature Engineering
To improve model performance, I added several engineered features on top of the baseline numeric features.
Added Features
duration_hours – flight duration converted from minutes to hours
is_long_flight – indicator for long flights (duration ≥ 600 minutes)
is_non_stop – indicator for non-stop flights (stops == 0)
is_night_arrival – indicator for night arrivals
K-Means Features
Using K-Means (k = 4), I created:
cluster_id – assigned cluster label
cluster_distance – distance to cluster centroid
PCA 2D visualization of clusters
image
These features help the models capture non-linear patterns and interactions in the data.
Part 5 - Improved Regression Models + Part 6 - Winning Model
Models:
Random Forest Regressor
Gradient Boosting Regressor
Results summary:
Model
MAE
RMSE
R²
Random Forest
best
best
highest
Gradient Boosting
slightly worse
slightly worse
lower
Random Forest was the winning regression model.
Feature Importance Plot
image
Conclusion: The Random Forest model identifies class as by far the most important feature for predicting ticket prices. Other features such as duration_in_min, route_index, and airline_index contribute much less but still show meaningful influence. This confirms that travel class is the strongest driver of ticket pricing in the dataset.
Part 7 - Regression-to-Classification
In addition to predicting the exact ticket price, I also reframed the problem as a 3-class classification task.
The goal here is to classify each flight into a price level (cheap / medium / expensive) based on the numeric target price.
Creating the price_class target
To create the classification labels I transformed the continuous price variable into three classes using quantile binning on the training set only:
Class 0 – cheap tickets – bottom 33% of prices
Class 1 – medium tickets – middle 33%
Class 2 – expensive tickets – top 33%
This strategy has two advantages:
It matches an intuitive business interpretation of cheap / medium / expensive tickets.
Using quantiles keeps the classes relatively balanced, so the classifier does not focus on one dominant class.
The resulting discrete labels are stored in a new target column called price_class, which is used throughout the classification part.
Class balance
After creating the labels, I checked the class distribution in both the train and test sets.
Each of the three classes (0 – cheap, 1 – medium, 2 – expensive) contains roughly one third of the samples in both splits (≈ 32–34% per class).
This means the classes are reasonably balanced and:
Overall Accuracy is a meaningful metric.
I also track Macro F1 score to make sure the model performs well across all classes, not just one.
image
Part 8 - Classification Models
Using the engineered features from the regression part, I trained several classification models to predict the new target price_class (0 = cheap, 1 = medium, 2 = expensive).
Business perspective: Precision vs Recall
From a user perspective, the most important group is the cheap tickets (class 0):
I prefer the model to catch as many truly cheap flights as possible, even if some non-cheap flights are mistakenly labeled as cheap.
Therefore, for the cheap class, Recall is more important than Precision.
In terms of error types:
A False Negative (a truly cheap flight classified as not cheap) is more critical than a False Positive (a not-cheap flight classified as cheap), because the user will never see a good deal that actually exists.
Trained classification models
I trained three different classifiers on the same feature matrix (X_train_fe, X_test_fe) and the new target price_class:
Logistic Regression (multiclass baseline) – a simple linear baseline for multi-class classification.
Random Forest Classifier – an ensemble of decision trees that can capture non-linear relationships.
Gradient Boosting Classifier – another tree-based ensemble that focuses on correcting previous errors and often gives strong performance.
All models were evaluated on the same train/test split using:
classification_report (precision, recall, F1, support) for train and test sets
Confusion matrix on the test set
Overall Accuracy and Macro F1 on the test set
Evaluation of classification models
On the test set, the models achieved the following results:
Model
Accuracy (test)
Macro F1 (test)
Logistic Regression
0.794
0.795
Gradient Boosting
0.869
0.869
Random Forest
0.953
0.952
Because the classes are balanced, these metrics can be compared fairly across models.
Types of mistakes (based on confusion matrices)
Logistic Regression
image
Makes many mistakes between class 0 (cheap) and class 1 (medium).
Also confuses class 2 (expensive) with class 1, so expensive tickets are often predicted as medium.
The confusion matrix is more “spread out”, which matches the lower accuracy/F1.
Gradient Boosting
image
Performs better than Logistic Regression but worse than Random Forest.
Still shows noticeable confusion between class 1 and 2.
The diagonal in the confusion matrix is strong, but off-diagonal cells are larger than in Random Forest.
Random Forest
image
The confusion matrix is strongly concentrated on the diagonal.
Only a small number of tickets are misclassified, mostly between neighbouring classes (0 vs 1, 1 vs 2).
Very few cheap tickets are predicted as expensive (0 → 2) or the opposite (2 → 0).
This model captures the non-linear relationships in the data much better than Logistic Regression.
Winning classifier
The Random Forest Classifier is the winning model for the classification task because:
It has the highest Accuracy and Macro F1 on the test set.
It makes relatively few “severe” mistakes between cheap and expensive tickets.
