DataRobot PartnersUnify all of your data, ETL and AI tools in our open platform with our Technology Partners, extend your cloud investments with our Cloud Partners, and connect with DataRobot Services Partners to help you build, deploy or migrate to the DataRobot AI Platform.
Delays are costly for airlines and their passengers. A 2010 study commissioned by the Federal Aviation Administration estimated that flight delays cost the airline industry $8 billion a year, much of it due to increased spending on crews, fuel, and maintenance. This does not yet include the cost to the passenger and the cost of lost demand. Tardiness creates other problems including missed connections, lost bags, and short tempers among frustrated travelers. While the airline cannot avoid delays entirely, minimizing their magnitude definitely will save the company a lot of money. A daily departure delay prediction could aid in the decision-making process of the tactical teams as they assess Air Traffic Control (ATC), maintenance, crew connections, ground handling, and scheduling integrity even before a delay happens, or even before the day starts. Currently, if an aircraft is delayed, all subsequent flights connected to that aircraft will be tagged as delayed in the system.
Airline and airport operations are the logical and best first areas for performance gains through the introduction of AI. The technology enables additional data points to be collected in a real-time situation, offering a bird’s eye view which helps the operators get the necessary equipment and people in place. This means that trends can be identified to predict future challenges for OTP and resolve them before they become problems.
There are three basic elements that can be used to create AI measures for OTP:
Proportion of flight departures departing on time (usually within a specified time window)
Proportion of flight arrivals arriving on time
Proportion of flights cancelled
Through AI, OTP can be improved by predicting whether a flight will be delayed using information such as the scheduled departure time and the day of the flight, etc.
How would I measure ROI for my use case?
The cost of delay to airline operations comprises several components, such as costs of passenger delays to the airline and crew and maintenance costs. During the airborne phase, fuel costs and (in the future) emissions charges need to be considered. The total cost is often dominated by the passenger component, which can be split into ‘hard’ costs (such as those due to passenger rebooking, compensation and care), and ‘soft’ costs.
The bottom up formula mentioned below to calculate cost is only for reference and just covers those costs related to customer delay. Note that costs will vary by aircraft type, departure or arrival airport, and (typically) time of the year.
As an example, the estimated cost to an airline due to delay from B737-300 (~3Mn USD) = $13.22 (cost per minute for delay between 1-15 min) * 1163 (Avg number of flights per airline / month) * 12 (months in a year) * 15 (minutes)
About the Data
For illustrative purposes, in this tutorial we are going to be using a dataset provided by the US Department of Transportation that tracks the on-time performance of domestic flights operated by large air carriers. The dataset for this problem is from Kaggle.
The target variablefor this use case is whether or not a flight departure will have been delayed by more than 15 minutes (Binary; True or False, 1 or 0, etc.). This choice in target makes this a binary classification problem. The target variable in our data is not explicitly defined. Hence we did a bit of feature engineering to define our target. A new feature, time_to_take_off, is created to calculate the time taken for a flight to take off (wheels off) from its scheduled departure time. The calculation for this is DEPARTURE_DELAY + TAXI_OUT. And finally the binary target variable (delay_more_then_15) is calculated as 1 if time_to_take_off is more than 15 otherwise 0.
The features below are the raw features pertaining to a flight history. They encompass each flight’s historical departure, arrival time, origination/destination airport, etc, which will help DataRobot find relevant patterns across the dataset.
Other external data sources that may also be relevant include: Bureau of Transportation Statistics (U.S.), weather, holidays, social media etc. There can be additional features which come from the aircraft (e.g., number of seats), airport (e.g., gate number) and flights (i.e., statistics about previous departing flight from the same gate).
Sample Feature List
Year of the Flight Trip
Month of the Flight Trip
Day of the Flight Trip
Day of week of the Flight Trip
Planned Departure Time
WHEEL_OFF – TAXI_OUT
Total Delay on Departure
The time duration elapsed between departure from the origin airport gate and wheels off
The time point that the aircraft’s wheels leave the ground
Planned time amount needed for the flight trip
The time duration between wheels_off and wheels_on time
Distance between two airports
The time point that the aircraft’s wheels touch on the ground
The time duration elapsed between wheels-on and gate arrival at the destination airport
Planned arrival time
Aircraft landed on airport that out of schedule
Flight Cancelled (1 = cancelled)
Reason for Cancellation of flight: A – Airline/Carrier; B – Weather; C – National Air System; D – Security
Delay caused by air system
Delay caused by security
Delay caused by the airline
Delay caused by aircraft
Delay caused by weather
DEPARTURE_DELAY + TAXI_OUT
If time_to_take_off > 15 then 1 otherwise 0 (TARGET)
There are 3 main tables to the dataset.
Airlines: Mapping of airline code and airline names
Airports: Information about an airport such as city, state, lat, long
Flights: Information of a particular flight with details such as departure airport, time, runway time, arrival airport, arrival time etc.
The original raw data consisted of ~6 million flight details for a time period of 1 year (2015). For the final data prep, filtering is done on the original data. For the model development (below), we use only data from April and remove all cancelled flights.
Use only April data.
Remove all cancelled flights.
The 3 data files are joined using the following keys:
DataRobot AutoML automates many parts of the modeling pipeline. Instead of having to hand-code and manually test dozens of models to find the one that best fits your needs, DataRobot automatically runs dozens of models and finds the most accurate one for you, all in a matter of minutes. In addition to training the models, DataRobot automates other steps in the modeling process such as processing and partitioning the dataset.
We will jump straight to interpreting the model results. Take a look here to see how to use DataRobot from start to finish and how to understand the data science methodologies embedded in its automation.
Features with flags for delay reasons are the top features for predicting on time performance. The most impactful feature is delay caused by an aircraft; as shown here, that feature is LATE_AIRCRAFT_DELAY.
If the delay caused by an aircraft is high, then the probability of a delay is higher.
Delays due to air systems decrease by up to 14-18 minutes and then increase drastically afterwards.
Delays due to the airline increase the probability of flight delays.
As the scheduled time for a flight increases, so does the probability for delay.
The probability for delay decreases as distance increases.
The best performing model’s AUC is 85.57 for the validation set and 84.71 for cross validation.
This gives a good understanding that the model is well generalized and not overfitting on the data.
The LiftChart also shows that the predicted values (blue line) are very close to the actual values (orange line) in all deciles, thus giving confidence of the model performance.
(The reason we use CV and not OTV for this scenario is because we are using data for one month in the domestic US market. )
After you are able to find the right model that best learns patterns in your data to predict flight delays, DataRobot makes it easy to deploy the model into your desired decision environment. Decision environments are the ways in which the predictions generated by the model will be consumed by the appropriate stakeholders in your organization and how they will ultimately make decisions using the predictions that impact your process.
This is a critical piece of implementing the use case as it ensures that predictions are used in the real world to reduce flight delays and reduce airline costs.
Automation | Augmentation | Blend
While there are use cases where decisions can be automated in a data pipeline, a flight delay model is geared to augment the decisions of network control centers. It acts as an intelligent machine that, combined with the expertise of NCC manager, determines flights that are likely to be delayed for the day so that crewing, maintenance, and airports can make optimal decisions quicker and earlier.
DataRobot provides various forms of deployment for its users. For this use case we developed a Tableau dashboard for users to consume flight delay predictions. Tableau provides an extension for DataRobot to build visualizations on the fly. (Check out this step-by-step guide to use the Tableau extension, https://www.datarobot.com/drstarterkit/.)
Decision executors for airline OTP prediction would be a network control center team to identify which flights could delay, on a daily basis.
Decision managers are the executive stakeholders who will monitor and manage the program to analyze the airline’s on-time performance.
Decision authors are the technical stakeholders who will set up the decision flow in place.
Ops analytics team
There are several decisions that could be taken by teams:
Proactive scheduling of maintenance and staff based on flight schedules
Manage the flight paths and routing
Smart gate allocation
Manage customers’ expectations and improve customer experience
With the OTP predictions, the team can then take preventive actions to manage on ground support, improve customer service, and eventually reduce costs.
Decision Operators: Ops analytics team
Prediction Cadence: Batch predictions generated on a daily basis
Model Retraining Cadence: Models retrained once data drift reaches an assigned threshold; otherwise, retrain the models at the beginning of every new operating quarter.
With the recent outbreak of COVID-19, several airlines have cancelled their operations. But as the curve flattens and airlines resume their operations, it is important to monitor and manage the models. Factors such as data drift and model accuracy become important to enable performance monitoring. Additionally, external data such as staff availability and the impact of COVID-19 in a country could also be considered to incorporate with the model.
Experience the DataRobot AI Platform
Less Friction, More AI. Get Started Today With a Free 30-Day Trial.
Airlines harness AI to optimize flight routes, enhance passenger experiences, and streamline operations. From predictive maintenance of aircraft to dynamic pricing strategies, AI empowers airlines to operate more efficiently and safely.