AI Experience Chicago: Highlights from TD Ameritrade, United Airlines, and Symphony Post Acute Network
The DataRobot AI Experience roadshow blew into the Windy City on Thursday, May 17. Hundreds of AI-focused professionals from a wide spectrum of industries gathered to experience first-hand the transformative power of automated machine learning.
Jonathan Dahlberg, a Customer-Facing Data Scientist at DataRobot, kicked off the event by sharing the obstacles companies face on the path to becoming an AI-driven enterprise — and how to overcome them — before ceding the floor to three DataRobot users at companies who have become AI-driven enterprises.
Beaumont Vance (TD Ameritrade), Nathan Patrick Taylor (Symphony Post Acute Network), and Michael Shores (United Airlines)
Speaker 1: Beaumont Vance, Head of Enterprise Analytics at TD Ameritrade
For Beau, DataRobot justifies its place in the analytics toolset by providing value and returning significant ROI immediately.
Beau focused on the importance of getting executive-level buy-in and how critical this is for true innovation to take place.
For Beau, DataRobot justifies its place in the analytics toolset by providing value and returning significant ROI immediately. The machine learning lifecycle with DataRobot solves the challenges of the traditional analytics framework, namely by answering four questions:
- Are you solving an actual tangible business problem with demonstrable ROI?
- What is more important – speed or accuracy?
- Can the solution be implemented into production?
- Is the framework scalable?
With DataRobot, Beau could satisfy the C-suite at TD Ameritrade with answers to these questions, rapidly and with precision. Previously, Beau’s Enterprise Analytics team dismissed projects less than $875k because the return would be too small for the effort. With DataRobot, Beau could have two employees working on that project and within one week: run it through the DataRobot modeling factory at 95% accuracy, productionalize it, get it up to speed, and make it ROI positive. That type of project identification is systematically done in every department throughout the organization. DataRobot opened up a whole universe of small, but very meaningful problems that Beau’s team previously had to dismiss.
Attempts to explain to management what exactly DataRobot does and the value it brought were not successful for Beau; any mention of any data science terminology and they were instantly lost. Finally, Beau said that the platform simply avoided utilizing 12 very expensive people that he would have otherwise had to hire to deliver the productivity brought on by the automated machine learning platform. With Beau’s team liberated from the drudgery of having to build, maintain, and score models, achieving significant ROI from machine learning projects was evident to executives at TD Ameritrade.
As a company of acquisition, TD Ameritrade needed solutions that scale. Beau’s CEO was very clear; a directive was given to Beau to leverage technologies that allow scaling across all departments and new organizations. For Beau’s team, DataRobot was just the technology solution they needed.
Speaker 2: Michael Shores, Senior Manager of Machine Learning at United Airlines
Michael gave examples of just how powerful pervasive AI can be when C-suite executives empower buy-in for machine learning projects across all departments within the organization.
With every department across United, working with Michael’s team, fully empowered and democratized to find analytics projects, it’s clear that AI is a relevant and thriving part of the company.
Michael’s team was tasked with helping find meaningful enterprise analytics projects across the organization– all with the goal of improving the customer experience at United Airlines. Some of those machine learning projects include:
- Predicting how many bags a particular flight will need to gate check. This prevents customers from having to “swim upstream” in the aisle after discovering there isn’t enough overhead cabin space for their suitcases, while also avoiding departure delays.
- Similarly, predicting how much cargo can fit under the plane.
- Recording maintenance logs. The traditional process at United airlines saw technicians filling out maintenance logs with a paper form in order to identify and predict trends on equipment failure. Michael’s team built out a classifier to classify millions of logs per year; the previous manual method could only manage 2% of that volume.
- Travel recommendations, predicting the best flights and destinations for customers.
- Predicting instances when inflight wifi will fail or be less reliable so that customers are notified and plan accordingly.
- Predicting the availability of Regional Premier Upgrades (RPUs). Elite loyalty members at United could apply for RPU coupons and, if one was available for a specific flight, that loyalty member would get an upgrade. If that price wasn’t available, customers went onto a waitlist.
This was often a frustrating customer experience, with premium United customers left wondering if they would receive an upgrade or not. Michael’s team wanted to do a better job predicting the availability of upgrades and making the whole process more transparent, while giving customers better value to stay with United as loyal premium customers and more choices with flights and upgrades.
Michael’s team identified the data — primarily from four buckets: flight demand, flight attributes, customer attributes, and ticket attributes — and once prepped, they simply dropped the data into DataRobot to “let it do its magic.” The models identified which variables from features like the traveler’s premier status, projected no-shows, and flight distance, were most predictive in projecting the number of RPUs available per flight.
The next step is to bring the model to life. United’s plan to do this is first to offer RPUs through an email campaign (“Apply for your RPU today!”) and, if that goes well, putting the application process and model right into the website. The project will ultimately be measured on customer engagement (Are customers clicking on these emails?), customer feedback (Do they like it or are they confused?), and predictive accuracy (constant tweaking of the model to make sure it isn’t slipping, especially toward the end of the calendar year).
With every department across United working with Michael’s fully empowered and democratized team to find analytics projects, it’s clear that AI is a relevant and thriving part of the company. DataRobot is helping drive significant value and productivity on United’s machine learning efforts.
Speaker 3: Nathan Patrick Taylor, Director of Data Science and Analytics at Symphony Post Acute Network
Nathan wrapped up the afternoon by discussing the three challenges of mainstreaming machine learning at any enterprise.
The fact that DataRobot was able to avoid black box syndrome quickly drew support from the IT team at Symphony.
The first challenge is implementation: most people have a hard time understanding math. For Nathan, when speaking with doctors and nurses at Symphony’s facilities, he needed to communicate the basics of how his models worked and why it impacted them without using data science terminology. As an example, “readmit probability” percentages don’t mean much to nurses but when categorized and color-coded, “readmit risk” made more sense.
The second challenge is technology: some of Symphony’s doctors and nurses simply said, “I will not let a robot make decisions for me.” A nurse once challenged Nathan’s model and pointed out that a patient who had been deemed a readmit risk should not be, and that the model was wrong. Nathan encouraged her to dig deeper into the patient’s data and the nurse discovered several risk factors that DataRobot had picked up, including a history of mental illness. The nurse was instantly converted, and as more “haters” became champions, the DataRobot technology caught on and changed people’s minds. The fact that DataRobot was able to avoid black box syndrome quickly drew support from the IT team at Symphony.
The third challenge is onslaught: stakeholders at Symphony now believed that Nathan possessed a magic wand that could come up with all sorts of accurate predictions. As Nathan remembers, DataRobot became sort of a mythical creature, a Kraken who could do whatever it wants. Nathan had to balance priorities among machine learning opportunities and projects, but DataRobot had empowered his small data science team to be much more productive.
With DataRobot, each member of Nathan’s team was now doing the job of typically 6 or 7 people, with a traditional process for machine learning. Automated machine learning had led to a 6x increase in productivity and efficiency, with time saved at every step of the machine learning life cycle. His team had also increased the number of models built by 3x.
As Nathan admitted, perhaps now he understood why his bosses considered DataRobot to be a powerful magic wand.
Attendees at Chicago got the chance to mingle with with and learn from professionals who are already changing their enterprises and industries through the transformative power of AI. Seeing the potency of the DataRobot automated machine learning platform with their very own eyes really does make data science seem like magic (just as Nathan’s CEO described it!).