How to Trust Your AI Vendor

If you’re like me and cannot write a single line of code and have no idea what linear regression is, then keep reading—because this is about your business and how not to end up in a mess with the wrong AI vendor.

Last week, I was reading a success story about an organization that implemented an AI solution and came across some unusual wording: business learning models’. It didn’t say “machine learning” or “predictive analytics”; it was a “business learning model.” I found it remarkable how accurate the wording was, because at the end of the day, that is what it’s all about — how to use AI to yield unprecedented insights that steer enterprise and societal progress.

As organizations start to embed AI into all their operational processes, 42% of senior executives, VPs, and IT leaders are still “very” to “extremely” concerned about AI bias. The top two biggest concerns are “compromised brand reputation” and “loss of customer trust,” as we found and published in our DataRobot Report: The State of AI Bias in 2019.

So, how do you bring on board something as powerful and much needed as AI, yet make sure you can trust the results it’s providing?

To help answer this question, I pulled in my teammate and data scientist Sarah Khatry. Together, we discussed the business and data science needs of someone searching for their AI vendor, and how to translate both of those needs into their core ideas and get to the point. Here, we’ve collected the seven points we found most crucial.

1. Ensure Data Quality or Garbage In, Garbage Out

“Garbage In, Garbage Out” is a motto data scientists know well, and everyone else should too. Models are only as good as the data they’re built on. Make sure your AI solution can help you review your data to identify abnormalities before modeling. For the most accurate predictions, you want to diversify your data as much as possible and ensure its consistency and integrity. It is also crucial to identify and surface a clear warning for features at risk of target leakage. Target leakage occurs when there is information in the training dataset that won’t be available at the moment of the prediction. For example, if you’re predicting loan default at the time of a loan application, you shouldn’t know if a collections agency will be contacted in the future, or if you’re predicting readmission to the hospital, you can’t build a model that has the readmission date already in the dataset.

2. No Free Lunch or Don’t Limit Yourself

The No Free Lunch Theorem is a theory in machine learning that promises that no one algorithm will be equally good for every problem. Research isn’t ever going to uncover an algorithm that’s the automatic best approach for every dataset you could have. Every business problem is different, and so should be the model for its solution. Depending on the problem, it is important to assess the trade-offs between the speed, accuracy, and complexity of different models and find a model that works best for that particular problem. Your AI vendor should help you dynamically generate a variety of modeling approaches, starting from the classic linear models to the most advanced gradient boosted tree classifiers and deep learning approaches, to find out which makes the most sense for you. In a perfect world, it will help you make that decision, and automatically identify contenders for the optimal end-to-end approach.

3. Interpretability or Why? Why? Why?

You shouldn’t have to be a tech genius to understand why your AI made a specific prediction. You deserve an understanding of how your decisions are made and what they are based on. For every prediction made, you want to know which data points had the most impact, what exactly influenced the particular outcome. For example, how much does someone’s income (say 70k) impact whether we decide to grant them the loan; or, did the fact that the patient is 90 lbs. and underweight greatly inform our prediction for his hospital readmission? This explainability brings transparency about the factors behind a decision.

4. Compliance Documentation or Loooooots of Paperwork

This is a nightmare for your data scientists/analysts. The need for documentation and compliance throughout every single step of the modeling process is paramount, starting from the data ingestion up to when the prediction is made, especially if you’re a financial institution or a highly-regulated organization. Search for the AI vendor that can help you generate detailed, well-formatted compliance documentation. That is, find one that helps you answer the fundamental questions of what, how, and why of your model from the high-level executive summary down to the technical details.

5. Deployment Flexibility or And Now What?

All of the above steps have very little value if we don’t take action on these predictions. We want information in the hands of someone who will use it to generate actual value. But how do you get it there? Depending on your use case, you may have different parameters for how you want to deploy your model. For instance, if you’re predicting customer churn, you might want to run it every two weeks when you’ll score all your customers at once; this is a batch prediction. On the contrary, credit lead scoring requires instantaneous predictions every time someone interacts with your tool. Based on your business needs, your AI vendor should support you in exploring a variety of deployment options based on your security concerns, how frequently and fast you need to make predictions, and how much data you need to score.

6. Monitoring or AI Watchdog

A model is based upon a snapshot in time, but things change — gradually, as markets drift over time, or suddenly, as in the case of a major event or an operational change in how you do your business. When that happens, your model likely will not perform as well as it did before. Trust in your AI systems doesn’t stop at model creation and must extend through models running in production. Predictive models must be monitored and updated regularly. Otherwise, they will eventually fail, and losing accuracy and relevance leads to catastrophic outcomes, including lost revenues and the trust of business executives, investors, and customers. Find an AI vendor that can help proactively manage productionalized models and alert you when something changes.

7. The Human Side or Personal AI Support

Obviously, no one will know your business better than you do, and industry knowledge is crucial. But in the same way, you are unlikely to know an AI better than the data scientists who created it. So, why not combine the efforts? Implementing AI is a long-term commitment and you want to dive in with a team who’ll be there for you. Literally. Guide you, teach you, support you, and learn from you. People buy from people, so talk to the vendor’s team, listen to what they have to say, ask what’s their vision specifically for your business and for theirs, discuss other success stories and how the goal was reached, watch how they act in critical situations. But most importantly, don’t waste those minutes of small talk at the beginning of the meetings. Pay attention to details of what they say about their lives, their coworkers, their families because this is when you can truly understand how ethical, reliable and trustworthy are the people behind your AI. You’re in this together.

According to trust researcher Rachel Botsman, Trust is the social glue that enables humankind to progress through interaction with each other and our environment, including technology. And while it takes seconds for us to listen to our guts and understand how we feel about a human being, in the case of AI, you’ll have to do your homework. There’s clearly a lot of buzz around artificial intelligence and machine learning nowadays, but what you want to do is to cut through all that jargon and really understand the core value of the AI solution you’re looking at by checking all those comprehensive elements listed above, and then start your AI journey.

About the author

Iryna Shevchenko

Sales Development, DataRobot

Iryna connects the DataRobot Marketing and Sales teams, making sure that both are addressing the needs of different industries and markets. She has a substantial background in tech sales, is passionate about AI Trust and its impact on customer success, and is a huge world traveler.

Meet Iryna Shevchenko

---

Sarah Khatry

Applied Data Scientist, DataRobot

Sarah is an Applied Data Scientist on the Trusted AI team at DataRobot. Her work focuses on the ethical use of AI, particularly the creation of tools, frameworks, and approaches to support responsible but pragmatic AI stewardship, and the advancement of thought leadership and education on AI ethics.

Meet Sarah Khatry