The Feedback Loop: How Humility in AI Impacts Decision Systems
Humble AI is a new feature in DataRobot that protects the quality of your predictions in situations where the model may be less confident. With Humble AI, users create rules for models in deployment for predictions made in real-time. These rules identify conditions that indicate quantifiably that a prediction may be unsure, and then can trigger actions like defaulting to a “safe” prediction, overriding outlier values, or not making a prediction at all.
While Humble AI is a new concept, it is clear that it holds tremendous value for decision systems, in particular, real-time decision systems, where each decision is an urgent and high-stakes one. But before we look at the value to business decision-making, let’s briefly go over the three most common frameworks around AI and human intelligence working together as parts of a single integrated decision system.
When it comes to choosing a level of automation for any given system, you have to consider the resulting trade-off. On one end of the spectrum, you have unbridled automation that approaches maximum efficiency, but it comes at the at the cost of losing the guardrails provided by humans looped into the process. If your risk appetite is low or you’re working in a highly regulated environment, human-out-of-the-loop systems may pose a serious business risk.
However, in some scenarios, like real-time ad bidding, it’s impossible to have a human in the loop. These systems run high volumes of predictions, with decisions made in a fraction of a second. And this is all done based on a head-spinning amount of anonymized user information and immediate behavioral patterns of each particular user. A lot of such digital advertising systems can work effectively only when they don’t have a human in the loop. When the decision window is immensely small, and the decisions are extraordinarily complex, how do you mitigate risks?
This is where the concepts of human-over-the-loop and humility in AI come into play. Human-over-the-loop systems can balance automation and human participation by enabling the system to perform in the fully automated mode with human intervention when needed.
Currently available model monitoring capabilities, like those available with DataRobot MLOps, support a long-term oversight framework by continuously collecting scoring data, predictions, and actual outcomes to compare trained and deployed models’ statistical properties. While this can help identify the critical moment when retraining the model is necessary, humility in AI dictates that human intervention should also be an option on the level of an instantaneous individual prediction. There are a few primary questions in this scenario.
How do you recognize when exactly humans should be involved? This depends on the confidence level around the model’s prediction. The confidence level could be gauged by monitoring:
- Uncertainty around predictions: Predicted values are outside of the range of expected values
- Outlying outputs: There are numeric features in the scoring data dissimilar to what the training model ingested
- Low observation regions: A categorical feature value is the one the user has specified as unexpected or inappropriate
But that’s just one piece of the puzzle. The next key step is how the system should respond to a specific lower confidence trigger. There are certain actions a system could perform in these cases:
- No operation: Monitor how often the rule’s condition is met without affecting predictions at all. You can use it to check if the condition was selected correctly and is not triggering too often or too rarely.
- Overriding prediction: Dictate the prediction regardless of the model’s output to a specified value. It can enable you to force business rules on your deployment since you control the predicted value. It may be that you specify a “safe” value for prediction, ensuring that when the model is unsure of the identified condition, the action taken introduces the least risk.
- Returning an error: Discard the prediction completely.
With DataRobot MLOPs, these capabilities are baked into the Humble AI feature. But you can read our latest ebook, Humility in AI, to explore other facets and ways to support humility in an AI system, regardless of your platform.
Applied Data Scientist, DataRobot
Sarah is an Applied Data Scientist on the Trusted AI team at DataRobot. Her work focuses on the ethical use of AI, particularly the creation of tools, frameworks, and approaches to support responsible but pragmatic AI stewardship, and the advancement of thought leadership and education on AI ethics.
We will contact you shortly
We’re almost there! These are the next steps:
- Look out for an email from DataRobot with a subject line: Your Subscription Confirmation.
- Click the confirmation link to approve your consent.
- Done! You have now opted to receive communications about DataRobot’s products and services.
Didn’t receive the email? Please make sure to check your spam or junk folders.
Accelerate Your AI Journey with the DataRobot Partner EcosystemMarch 28, 2023· 3 min read
How MLOps Enables Machine Learning Production at ScaleMarch 23, 2023· 4 min read
How the DataRobot AI Platform Is Delivering Value-Driven AIMarch 16, 2023· 4 min read
At DataRobot, we believe humility is a key step in building more trustworthy AI models. What do we mean by humility in AI? We mean that even a model built according to data science best practices may still have particular areas of weakness or vulnerabilities when confronted with new data to score. For example, that data may have quality issues,…
Cutting-edge algorithms and new research will continue to drive the advancement of machine learning. However, there’s a more straightforward way of resolving many challenges in machine learning today, especially when it comes to ethics and better alignment with human values. And it is not surprisingly, not focused on the technology, but rather on the people using it. For advanced AI and machine learning systems already in production, the focus is on delivering the intended value of the system, which is no longer a question of leveling up the technology or mathematical techniques behind it. Value-oriented approaches can be supported holistically by both the human expertise involved in creating AI and by the technology itself.
Do you trust the predictions your models are generating? Let’s think about this for a moment. Machine learning models learn from historical data and use the patterns they discover to make predictions about new data. If this were a perfect world, the model would have a stable problem to solve, copious amounts of training data, and the new data received…