Introducing MLOps Portable Prediction Servers
Most organizations invested in machine learning are experiencing what we at DataRobot call the AI production gap, which refers to the transition of models from the data science teams who develop them to the IT and DevOps teams who are responsible for deploying and monitoring them in production.
Production AI Challenges
Enterprises increasingly rely on AI and machine learning to build a competitive advantage, but the challenges involved in developing these assets do not end when the model is complete. In fact, that’s just the start of a new challenge—the journey to production—a journey that according to market surveys, fails at an alarming rate.
Of the models that are deployed, few are equipped with monitoring capabilities sufficient to draw attention to health and performance over their lifetime. This is incredibly important because models that are accurate today are rarely accurate tomorrow. They require constant monitoring, retraining, or replacement.
IT and DevOps teams are in a tough position. They need to support many teams across their organization, some of which use different tools, languages, libraries, and environments. These teams seldom take production deployment into consideration, and certainly not from an enterprise perspective. The result is a proliferation of IT tools or large engineering efforts to build out custom frameworks to manage models in production.
To be successful with production AI across the enterprise, IT and DevOps teams need a standard and centralized approach for deploying and managing production models. Ideally, this approach uses familiar DevOps tools to containerize models and artifacts and includes out-of-the-box health and performance monitoring capabilities.
New in DataRobot MLOps Release 6.3: Portable Prediction Servers
In MLOps Release 6.3, DataRobot introduces Portable Prediction Servers. An MLOps Portable Prediction Server is an easy-to-use Docker container that can host one or more production models. The models are accessible through a production-grade REST interface for predictions and prediction explanations.
Your DevOps and IT departments are already familiar with Docker. They can now easily integrate production models into pipelines and applications across the most popular cloud platforms, including Amazon Web Services, Microsoft Azure, and Google Cloud Services, and on-premise platforms without time-consuming software engineering efforts. And container orchestration tools, such as Kubernetes, provide a straightforward path to scale out as model demand requires.
If you’re using DataRobot MLOps 6.3, you can now define external prediction environments for a centralized view of where models are running. It doesn’t matter if the models were built using DataRobot or developed externally, outside of DataRobot, by your data science team. Your MLOps engineers and administrators can use this new functionality to quickly identify the locations of all production models.
You can use DataRobot Portable Prediction Servers or an internal DataRobot Prediction Server to deploy models to these external prediction environments. Managing all deployments from DataRobot MLOps ensures adherence to your organization’s governance and approval policies through a tightly gated workflow.
The MLOps product guides you through the process of deploying a model with a Portable Prediction Server. To start the server that contains a model, you simply download the package and issue a single docker run command.
You use REST APIs to interact with the server. You can make real-time and batch predictions using CSV or JSON files. You can even request prediction explanations to understand the reasons behind a model’s outcomes.
Monitoring Portable Prediction Servers with MLOps Agents
The flexibility to deploy models in a variety of environments can present challenges in the monitoring of their health and performance. That’s why the Portable Prediction Servers use the agent concept familiar in DevOps to track models in your preferred infrastructure while monitoring them centrally. Just imagine being able to monitor models created in any location and deployed on any infrastructure with minimal time or performance overhead.
The combination of the Portable Prediction Server and the Monitoring Agent offers your MLOps administrators deployment flexibility and a single pane of glass from which to observe model performance. Visit here to learn more about MLOps Agents.
Learn More About the Portable Prediction Server
This new model deployment option is included with your DataRobot MLOps license. It is generally available in Release 6.3.
For a demonstration, see this video on the DataRobot Community page. For information about how to use the Portable Prediction Server, see instructions for your cloud platform:
- Azure Kubernetes Service (AKS)
- Amazon Elastic Kubernetes Service (Amazon EKS)
- Google Kubernetes Engine (GKE)
Chris Cozzi is a product manager on DataRobot MLOps. Before he joined DataRobot, Chris worked on product management in enterprise analytics and healthcare marketing.
We will contact you shortly
We’re almost there! These are the next steps:
- Look out for an email from DataRobot with a subject line: Your Subscription Confirmation.
- Click the confirmation link to approve your consent.
- Done! You have now opted to receive communications about DataRobot’s products and services.
Didn’t receive the email? Please make sure to check your spam or junk folders.
Accelerate Your AI Journey with the DataRobot Partner EcosystemMarch 28, 2023· 3 min read
How MLOps Enables Machine Learning Production at ScaleMarch 23, 2023· 4 min read
How the DataRobot AI Platform Is Delivering Value-Driven AIMarch 16, 2023· 4 min read
DataRobot AI Platform announces new capabilities to streamline ML lifecycle, promote collaboration, scale model performance, and ensure compliance and governance.
DataRobot launched a new AI platform to help businesses achieve measurable value from AI. We are offering rapid experimentation and reducing enterprise risk.
The new DataRobot Notebooks offering plays a crucial role in providing a collaborative environment for AI builders to use a code-first approach to accelerate one of the most time-consuming parts of the machine learning lifecycle.