Autoscaling Deployment with MLOps
Investing in AI/ML is no longer an option but is critical for organizations to remain competitive. However, machine learning utilization is often unpredictable, which makes scaling sometimes a huge challenge. Many engineering teams don’t pay the necessary attention to it. The main reason is that they don’t have a clear plan to scale things up from the beginning. From our experience working with organizations across different industries, we learned about the main challenges related to this process. We combined the resources and expertise of DataRobot MLOps and Algorithmia to achieve the best results.
In this technical post, we’ll focus on some changes we’ve made to allow custom models to operate as an algorithm on Algorithmia, while still feeding predictions, input, and other metrics back to the DataRobot MLOps platform—a true best of both worlds.
Data Science Expertise Meets Scalability
DataRobot AI Cloud platform has an absolutely fantastic training pipeline with AutoML and also has a rock-solid inference system. However, there are some reasons why your workflow might not make sense as a typical DataRobot deployment:
- Deep Learning Acceleration (GPU enablements)
- Custom logic, utilizing existing algorithms, acting as part of a larger workflow
- Already have your own training pipeline, have automatic retraining pipelines in development
- Want to save costs by being able to scale to zero workers; don’t need always-on deployments; want to be able to scale to 100 in the event your project becomes popular
But have no fear! Since the integration of DataRobot and Algorithmia, we now have the best of both worlds, and this workflow enables that.
Autoscaling Deployments with Trust
Our team built a workflow that enables the ability to deploy a custom model (or algorithm) to the Algorithmia inference environment, while automatically generating a DataRobot deployment that is connected to the Algorithmia Inference Model (algorithm).
When you call the Algorithmia API endpoint to make a prediction, you’re automatically feeding metrics back to your DataRobot MLOps deployment—allowing you to check the status of your endpoint and monitor for model drift and other failure modes.
The Demo: Autoscaling with MLOps
Here we will demonstrate an end-to-end unattended workflow that:
- trains a new model on the Fashion MNIST Dataset
- uploads it to an Algorithmia Data Collection
- creates a new algorithm on Algorithmia
- creates DataRobot deployment
- Links everything together via the MLOps Agent. The only thing you need to do is to call the API endpoint with the curl command returned at the end of the notebook, and you’re ready to use this in production.
If you want to skip ahead and go straight to the code, a link to the Jupyter notebook can be found here.
Operationalize ML Faster with MLOps Automation
As we know, one of the biggest challenges that data scientists face after exploring and experimenting with a new model is taking it from a workbench and incorporating it into a production environment. This usually requires building automation for both model retraining, drift direction, and compliance/reporting requirements. Many of these can automatically be generated by the DataRobot UI. However, most of the time it can be easier to build your own dashboards specific to your use case.
In this demo, we are completely unattended. There are no web UIs or buttons you need to click. You interact with everything via our Python clients wrapping our API endpoints. If you want to take this demo and rip out a few parts to incorporate into your production code, you’re free to do so.
See Autoscaling with MLOps in Action
Here I will demontstrate an end-to-end unattended workflow, all you need is a machine with a Jupyter notebook server running, an Algorithmia API Key, and a DataRobot API key.
How to Get an Algorithmia API Key
If you’re already an Algorithmia / Algorithmia Enterprise customer, please select your personal workspace and then select API Keys.
You’ll need to select an API key that is management capable. Admin keys are not required for this demo. This may be a different path depending on your Algorithmia Cluster environment, if you’re having difficulties reach out to the DataRobot and Algorithmia team.
If you aren’t an existing Algorithmia / Algorithmia Enterprise customer and would like to see the Algorithmia offering, please reach out to your DataRobot account manager.
How to Get your DataRobot API Token
To get your DataRobot API token, you first must ensure that MLOps is enabled for your account.
After, under your profile, select developer tools to open the token window.
Create new key. You should typically create a new API Key for every production model you have so that you can isolate them and disable them if they ever leak.
This process may be different depending on your version of DataRobot. If you have any questions, please reach out to your account manager.
Incorporating Your Tokens into the Notebook
You’ve got your tokens, now lets add them to the notebook.
datarobot_api_token = "DATAROBOT_API_TOKEN" algorithmia_api_key = "ALGORITHMIA_API_TOKEN" algorithm_name = "fashion_mnist_mlops" algorithmia_endpoint = "https://api.algorithmia.com" datarobot_endpoint = "https://app.datarobot.com"
Insert your API Tokens, along with your custom endpoints for DataRobot and Algorithmia. If your Algorithmia url is https://www.enthalpy.click, add https://api.enthalpy.click here to ensure we can connect. Do the same for your DataRobot endpoint.
If you are not sure or you are using the serverless versions of both offerings, leave these as default and we can move on.
Running the Notebook
Now that your credentials have been added, you can train a model, create a DR deployment; create an algorithm on Algorithmia, and finally connect them together, automatically.
Fashion MNIST Automatic Deployment Notebook
Maximize Efficiency and Scale AI Operations
At DataRobot, we’re always trying to build the best development experience and best productionization platform anywhere. This integration was a big step toward helping organizations to maximize efficiency and scale their AI operations; if you want to know more about DataRobot MLOps or have any suggestions on feature enhancements that can improve your workflow, reach out to us.
Principal ML Engineer, DataRobot
James Sutton is part of the machine learning team working in the Office of the CTO at DataRobot. Previously, James was on the ML Engineering team at Algorithmia and was involved in building GPU support, the Python client, and a few other things. His big focus is building features and enhancing functionality that directly improves DataRobot’s product offerings and provides direct value to customers and developers.
We will contact you shortly
We’re almost there! These are the next steps:
- Look out for an email from DataRobot with a subject line: Your Subscription Confirmation.
- Click the confirmation link to approve your consent.
- Done! You have now opted to receive communications about DataRobot’s products and services.
Didn’t receive the email? Please make sure to check your spam or junk folders.
New DataRobot and Snowflake Integrations: Seamless Data Prep, Model Deployment, and MonitoringMarch 16, 2023· 5 min read
A New Era of Value-Driven AIMarch 16, 2023· 2 min read
How the DataRobot AI Platform Is Delivering Value-Driven AIMarch 16, 2023· 4 min read
DataRobot AI Platform announces new capabilities to streamline ML lifecycle, promote collaboration, scale model performance, and ensure compliance and governance.
DataRobot launched a new AI platform to help businesses achieve measurable value from AI. We are offering rapid experimentation and reducing enterprise risk.
The new DataRobot Notebooks offering plays a crucial role in providing a collaborative environment for AI builders to use a code-first approach to accelerate one of the most time-consuming parts of the machine learning lifecycle.