Take the Route to AI Success with DataOps and MLOps

If you’ve been keeping up with business literature lately, you know that adopting artificial intelligence (AI) strategies can increase company revenue, improve efficiency, and keep customers happy. But even the best models cannot improve performance until they are put into production.

What are companies actually doing today?

Alexander Rode and Timm Grosser, analysts at the Business Application Research Center (BARC), decided to find out by surveying 248 companies from a variety of industries about this question.

Survey Demographics

The companies ranged in size from under 500 (35%) employees to 5000 (34%) or more. In terms of location, 66% were in Europe, 27% were in North America, and 6% were in Asia and the Pacific. 

Survey Questions

The survey asked companies how they used two overlapping types of tools to deploy analytical models:

• Data operations (DataOps) tools, which focus on creating a manageable, maintainable, automated flow of quality-assured data.
• Machine learning operations (MLOps) tools, which handle model retraining, testing, metrics tracking, versioning, and management.

Deploying Models Today

Developing models was clearly easier than deploying them. More than half the survey participants (55%) had not yet put a model into operation, while 37% had not even started building one. 

Reducing Deployment Challenges

Delivering well-managed, high-performing models is a high-stress task. It requires companies to build on prior work, identify dependencies, maintain current applications, and monitor important artifacts.

For 44% of DataOps and MLOps practitioners and 38% of beginners, the biggest issue was restricted access to data silos, a problem which is best addressed by an overarching data management strategy.

Companies using Data/MLOps tools do particularly well in versioning and creating documentation, providing management frameworks, and testing. They also appear to be better at overcoming the barriers that limit cooperation among stakeholders.

Tool adopters are more able to plan their projects, as they are 4.2 times more likely to be able to deploy their models quickly and 3.5 times less likely to be confronted with projects with overwhelming complexity. If deployment goes wrong, DataOps/MLOps can even help solve the problem.

Because most of these companies surveyed have not yet deployed models, only 26% currently use DataOps/MLOps. But 45% are already planning to use these tools in the future. 

Improving Success

When asked how DataOps/MLOps tools had increased their success, 59% of the adopters claimed that they had achieved higher levels of automation. Overall, 97% of the adopters listed a wide range of benefits from using these tools, including more robust applications, better collaboration, and faster time to market.

Realistic Expectations

Of the DataOps/MLOps adopters, 53% said their expectations of ML impact had been met, suggesting that they had realistic expectations about what they could achieve. In fact, 41% described the level of complexity encountered “as expected.”

About 76% of the companies considering the use of DataOps/MLOps tools say they underestimated the difficulty of putting models into production. Presumably, these companies started to explore the products only after being overwhelmed by AI difficulties.

Fast Deployment

Adopters of DataOps/MLOps products benefit from faster time to market, higher productivity, better scalability, and higher levels of automation — all measures of improved efficiency and speed in delivering results. Deployments lasting just weeks or days are common among DataOps/MLOps adopters but unheard of among companies using other approaches.

ML Software Development

For model development, half of the companies use open source tools, almost a third (31%) use commercial tools, and 19% build their own tools. DataOps/MLOps adopters did not differ significantly from other groups in terms of the tool stacks they used to develop their applications.

Beginners felt more confident using open source or self-developed tools, possibly because they did not take operational concerns seriously. However, they often struggled with complex systems and slow deployment, while commercial tool users enjoyed increased efficiency and could develop models in shorter periods of time (days or weeks).

Importance of Enterprise Support

Organizations need to clearly communicate the ROI of ML models because employee resistance and fear may create barriers to progress. If the benefits of advanced analytics are not widely understood, it is difficult to establish new strategies for digital transformation. The adoption of DataOps/MLOps should always be part of a company-wide initiative to increase data literacy within the organization.

Get the Whole Story

Download the free BARC survey Driving Innovation with AI: Getting ahead with DataOps and MLOps.