The 4 Ways Intelligent Automation Helps Achieve DataOps Background

The 4 Ways Intelligent Automation Helps Achieve DataOps

August 13, 2019
by
3 min

To accommodate the growing popularity of self-service data preparation among our customers, we focused our most recent 2019 release on fusing the generation of data prep flows and the operationalization of them. The larger the team of data analysts, data engineers, and data practitioners, the more data preparation flows get created and reused across the organization. Instead of making operationalization of these flows a separate task, we wanted the system to handle that intelligently.

Say hello to Intelligent Automation.

In speaking with customers who are using the new release, it is fulfilling to hear them express the benefits of our recent innovation. I repeatedly hear customers articulate one of four particular aspects of Paxata Intelligent Automation.

1) Automated Runtime and Dependency Management

With a single click, Paxata automatically detects and sequences all related data prep projects into a single data flow. For example, take one data preparation project that is dependent on a dozen other projects, all curated from the teamwork and collaboration of several data practitioners.

Then, instead of manually linking the various data flows together—typically a task done by IT or a data engineer—Paxata Intelligent Automation will recursively map all the dependencies. That way, when a single project is set to run, the rest execute accordingly.

“Paxata’s new Intelligent Automation greatly improves our organization’s ability to accelerate the continuous delivery of our customer master and data quality projects. We can now automate any curated data preparation workflow with a single click, and Paxata intelligently discovers and links all the dependent projects together, which helps us save time and simplify our data operations.” — Byron Hernandez, Senior Data Analyst, Cox Automotive.

2) Visual Impact Analysis

The new Automated Project Flow graph is useful to see how incoming data is impacting downstream processes. This view gives full visibility to all upstream sources and downstream outputs for any given data preparation project, helping the user visualize the impact before making any changes.

Data project flowchart starting with dataset and ending with final project


3)
Tracking the History and Analyzing Data Snapshots

For an operationalized process, the ability to record (what ran when to generate what results), audit, and even troubleshoot is very important. Paxata Intelligent Automation has this functionality built in, so business analysts or data engineers can set it and forget it. For every automation job that runs, Paxata automatically snapshots the version of Dataset and Projects that were executed. If you need to go back to the last quarter and analyze what data and transformations resulted in the answers, you have the ability to do so quickly and easily.

Paxata also automatically versions the results so you can whip up a periodic trend report.

4) Integration with Other Data Processes

In enterprises where an orchestration engine already exists, Paxata’s Intelligent Automation can be invoked within that context using REST APIs. This helps processing of data preparation flows within a larger data architecture. The same automated recursive map ensures that all the dependencies are sequenced appropriately for every run. 

Many of these points reinforce the core principles of DataOps, ensuing the orchestration of data to production use cases and agile deployment of new changes. To scale data preparation at the enterprise, one can either hire and assemble a team of DataOps professionals or take advantage of intelligence algorithms that facilitate the same process with no or little DataOps oversight. The choice, I think you’ll agree, is an easy one.

Free Trial
DataRobot Paxata

Interactively explore, combine, and shape diverse datasets into data ready for machine learning and AI applications

About the author
DataRobot

Enabling the AI-Driven Enterprise

The leader in enterprise AI, delivering trusted AI technology and enablement services to global enterprises competing in today’s Intelligence Revolution. Its enterprise AI platform maximizes business value by delivering AI at scale and continuously optimizing performance over time.

Meet DataRobot
Share this post
Subscribe to our Blog

Thanks! Check your inbox to confirm your subscription.

Thank You!

We’re almost there! These are the next steps:

  • Look out for an email from DataRobot with a subject line: Your Subscription Confirmation.
  • Click the confirmation link to approve your consent.
  • Done! You have now opted to receive communications about DataRobot’s products and services.

Didn’t receive the email? Please make sure to check your spam or junk folders.

Close

Newsletter Subscription
Subscribe to our Blog