Cloudera + DataRobot

Automated machine learning powered by the enterprise data hub
Watch the video

Get the most value out of your data by integrating enterprise-grade scalability and predictive modeling.

Cloudera and DataRobot have partnered to deliver an integrated technology solution that runs in Hadoop and utilizes the power of automation and machine learning to extract the most value out of all your data.

The DataRobot integration with Cloudera Enterprise incorporates the best of data science, automated machine learning, and the massive processing power inherent in the Hadoop environment to empower users to get to business insight quickly by discovering and optimizing predictive models in less time.
The DataRobot integration with Cloudera Data Science Workbench enables data scientists and business analysts with a self-service approach to data science:

  • Develop models in Python, R or Scala without having to worry about the details of Hadoop and Spark
  • Utilize automated machine learning to optimize and tune models in order to create the most accurate models
  • Easy and secure access to data and distributed processing
Cloudera logo

Cloudera was founded in 2008 by some of the brightest minds at Silicon Valley’s leading companies, including Google (Christophe Bisciglia), Yahoo! (Amr Awadallah), Oracle (Mike Olson), and Facebook (Jeff Hammerbacher).

Their founders held at their core the belief that open source, open standards, and open markets are best. That belief remains central to our values. Doug Cutting, co-creator of Hadoop, joined the company in 2009 as Chief Architect and remains in that role.

cloudera.com  
management tune equaliser filter
Management
To manage and monitor the application, DataRobot uses Cloudera Custom Service Descriptors. You manage and monitor DataRobot using the exact same tools you use today. DataRobot uses Cloudera Manager to distribute runtime libraries to the Hadoop Data nodes; there are no extra installation and configuration tasks when you add a node.
shield enterprise security datarobot
Security & Encryption
DataRobot uses Sentry for fine-grained role-based authorization, and supports Kerberos and LDAP protocols. With DataRobot, your security protocols are exactly the same as they are for your other applications. Because DataRobot integrates natively and leverages HDFS, it inherits the encryption policies you implement in Cloudera. You do not need to implement additional controls to ensure the security of your data.
gear process create done check
Auditing & Lineage
Cloudera Manager tracks DataRobot lifecycle events and security-related events just like any other process running in your Cloudera cluster. Cloudera also tracks the DataRobot analysis and model files, and Cloudera Navigator provides you with a visualization of data lineage.
yarn
YARN for Workload Management
DataRobot workloads runs in YARN containers, so DataRobot coexists with your other applications. You do not need to partition your cluster to prevent resource conflict – YARN handles that for you.
hadoop
Light Footprint
DataRobot runs on standard Hadoop-spec commodity hardware. It requires no long-running processes on Hadoop Data nodes. Unlike some commercial tools, you will not need to replace or upgrade your Hadoop servers. In addition, DataRobot does not need a proprietary storage layer; it works directly with HDFS. DataRobot users can work directly with HDFS files, and the application uses HDFS to store analysis datasets and models.
apache spark
Spark Scoring
DataRobot uses Apache Spark for high-performance in-memory model scoring. Since DataRobot leverages Cloudera’s scale-out architecture, you can provision scoring for the scoring volume you need at your desired level of throughput.
  • management tune equaliser filter
    Management
    To manage and monitor the application, DataRobot uses Cloudera Custom Service Descriptors. You manage and monitor DataRobot using the exact same tools you use today. DataRobot uses Cloudera Manager to distribute runtime libraries to the Hadoop Data nodes; there are no extra installation and configuration tasks when you add a node.
  • shield enterprise security datarobot
    Security & Encryption
    DataRobot uses Sentry for fine-grained role-based authorization, and supports Kerberos and LDAP protocols. With DataRobot, your security protocols are exactly the same as they are for your other applications. Because DataRobot integrates natively and leverages HDFS, it inherits the encryption policies you implement in Cloudera. You do not need to implement additional controls to ensure the security of your data.
  • gear process create done check
    Auditing & Lineage
    Cloudera Manager tracks DataRobot lifecycle events and security-related events just like any other process running in your Cloudera cluster. Cloudera also tracks the DataRobot analysis and model files, and Cloudera Navigator provides you with a visualization of data lineage.
  • yarn
    YARN for Workload Management
    DataRobot workloads runs in YARN containers, so DataRobot coexists with your other applications. You do not need to partition your cluster to prevent resource conflict – YARN handles that for you.
  • hadoop
    Light Footprint
    DataRobot runs on standard Hadoop-spec commodity hardware. It requires no long-running processes on Hadoop Data nodes. Unlike some commercial tools, you will not need to replace or upgrade your Hadoop servers. In addition, DataRobot does not need a proprietary storage layer; it works directly with HDFS. DataRobot users can work directly with HDFS files, and the application uses HDFS to store analysis datasets and models.
  • apache spark
    Spark Scoring
    DataRobot uses Apache Spark for high-performance in-memory model scoring. Since DataRobot leverages Cloudera’s scale-out architecture, you can provision scoring for the scoring volume you need at your desired level of throughput.

Related resources

See All Resources
Start Your Journey to Become an AI-Enterprise