DataRobot has leveraged the power of Hadoop to develop a ground-breaking predictive analytics platform and is seeking experienced Hadoop Automation Engineers to develop and support integrations with customer environments and Hadoop technologies. As a Hadoop Automation Engineer at DataRobot you will enable our development teams to deliver advanced integrations with Hadoop technologies to any Hadoop cluster in the world. Your deep knowledge of Hadoop internals and programming will be key in integrating and automatically testing DataRobot software in challenging Hadoop environments.
Duties and Responsibilities
- Add features to and extend DataRobot’s Yarn application for distributing our data science workloads to Hadoop clusters
- Develop new integrations with multiple Hadoop platforms (e.g. CDH, HortonWorks, EMR)
- Develop automated build and testing suites to ensure a stable, reliable product.
- Provide direct tier-3 support to customer installations of DataRobot, being our support team’s trusted advisor for the most technically challenging problems.
- Help expand Hadoop expertise within the organization by pairing with and mentoring colleagues on Hadoop technologies and best practices.
- Profound knowledge of at least one Hadoop distribution (Cloudera, Hortonworks, MapR, etc.).
- Deep experience with Hadoop ecosystem security technologies such as Kerberos, LDAP, and Active Directory.
- Detailed knowledge of core Hadoop architecture components (HDFS, YARN, Spark, etc).
- Familarily with Scala programming (expertise not required).
- Experience with Java application lifecycle; development, testing, build, and release.
- Excellent written and verbal communication skills.
- Great technical troubleshooting skills.
- Extensive experience as a technical lead.
Bonus Points for
- Scripting experience in Python and Bash.
- Experience working with RabbitMQ.
- Scala Expertise
- Hands-on experience using Hadoop data analysis tools.
- Experience with systems and application performance tuning.
- Experience with complex software packaging, delivery and maintenance.
- Previous startup experience.
DataRobot Engineering is a hard-working, fast moving, fun-loving team of developers who put product before pride. Our team is flexible and adaptable. We genuinely like each other and work hard to make sure that we all succeed, both for individual and company success, because we believe that one doesn’t happen without the other.