Nexus Cognitive

Data Platform Engineer

Nexus Cognitive

full-time

Posted on:

Location Type: Hybrid

Location: Pune • India

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AirflowAWSAzureCloudGoogle Cloud PlatformHadoopKafkaLinuxOpen SourceSpark

About the role

  • Manage daily operations of Nexusone products on on-premise and cloud based technologies using open-source Linux-based data technologies.
  • Install, configure, and support CDP clusters in both cloud (AWS, Azure, GCP) and on-premises environments, ensuring seamless integration and functionality.
  • Maintain, patch, and upgrade existing CDP setups, ensuring minimal downtime and adherence to Cloudera’s best practices.
  • Evaluate and optimize complex distributed production deployments, identifying bottlenecks and recommending performance enhancements.
  • Configure and manage security using tools like Ranger and Kerberos, ensuring robust data protection and compliance.
  • Configure Datadog and develop integrations to monitor and alert operations teams.
  • Develop and maintain technical documentation, including administration runbooks and knowledge base articles, to support operational excellence and knowledge sharing.
  • Work in an agile team and Participate in an on-call rotation, providing expert-level support and troubleshooting for critical issues.
  • Lead and mentor junior team members, fostering a culture of continuous learning and improvement. Collaborate with cross-functional teams, including data engineers, solution architects, and DevOps, to design and implement scalable data solutions.
  • Stay current with emerging technologies and industry trends, applying this knowledge to improve the CDP environment and drive innovation.

Requirements

  • Minimum of 5 years of experience in installing and administering the Cloudera Data Platform (CDP), with a proven track record of managing large-scale deployments.
  • Hands-on experience with open-source Linux-based data technologies, including Iceberg, Spark, Nifi, Jupyter Notebooks, Cloudera, Databricks, Kubeflow, MLFlow, and Kafka.
  • Proven ability to build and support solutions across major cloud platforms (AWS, Azure, GCP), leveraging cloud-native tools and open source for optimal performance.
  • Strong Cloudera expertise, with a deep understanding of Spark and Airflow for orchestrating complex data workflows.
  • Experience integrating Azure Active Directory with FreeIPA and other directory services such as LDAP, enhancing security and user management.
  • Up-to-date knowledge of the Hadoop Big Data ecosystem, staying current with the latest technologies and best practices.
  • Excellent troubleshooting skills, with a thorough understanding of CDP capacity planning, identifying bottlenecks, and optimizing memory utilization, CPU usage, OS performance, storage, and network configurations.
  • Strong analytical and problem-solving abilities, capable of diagnosing and resolving complex technical issues efficiently.
  • Cloudera Certified Administrator certifications is a plus.
Benefits
  • A collaborative team culture built on curiosity and respect
  • Challenging work where your contributions clearly matter
  • A leadership team that invests in learning and development
  • The opportunity to work at the intersection of cloud, data, and AI innovation
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
Cloudera Data Platform (CDP)Linux-based data technologiesIcebergSparkNifiJupyter NotebooksDatabricksKubeflowMLFlowKafka
Soft Skills
troubleshootinganalytical skillsproblem-solvingleadershipmentoringcollaborationcommunicationcontinuous learningagile methodologyknowledge sharing
Certifications
Cloudera Certified Administrator