DevOps Engineer

• Manage, capacity plan, and operate workloads utilizing EC2 clusters via DataBricks/EMR to ensure efficient and reliable data processing
• Collaborate with stakeholders to design and implement a Data Mesh architecture across multiple closely related but separate enterprise entities
• Utilize Infrastructure as Code (IaC) tools such as CloudFormation or Terraform to define and manage data platform user access to data and compute resources.
• Implement role-based access control (RBAC) mechanisms using IaC templates to enforce least privilege principles and ensure secure access to data and compute resources
• Collaborate with cross-functional teams to design, implement, and optimize data pipelines and workflows
• Utilize distributed engines such as Spark to process and analyze large volumes of data efficiently when required
• Develop and maintain operational best practices for Spark and other data warehousing tools to ensure system stability and performance
• Implement and manage storage technologies to efficiently store and retrieve data as per business requirements
• Troubleshoot and resolve platform-related issues in a timely manner to minimize downtime and disruptions
• Stay updated on emerging technologies and industry trends to continuously enhance the data platform infrastructure
• Document processes, configurations, and changes to ensure comprehensive system documentation.

Senior DevOps, Data

Senior DevOps Engineer, Cloud, AWS