Design and implement a scalable Lakehouse environment. Own the lifecycle of efficient storage, including schema evolution, partition transformation, and snapshot management.
Build and manage real-time and batch data ingestion pipelines using Kafka and Spark
Deploy, manage, and scale data workloads (such as Spark executors) using Kubernetes (EKS).
Develop complex ETL workflows. Optimize jobs for large-scale data processing focusing on memory management and shuffle optimization.
Manage and monitor end-to-end data lifecycles using orchestration tools
Write and tune high-performance queries against the Lakehouse. Implement optimization strategies, compaction, and data skipping to reduce latency and cloud costs

Requirements

5+ years in Data Engineering, with at least 2 years focused on Lakehouse or modern Data Lake environments
Hands-on experience with AWS ecosystems, async/queue processing in a production environment and working with a Lakehouse
Expert in workflow management, data transformation, storage and exchange at scale.
Proven experience with events driven job processing. Mastery in data ingestion, processing large-scale data and performance tuning.
Experience with Kubernetes and infrastructure as code

Benefits

Medical, dental, vision, and basic life insurance
PTO and company-paid holidays
Retirement programs
1% charitable giving program

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringLakehousedata ingestionETL workflowsperformance tuningdata transformationschema evolutionpartition transformationsnapshot managementmemory management

Soft Skills

workflow managementoptimization strategiesproblem-solvingcommunication