3+ years of hands-on experience in building large-scale data pipelines.
Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads
Proficiency in Pyspark/Python, including building Kafka producers for data ingestion
Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.
Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager
Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)
Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog
Strong SQL skills for validating data accuracy and developing job-level metrics
Experience with building alerting mechanisms for pipeline failures and performance issues
Strong collaboration and communication skills
Proven ability to take ownership of technical deliverables, driving execution from design to deployment
Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices
AWS Data Engineering - Associate Certified.
Good to have skills: Experience with orchestration tools like Apache Airflow or AWS Step Functions
Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi
Exposure to Terraform for infrastructure automation
Familiarity with CI/CD pipelines for data workflows

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data pipelinesSpark StreamingPysparkPythonKafkaSQLAWS GlueAWS EMRmonitoring dashboardsalerting mechanisms

Soft Skills

collaborationcommunicationmentoringownershiptechnical direction

Certifications

AWS Data Engineering - Associate Certified