AWS Data Lake Engineer
Worldpay
full-time
Posted on:
Location Type: Office
Location: Bangalore • 🇮🇳 India
Visit company websiteJob Level
Mid-LevelSenior
Tech Stack
AWSPandasPySparkPython
About the role
- Design, develop, and maintain scalable data pipelines using Python and AWS services
- Implement and manage AWS Data Lake solutions, including ingestion, storage, and cataloging of structured and unstructured data
- Apply data tokenization and masking techniques to protect sensitive information in compliance with data privacy regulations (e.g., GDPR, HIPAA)
- Collaborate with data engineers, architects, and security teams to ensure secure and efficient data flows
- Optimize data workflows for performance, scalability, and cost-efficiency
- Monitor and troubleshoot data pipeline issues and implement robust logging and alerting mechanisms
- Document technical designs, processes, and best practices
- Provide support on Databricks and Snowflake
- Maintain comprehensive documentation for configurations, procedures, and troubleshooting steps.
Requirements
- 5+ years of experience working as a Python developer/architect
- Strong proficiency in Python, with experience in data processing libraries (e.g., Pandas, PySpark)
- Proven experience with AWS services such as S3, Glue, Lake Formation, Lambda, Athena, and IAM
- Solid understanding of data lake architecture and best practices
- Experience with data tokenization, encryption, and anonymization techniques
- Familiarity with data governance, compliance, and security standards
- Experience with Snowflake and/or Databricks (Nice to have)
- Experience with CI/CD tools and version control (e.g., Git, CodePipeline)
- Strong problem-solving skills and attention to detail.
Benefits
- A competitive salary and benefits
- Time to support charities and give back to your community
- Parental leave policy
- Global recognition platform
- Virgin Pulse access
- Global employee assistance program
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
Pythondata processingPandasPySparkAWS S3AWS GlueAWS Lake FormationAWS LambdaAWS AthenaAWS IAM
Soft skills
problem-solvingattention to detailcollaboration