Tech Stack
AirflowAmazon RedshiftApacheAWSCloudDynamoDBETLKafkaKubernetesNoSQLNumpyPandasPostgresPySparkPythonSparkSQLTerraform
About the role
- Provide strategic vision and technical leadership in data engineering, guiding advanced data strategies
- Design and architect complex data pipelines and scalable architectures using tools like Apache Kafka and Kubernetes
- Optimize ETL processes, data modelling, and data warehousing for efficiency, scalability, and reliability
- Collaborate closely with cross-functional teams and stakeholders to design and implement robust data solutions
- Mentor and coach junior team members and foster growth in data engineering practices
- Lead exploration and adoption of new technologies and methodologies to drive innovation and continuous improvement
- Contribute to thought leadership through technical articles, conference presentations, and industry participation
Requirements
- 8+ years of experience in data engineering
- Strong expertise in designing and implementing data warehouse and data lake architectures, particularly in AWS environments
- Extensive experience with Python for data engineering tasks and familiarity with common libraries/frameworks
- Proven experience with data pipeline orchestration using Airflow, Databricks, DBT or AWS Glue
- Hands-on experience with data analysis tools and libraries such as PySpark, NumPy, Pandas, or Dask
- Proficiency with Spark and Databricks (highly desirable)
- Experience with SQL and NoSQL databases including PostgreSQL, Amazon Redshift, Delta Lake, Iceberg, and DynamoDB
- Deep knowledge of data architecture principles and best practices in cloud environments
- Proven experience with AWS services, AWS CLI, SDK, and IaC tools (Terraform, CloudFormation, or AWS CDK)
- Deep expertise in ETL processes, data modelling, and data warehousing
- Exceptional communication skills for technical and non-technical stakeholders
- Demonstrated ability to quickly adapt to new tasks and roles in a dynamic environment