Tech Stack
AirflowAWSCloudGoogle Cloud PlatformKubernetesPythonTerraform
About the role
- Develop new tooling for product teams to boost their efficiency
- Maintain, evolve and scale data infrastructure solutions on AWS and Snowflake
- Onboard new ingestion sources and maintain smooth running and monitoring
- Ensure platform robustness through automated testing and monitoring of data pipelines
- Collaborate with stakeholders to translate business requirements into scalable technical solutions
- Optimise CI/CD pipelines for faster cycle times and increased reliability
- Implement security best practices for data management and infrastructure
- Design and deploy infrastructure as code to manage cloud resources efficiently
- Troubleshoot and resolve production issues to minimise downtime and improve user satisfaction
Requirements
- Strong software development, particularly in Python or a similar language
- Solid engineering practices, including automated testing, deployment systems, and configuration as code
- Experience with building data intensive applications and containerised services
- Experience with cloud services such as AWS, GCP, or equivalent (preference for AWS)
- Experience with infrastructure as code, preferably Terraform
- Knowledge of columnar databases, such as Snowflake
- Experience in developing and optimising CI/CD pipelines, with a preference for GitHub Actions
- Excellent communication skills for effective collaboration with business analysts and stakeholders
- Experience with data ingestion tools, like Fivetran
- Advantageous: exposure to Kubernetes
- Advantageous: experience with Data Orchestrator tools (Airflow, Prefect, etc.)
- Advantageous: experience with Data Observability tools (Montecarlo, Great Expectations, etc.)
- Advantageous: experience with Data Catalog tools (Amundsen, OpenMetadata, etc.)