Tech Stack
GoGrafanaKubernetesPrometheusPythonTerraform
About the role
- You will be part of the data platform team, which enables applied research teams at poolside to build high quality and large datasets for pretraining and postraining, run large-scale experiments end to end from data retrieval to running evaluations and benchmarks.
- Develop a robust platform that supports the creation of high-quality data pipelines and enables fast, efficient, end-to-end experimentation.
- Build modular, reusable data processing frameworks and libraries in Python.
- Work with applied research teams to build, test, scale and improve specific pipelines.
- Work with both open-source big data solutions and internal systems.
- Enhance system performance and reliability to allow teams to iterate faster.
- Lead best practices in Python development.
Requirements
- Proficient in Python
- Debugging and problem-solving: using debugging tools, profilers, etc.
- Experience with build tools, packaging and CI/CD pipelines
- Experience with testing code: unit testing, integration testing
- Experience with Kubernetes, Terraform
- Experience with monitoring and alerting (Grafana, Prometheus, Datadog)
- Programming skills in Go or other languages