Tech Stack
AirflowApacheBigQueryETLKubernetesPythonSQLTerraform
About the role
- Data collection and integration from third-party services (APIs, SFTP, databases, etc.)
- Design and development of ETL/ELT processes for analytical data warehouse.
- Assistance to analytics team with queries refactoring and optimization.
- Code refactoring and performance optimization of existing data pipelines.
- Ensuring data quality and consistency across the warehouse.
- Monitoring and maintaining data infrastructure.
- Participate in developing a high-load analytical platform, work with Big Data, and build scalable solutions for data storage and processing.
- Work directly impacting key business decisions and product development.
Requirements
- Experience in Python, middle+ or senior level.
- Experience with Apache Airflow (2.8+) for data orchestration.
- Deep knowledge of SQL and experience with database and queries optimization.
- Experience with columnar databases (Starrocks, ClickHouse, Vertica, etc.)
- Experience in ELT processes design and implementation.
- Experience working with large datasets and performance optimization.
- Nice to Have: Experience with StarRocks and BigQuery.
- Nice to Have: Infrastructure-as-Code experience with Terraform.
- Nice to Have: Experience with DBT.
- Nice to Have: Experience with Kubernetes (as a user).