Commerzbank Digital Technology Centre Poland

Senior Data Engineer

Commerzbank Digital Technology Centre Poland

full-time

Posted on:

Origin:  • 🇵🇱 Poland

Visit company website
✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowApacheBigQueryCloudETLGoogle Cloud PlatformPySparkPythonSQLTerraform

About the role

  • Lead and execute the migration of ETL processes and workflows to Google Cloud Platform, ensuring high performance and minimal disruption
  • Writing efficient cloud-native code, optimizing it for scalability, performance, and maintainability
  • Contribute to the wider lifecycle of product development, identifying opportunities for functional improvements, enhancements, and innovations
  • Design and build interfaces between data storage and the data transformation engine in the form of scalable and secure APIs
  • Work closely with cross-functional teams, including data scientists, product managers, and software developers, to deliver high-quality solutions

Requirements

  • Very good knowledge of data pipeline orchestration (design scalable, cloud-native data pipelines for data transformation and aggregation based on business use cases)
  • Very good knowledge of GCP (or other Cloud) and creating Cloud based architecture (BigQuery, Dataproc/PySpark, Cloud Composer/Apache Airflow)
  • Very good knowledge of Python and SQL
  • Good knowledge of API (understanding of how frontend apps interact with backend APIs in a full-stack environment)
  • Good knowledge of GIT and CI/CD solutions
  • English level B2
  • IaC (Terraform) - Nice to have
  • German - would be a plus