BioCatch

Data Engineer

BioCatch

full-time

Posted on:

Origin:  • 🇮🇱 Israel

Visit company website
✨ AI Apply
Manual Apply

Job Level

Mid-LevelSenior

Tech Stack

AirflowAmazon RedshiftAzureBigQueryCloudDockerETLGoogle Cloud PlatformJenkinsKafkaKubernetesNeo4jPrometheusPySparkPythonRabbitMQSparkSQLTerraform

About the role

  • Provide the direction of our data architecture. Determine the right tools for the right jobs. We collaborate on the requirements and then you call the shots on what gets built. Manage end-to-end execution of high-performance, large-scale data-driven projects, including design, implementation, and ongoing maintenance. Optimize and monitor the team-related cloud costs. Design and construct monitoring tools to ensure the efficiency and reliability of data processes. Implement CI/CD for Data Workflows

Requirements

  • 3+ Years of Experience in data engineering and big data at large scales. - Must Extensive experience with modern data stack - Must: Snowflake, Delta Lake, Iceberg, BigQuery, Redshift Kafka, RabbitMQ, or similar for real-time data processing. Pyspark, Databricks Strong software development background with Python/OOP and hands-on experience in building large-scale data pipelines. - Must Hands-on experience with Docker and Kubernetes. - Must Expertise in ETL development, data modeling, and data warehousing best practices. Knowledge of monitoring & observability (Datadog, Prometheus, ELK, etc) Experience with infrastructure as code, deployment automation, and CI/CD. Practices using tools such as Helm, ArgoCD, Terraform, GitHub Actions, and Jenkins.