Flex

Staff Data Engineer

Flex

full-time

Posted on:

Origin:  • 🇺🇸 United States • California

Visit company website
✨ AI Apply
Manual Apply

Salary

💰 $199,000 - $262,000 per year

Job Level

Lead

Tech Stack

AirflowAWSCassandraETLHadoopJavaKafkaNoSQLPythonSparkSQLTerraform

About the role

  • Design, implement, and maintain high-quality data infrastructure services, including Data Lake, Kafka, Amazon Kinesis, and data access layers
  • Develop robust and efficient DBT models and jobs to support analytics reporting and machine learning modeling
  • Closely collaborate with the Analytics team for data modeling, reporting, and data ingestion
  • Create scalable real-time streaming pipelines and offline ETL pipelines
  • Design, implement, and manage a data warehouse that provides secure access to large datasets
  • Continuously improve data operations by automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability
  • Create engineering documentation for design, runbooks, and best practices
  • Shape the future of finance data platforms to enable Machine Learning and Business Intelligence across the company

Requirements

  • A minimum of 8 years of industry experience in the data infrastructure/data engineering domain
  • A minimum of 8 years of experience with Python and SQL
  • Java experience is a plus
  • A minimum of 4 years of industry experience using DBT
  • A minimum of 4 years of industry experience using Snowflake and its basic features
  • A minimum of 4 years of industry experience using Infrastructure as Code tools, specifically CDK and Terraform
  • Strong written and verbal communication skills for key collaboration
  • Familiarity with AWS services (Lambda, Step Functions, Glue, RDS, EKS, DMS, EMR)
  • Industry experience with big data platforms and tools such as Kafka, Hadoop, Hive, Spark, Cassandra, Airflow
  • Industry experience working with relational and NoSQL databases in a production environment
  • Strong fundamentals in data structures, algorithms, and design patterns
  • Prior experience working on cross-functional teams
  • Experience with CI/CD to improve code stability and code quality
  • Motivated to help other engineers succeed and be effective
  • Comfortable working in an ambiguous, fast-paced, high-growth environment