brightwheel

Senior Data Engineer

brightwheel

full-time

Posted on:

Origin:  • 🇧🇷 Brazil

Visit company website
✨ AI Apply
Apply

Salary

💰 $81,000 - $93,500 per year

Job Level

Senior

Tech Stack

AirflowAmazon RedshiftAWSBigQueryCloudETLHadoopHDFSMicroservicesPythonSparkSQL

About the role

  • Use modern tooling to build robust, extensible, and performant data platform
  • Build thoughtful and reliable data acquisition and integration solutions to meet business requirements and reporting needs
  • Deliver best in class application reporting solutions for customers
  • Troubleshoot, improve and scale existing data pipelines, models and solutions
  • Build upon data engineering's CI/CD deployments, and infrastructure-as-code for provisioning AWS services
  • Partner with technical leadership to craft and implement data strategy
  • Build and scale data pipelines that transform billions of records across numerous systems into measurable data that enable insights for the Analytics team and customers
  • Take full ownership of high impact projects with visibility throughout the organization

Requirements

  • 3+ years of work experience as a data engineer/dev ops/software engineering, coding in Python
  • 1+ years experience deploying data processing infrastructure as Code into AWS / cloud environment
  • 1+ years experience streaming real time data into a data lake / lake house architecture
  • Advanced understanding of how at least one big data processing technology works under the hood (e.g. Spark / Hadoop / HDFS / Redshift / BigQuery / Snowflake)
  • Demonstrated ability in writing and analyzing highly complex SQL queries to optimize performance
  • Excellent analytical, problem solving, and troubleshooting skills to manage complex process and technology issues without much guidance
  • 2+ years experience building streaming data pipelines in a cloud based data platform (preferred)
  • 1+ years deploying Infrastructure as Code within AWS CDK or similar (preferred)
  • 1+ years deploying microservices and/or APIs within cloud environment (preferred)
  • Experience with building ETL pipelines within Airflow / Python (preferred)
  • Experience with one or more CRMs such as Hubspot or Salesforce (preferred)
  • Experience with one or more clickstream processing tools such as Mixpanel (preferred)
  • Experience with serverless and/or event driven architecture (Databricks / Glue / Lambda) (preferred)
  • Experience with CubeJS, or relevant customer facing embedded analytics backend (preferred)