Arrive Logistics

Senior Data Engineer

Arrive Logistics

full-time

Posted on:

Origin:  • 🇺🇸 United States

Visit company website
✨ AI Apply
Manual Apply

Job Level

Senior

Tech Stack

AirflowAmazon RedshiftAzureBigQueryCloudKafkaPythonSQL

About the role

  • Design, build, and optimize robust, scalable, and reliable data pipelines to ingest and process data from a wide variety of sources.
  • Develop and maintain our data warehousing solutions (e.g. Snowflake)
  • Ensure data quality and integrity by implementing data validation frameworks, monitoring systems, and anomaly detection protocols.
  • Architect and manage cloud-based data infrastructure.
  • Collaborate with stakeholders, including data scientists, analysts, and product managers to understand data requirements and deliver actionable solutions.
  • Be a leader, mentor, and subject matter expert for the team, stakeholders, and peers. Foster a collaborative environment that drives solutions forward at a larger scope.
  • Improve the performance and efficiency of existing data systems and processes.
  • Work alongside Data and Engineering leadership to weigh tradeoffs in build vs buy decisions and tradeoffs in product roadmap vs technical initiatives
  • Interact with vendors to craft the adoption of new technology and infrastructure for the team

Requirements

  • Bachelor’s degree in Computer Science, Engineering, or a related field or equivalent professional experience.
  • 5+ years experience in data engineering.
  • Expert proficiency in SQL and strong programming skills in Python.
  • Proven experience building and orchestrating data pipelines using dbt and Airflow
  • Hands-on experience developing data solutions in a major cloud environment, with a strong preference for Microsoft Azure.
  • Solid understanding of BI-tooling integrations with a cloud data warehouse, with a strong preference for PowerBI
  • Solid understanding of event-driven architecture and the use of messaging systems like Kafka to enable real-time data flow.
  • Experience with a modern cloud data warehouse like Snowflake, Databricks, BigQuery, or Redshift.
  • Familiarity with CI/CD, microservice architecture, domain driven design, data contracts
  • Solid understanding with best practices around data security/privacy, role-based access control
  • Experience with machine learning concepts, frameworks, and infrastructure