Senior / Staff Data Engineer
AirOps
full-time
Posted on:
Location Type: Hybrid
Location: New York City • California • New York • United States
Visit company websiteExplore more
Job Level
Tech Stack
About the role
- Reliability and completeness of our multi-provider ingestion pipeline: build failover logic, track daily completion rates per AI provider, and maintain the systems that ensure customers always get accurate, timely data
- Data quality and consistency end to end: own the contracts between ingestion, processing, and downstream serving so that what customers see in the product is correct and trustworthy
- Extraction pipeline performance: design and maintain async processing patterns that keep our ingestion throughput ahead of demand as we scale
- Data readiness for new features: partner with engineering upstream so instrumentation, schema changes, and pipeline capacity are in place before features ship
- Pipeline observability: build and maintain health monitoring across the full data stack so the team knows the state of the system in real time
Requirements
- Built and maintained production data pipelines at meaningful scale, including experience with ingestion, ETL, and distributed job queue systems
- Hands-on experience with ClickHouse or a comparable columnar store, including schema design, materialized views, and query optimization
- Experience working with third-party data providers or scraping infrastructure at scale
- Designed and owned reliability and observability systems: you have built the dashboards and alerting that tell you what is wrong before users do
- Comfortable working close to the application layer: our stack is Rails and Postgres and this person will need to understand how backend systems feed the data platform
Benefits
- Equity in a fast-growing startup
- Competitive benefits package tailored to your location
- Flexible time off policy
- Parental Leave
- A fun-loving and (just a bit) nerdy team that loves to move fast!
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data pipelinesETLdistributed job queue systemsClickHouseschema designmaterialized viewsquery optimizationreliability systemsobservability systemsbackend systems
Soft Skills
problem-solvingcollaborationcommunicationattention to detailproactive monitoring