Spear AI

Senior Data Engineer

Spear AI

full-time

Posted on:

Location Type: Remote

Location: United States

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowApacheDistributed SystemsIoTKafkaPostgresPython

About the role

  • Implement real-time data pipelines with MQTT and Redpanda for stream processing.
  • Implement offline data pipelines using Dagster for batch processing.
  • Parse and process binary message formats from various data sources.
  • Build data warehouses using Postgres, Apache Iceberg, Parquet, and S3.
  • Design data models that allow for high-performance queries.
  • Validate and normalize data sources.
  • Improve local development and CI/CD using modern tooling and GitHub Actions.

Requirements

  • Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.)
  • Experience with binary message parsing.
  • Proficiency in Python for data engineering workflows.
  • Experience with row-based & columnar-based data formats.
  • Experience with OLTP & OLAP databases.
  • Knowledge of distributed systems, streaming architectures, and batch processing patterns.
  • Hands-on experience with a batch orchestrator such as Dagster/Airflow.
  • Hands-on experience with a streaming platform such as Redpanda/Kafka.
  • Hands-on experience with binary message formats such as Protobuf.
  • Nice To Have: Experience with IoT devices and sensors.
  • Nice To Have: Digital signal processing experience.
  • Nice To Have: Geospatial analysis and GIS experience.
  • Nice To Have: Familiar with working in monorepos.
Benefits
  • Unlimited PTO — Take the time you need to recharge and maintain work-life balance.
  • Dedicated Sick Time — Your health and well-being come first.
  • Comprehensive Health & Benefits – Medical, dental, and vision coverage to keep you and your family protected.
  • 11 Paid Holidays — Enjoy time off throughout the year to celebrate and spend with loved ones.
  • Professional Development — Educational opportunities and resources to help you grow your skills and advance your career.
  • Collaborative Environment — Work directly with leadership in our flat organizational structure, where your ideas and contributions matter.
  • Mission-Driven Work — Contribute to projects that directly support national security and make a real-world impact.
  • Growth Opportunities — Join us during an exciting expansion phase where you can help shape our future.
  • 401(k) with company match.
  • Onsite / Remote / Flexible work arrangements or hybrid options (position dependent).
  • Relocation assistance (position dependent).
  • Referral bonuses.
  • Performance bonuses.
  • Life insurance and disability coverage.
  • Technology home office setup stipend.
  • Professional certification reimbursement (position dependent).
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
real-time data pipelinesoffline data pipelinesdata processingbinary message parsingPythondata engineering workflowsOLTP databasesOLAP databasesbatch processingstreaming architectures