Senior Data Engineer

Hack The Box

Senior Data Engineer at Hack The Box owning and evolving data pipelines on GCP. Collaborating with teams for modern data architecture and ML applications.

Posted 7/16/2026full-timeRemote • Florida, New Jersey, New York, North Carolina, Virginia • 🇺🇸 United StatesSenior💰 $140,000 - $160,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and optimizing data pipelines on GCP, with a strong focus on data quality, modeling, and orchestration using tools like Airflow. Proficient in integrating machine learning workflows and ensuring reliable data availability for analytics and inference.

Highest-signal resume keywords

GCP Data ServicesData ModelingWorkflow Orchestration with AirflowStreaming Pipelines with DataflowSQL and Python Proficiency

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data ModelingSQLPythonBigQueryDataflowFlinkSpark Structured StreamingClickHouseAirflowCI/CD

Tools & Technologies

Pub/SubKafkaDockerKubernetesDbtCloud ComposerVertex AIFeature Store

Industry Keywords

ETLELTData QualityData ReliabilityFeature EngineeringModel DeploymentDrift Monitoring

Tech Stack

Tools & technologies

AirflowAmazon RedshiftBigQueryCloudDockerETLGoogle Cloud PlatformKafkaKubernetesPythonSparkSQL

About the role

Key responsibilities & impact

You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation.
Designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark).
Orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference.
Partnering with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed.
Consuming and building REST APIs, integrate with third-party SaaS sources, and treating infrastructure as code.
Collaborating closely with Infrastructure, Software Engineering, Product, and ML/AI engineers.
Helping drive the migration off Snowflake onto GCP-native stack — and retire shadow pipelines along the way.
Owning the orchestration layer in Airflow, including SLAs, retries, and data quality gates.
Modeling data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference.
Capturing requirements from stakeholders and translating them into pragmatic, well-scoped data products.
Continuously improving data quality, reliability, observability, and cost efficiency.
Identifying new data sources worth acquiring and integrating them cleanly.

Requirements

What you’ll need

Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns)
Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses
Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub
Solid SQL and strong Python — you write production-quality code, not just notebooks
Experience with ClickHouse or another columnar OLAP engine in production
Workflow orchestration experience with Airflow (or Prefect/Dagster)
Comfortable with dbt or equivalent transformation frameworks
Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus
Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining
Docker & Kubernetes experience
CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems
Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store

Benefits

Comp & perks

Medical, Dental & Vision (employee coverage 100% paid for by Hack The Box)
401K w/ employer match
Employer-paid Life and AD&D Insurance
Supplemental Life Insurance
Short-term and Long-term Disability
Healthcare and Dependent Care FSA
Paid parental leave
25 annual leave days
Home Office Allowance
Dedicated budget for training and professional development, participation in conferences
State-of-the-art equipment
Full access to the Hack The Box lab offerings; so you can learn how to hack 😉