Senior Lead Data Engineer, Content Engineering

Paramount

Senior Lead Data Engineer building and scaling data foundations for next-generation personalization systems at Paramount. Collaborating with teams to deliver reliable, ML-ready data at global scale.

Posted 5/6/2026full-timeRemote • New York • 🇺🇸 United StatesSenior💰 $156,800 - $235,200 per yearWebsite

Tech Stack

Tools & technologies

AirflowAmazon RedshiftAWSAzureGoogle Cloud PlatformKafkaPySparkPythonScalaSparkSQLTerraform

About the role

Key responsibilities & impact

Build & Operate Large-Scale Feature Pipelines: Design and maintain batch/streaming pipelines (Spark, Flink, Databricks, Airflow) producing ML features for ranking models.
Ensure Point-in-Time Correctness: Develop feature sets that enable unbiased offline training and credible online inference.
Develop Embedding & Content Pipelines: Build scalable workflows for metadata, imagery, and multimodal representations; partner with Science teams to operationalize new models.
Architect Data Foundations: Design Delta/Parquet data models and medallion layers, optimizing storage layout and partitioning for latency and cost.
Real-Time Engineering: Build Kafka-based systems for real-time features and user-activity aggregations, ensuring robust handling of out-of-order events and exactly-once semantics.
Governance & Leadership: Define data quality rules and schema evolution processes while collaborating across ML pods to translate model needs into infrastructure.

Requirements

What you’ll need

7+ years of experience in large-scale data or software engineering
Deep experience with Spark (PySpark/Scala), Databricks, Airflow, and Kafka.
Proficiency in feature pipelines, temporal joins, and mitigating training-serving skew.
Experience with AWS/Azure/GCP and high-performance engines like Snowflake or Redshift.
Proficient programming skills in Python and SQL with a focus on performance optimization.
Experience in personalization domains (search, ranking, or recommender systems).
Experience supporting petabyte-scale data lakehouses or feature stores.
Familiarity with GenAI/RAG systems, multimodal content, or Delta Live Tables.
Knowledge of Causal Inference, experimentation signals, or ML evaluation workflows.
Experience with Terraform for governed, repeatable deployments.

Benefits

Comp & perks

medical
dental
vision
401(k) plan
life insurance coverage
disability benefits
tuition assistance program
PTO

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

SparkFlinkDatabricksAirflowKafkaPythonSQLTerraformDeltaParquet

Soft Skills

governanceleadershipcollaboration