Fluent, Inc

Senior Data Engineer – Databricks

Fluent, Inc

full-time

Posted on:

Location Type: Hybrid

Location: Toronto • 🇨🇦 Canada

Visit company website
✨ AI Apply
Apply

Salary

💰 CA$110,000 - CA$150,000 per year

Job Level

Senior

Tech Stack

ApacheAWSKafkaPySparkSparkSQLUnity

About the role

  • Design, build, and support scalable real-time and batch data pipelines using PySpark and Spark Structured Streaming.
  • Develop pipelines following the Bronze → Silver → Gold architecture using Delta Lake and Enterprise Data Model best practices.
  • Integrate with Kafka for event-driven ingestion and stream processing.
  • Orchestrate workflows with Databricks Workflows/Jobs and DABs.
  • Implement monitoring and observability—Databricks metrics, dashboards, and alerts to ensure pipeline reliability and performance.
  • Collaborate cross-functionally in agile sprints with Product Managers, Data Scientists, and downstream data consumers.
  • Partner closely with Data Architects to translate Enterprise Data Models into performant physical data models.
  • Write clean, modular, and version-controlled code in Git-based CI/CD environments; perform rigorous peer reviews.
  • Implement robust logging, error handling, and data quality validation throughout pipelines.
  • Utilize AWS services (S3, IAM, Secrets Manager) for storage and infrastructure.
  • Evangelize engineering best practices through brown bags, tech talks, and documentation.
  • Stay current on emerging trends within the Databricks and data engineering ecosystem.

Requirements

  • 5+ years of experience in Data Engineering, including strong Spark (PySpark) and SQL expertise.
  • 3+ years of hands-on experience building pipelines on Databricks (Workflows, Notebooks, Delta Lake).
  • Deep understanding of Apache Spark distributed processing model and internals.
  • Strong experience with streaming data architectures and event-driven processing using Kafka.
  • Familiarity with Databricks metrics, observability, and monitoring features.
  • Understanding of Unity Catalog and Lakehouse architecture.
  • Knowledge of idempotent processing patterns and robust data modeling.
  • Proficiency in Git-based, CI/CD-driven development workflows.
  • Strong debugging, optimization, and performance tuning skills.
  • Proven experience building large-scale data pipelines handling massive volumes of data.
Benefits
  • Competitive compensation
  • Ample career and professional growth opportunities
  • New Headquarters with an open floor plan to drive collaboration
  • Health, dental, and vision insurance
  • Pre-tax savings plans and transit/parking programs
  • 401K with competitive employer match
  • Volunteer and philanthropic activities throughout the year
  • Educational and social events
  • The amazing opportunity to work for a high-flying performance marketing company!

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
PySparkSpark Structured StreamingSQLDelta LakeKafkaGitCI/CDdata quality validationdebuggingperformance tuning
Soft skills
collaborationcommunicationagile methodologypeer reviewsengineering best practices