Decision Foundry

Senior Data/Platform Engineer – Embedded, Data & Analytics Engineering

Decision Foundry

contract

Posted on:

Location Type: Remote

Location: India

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowAWSCloudPythonTerraform

About the role

  • **Key Responsibilities:**
  • **What You’ll Work On**
  • You will integrate into our team to accelerate well-scoped execution work, including:
  • - **Data pipeline and ingestion expansion** across multiple sources and delivery patterns
  • - **Platform hardening and refactoring initiatives** to improve scalability and maintainability
  • - **Observability, testing, and reliability improvements** across orchestration and batch workloads
  • - **Deployment and modularization** of pipeline components to support repeatable onboarding of net-new data capabilities
  • - Supporting **dbt model and mart development** (big plus) and maintaining analytics transformations in Snowflake
  • ****Core Responsibilities**
  • - Build and maintain **serverless, containerized batch pipelines** orchestrated via **Prefect** (similar to Airflow)
  • - Expand ingestion and connectivity patterns across:
  • - **APIs**
  • - **S3-based sources**
  • - **SFTP infrastructure**
  • - **Email scraping**
  • - **Web scraping**
  • - Develop and enhance internal **Python libraries** used to standardize ingestion, transformation, and pipeline deployment patterns
  • - Implement and improve **data observability** practices including monitoring, alerting, and failure diagnostics
  • - Contribute to **infrastructure-as-code** using **Terraform** to support repeatable deployments and environment consistency
  • - Support and improve the data warehouse ecosystem:
  • - **Snowflake** as the primary data warehouse
  • - **dbt on Snowflake** for modeling and analytics transformations
  • - Collaborate closely with internal engineers through PR reviews, sprint workflows, and team standards.
  • - Operate within existing repos, processes, and CI/CD workflows to increase throughput while maintaining quality

Requirements

  • **Required Qualifications**
  • - 6+ years of experience in **Data Engineering, Platform Engineering, or Software Engineering** with strong data systems exposure
  • - **Expert-level Python** skills with a track record of building production-grade libraries and services
  • - Strong experience building and operating **batch pipeline infrastructure** in cloud environments (AWS preferred)
  • - Experience with workflow orchestration tools such as **Prefect, Airflow, Dagster**, etc.
  • - Strong understanding of data pipeline design: modularity, idempotency, retries, deployment patterns, and maintainability
  • - Experience implementing **data observability**, monitoring, logging, alerting, and testing frameworks
  • - Hands-on experience with **Terraform** or similar infrastructure-as-code tooling
  • - Comfortable working in an embedded model: collaborating inside existing repos, PR workflows, and delivery processes
  • ****Preferred / Nice-to-Have**
  • - Strong experience with **dbt** (models, marts, testing, documentation)
  • - Experience with **Snowflake performance optimization** and warehouse best practices
  • - Experience with **web scraping and/or email scraping pipelines**
  • - Familiarity with containerized workloads and serverless compute patterns
  • - Strong instincts for platform refactoring, system hardening, and reliability engineering
Benefits
  • **Equal Opportunity Statement**
  • We are committed to building a diverse and inclusive team. We welcome applications from candidates of all backgrounds and are an equal opportunity employer. We provide equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, gender identity, sexual orientation, marital status, or veteran status.
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
Pythondata pipeline designbatch pipeline infrastructuredata observabilityinfrastructure-as-codedbtweb scrapingemail scrapingSnowflakeTerraform
Soft Skills
collaborationcommunicationproblem-solvingattention to detailadaptabilityteamworkcritical thinkingtime managementleadershiporganizational skills