Develop reusable ingestion frameworks (Python/Airflow/AWS Glue) for APIs and unstructured sources beyond Fivetran, handling various data formats (JSON, Parquet, etc.).
Own the end-to-end Medallion (bronze/silver/gold) architecture for core domains, ensuring robust lineage and metadata across diverse data sources.
Implement data observability (native tests, alerts, lineage hooks); lead incident management and root-cause analysis (RCA) for data.
Help standardize reusable “paved-road” patterns (e.g. CI templates, ingestion operators) to improve developer productivity.
Prepare datasets for AI/LLM use cases (feature stores, embeddings/RAG prep).

Requirements

3–5+ years of data engineering with strong Python and SQL; hands-on Spark/PySpark (ideally via AWS Glue).
Deep experience in AWS (S3, IAM, Lambda, CloudWatch) running secure, observable data workloads.
Proficiency operating Snowflake (warehouse sizing, RBAC, resource monitors, clustering/partitioning).
Proven governance/security patterns: masking policies, row-level security, and auditability.
Orchestration experience (Airflow/MWAA) and event/file/API ingestion beyond managed connectors.
CI/CD for data with GitHub Actions; test/promotion workflows; secrets and PII handling.
Solid grasp of Medallion architecture, dimensional modeling (star schema), and data quality frameworks.
Ownership of incident management and RCA with measurable reduction in MTTR.

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

PythonSQLSparkPySparkAWS GlueAWS S3AWS IAMAWS LambdaAWS CloudWatchSnowflake

incident managementroot-cause analysisdeveloper productivitydata observabilitygovernancesecurity patterns