Senior Data Engineer

8020REI

Senior Data Engineer designing and optimizing the AWS data platform at 8020REI. Collaborate in a remote team to enhance AI-driven real estate solutions.

Posted 7/14/2026full-timeRemote • 🇨🇴 ColombiaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and optimizing PySpark ETL/ELT pipelines on AWS, with a strong focus on data quality, infrastructure as code, and collaboration with data science teams. Proficient in managing large-scale distributed data systems and ensuring production ownership through effective monitoring and documentation.

Highest-signal resume keywords

Advanced PysparkDeep AWS ExperienceInfrastructure As CodeData Quality MindsetAdvanced SQL

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PySparkPythonSQLAWS EMRAWS GlueAWS LambdaAWS S3AWS AthenaTerraformApache Hudi

Soft Skills

DocumentationCollaboration

Tools & Technologies

AWS Step FunctionsEventBridgeAPI GatewayDynamoDBAurora PostgreSQL

Industry Keywords

Data EngineeringLakehouseETLData QualityCI/CD

Tech Stack

Tools & technologies

ApacheAWSCloudDynamoDBETLPostgresPySparkPythonScikit-LearnSQLTerraform

About the role

Key responsibilities & impact

Own Big Data pipelines end to end.
Design, build, and optimize PySpark ETL/ELT pipelines on Amazon EMR and AWS Glue that process nationwide, county-partitioned property data (First American, BuildZoom permits, market comps) on daily and monthly cadences.
Run our lakehouse.
Operate and evolve our Bronze → Silver → Gold data lake on S3 with Apache Hudi and the AWS Glue Data Catalog, queried through Athena, including schema contracts, partitioning strategy, compaction, and performance tuning.
Orchestrate and automate.
Build reliable orchestration with AWS Step Functions, EventBridge, and Lambda; make reruns, backfills, and failure recovery boring and documented.
Enforce data quality.
Implement and extend our Data QA Audit Standard: layer contracts, write-audit-publish gating, quarantine flows, drift monitoring, and actionable Slack alerting, so bad data never reaches a client list.
Operate production databases and APIs.
Manage Aurora PostgreSQL and DynamoDB workloads, and run data-serving APIs (API Gateway, SQS-backed async workers), such as our Address Resolution Service, to production SLAs with dashboards and runbooks.
Own cloud cost.
Monitor, report, and reduce the AWS data-platform bill (EMR cluster sizing, Glue/Lambda usage, S3 lifecycle, Athena scan costs) as a first-class engineering responsibility.
Ship infrastructure as code.
Define infrastructure with Terraform and CloudFormation, delivered through GitHub Actions CI/CD with tests, linting, and coverage gates, we run a disciplined PR, branch-policy, and code-review culture.
Partner with Data Science.
Build the feature pipelines, training datasets, and serving paths behind our ML scoring and valuation models (scikit-learn/XGBoost-family stack), and co-own the handoff contracts between DS and DE.
Document like a pro.
Maintain runbooks, architecture docs, and data dictionaries (Confluence) so any teammate can operate what you build.

Requirements

What you’ll need

4+ years of hands-on data engineering with large-scale distributed data systems and a track record of production ownership (not just development).
Advanced PySpark : performance tuning, partitioning strategy, and cost-aware cluster sizing on real workloads (EMR or equivalent).
Strong Python : clean, tested, production-grade code (we use pytest, ruff, mypy, and coverage gates in CI).
Deep AWS experience : EMR, Glue, Lambda, S3, Athena, Step Functions, EventBridge, IAM, and VPC networking; comfort operating (not just deploying to) these services.
Advanced SQL : complex analytical queries, query optimization, and data modeling on both a warehouse/lake engine (Athena/Presto) and PostgreSQL.
Lakehouse experience : hands-on production work with at least one open table format (Apache Hudi strongly preferred; Iceberg or Delta Lake also valued) and medallion-style architecture.
Data quality mindset : experience implementing validation, quality gates, monitoring, and incident response for production data.
Infrastructure as Code : Terraform and/or CloudFormation in a CI/CD workflow.
English and Spanish : professional working proficiency in both (B2+).
Bachelor’s degree in Computer Science, Systems Engineering, Data Engineering, or equivalent practical experience.

Benefits

Comp & perks

Competitive Base Compensation
Profit Share Bonus
Flex PTO (up to 26 days per year)
Home Office Upgrade Bonus
HMO Bonus
Full-time Remote Work
Opportunity for growth and team-building potential
Ongoing support and budget to develop new skills