Senior Data Engineer – Databricks, AWS

nDeavour Consulting

Senior Data Engineer at Mobile Wave Solutions designing, building, and optimizing Lakehouse data platform. Seeking proactive problem-solver experienced in cloud data engineering.

Posted 7/14/2026full-timeRemote • 🇺🇸 United StatesSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing scalable data ingestion and transformation pipelines using Databricks and PySpark, with a strong focus on data governance and quality frameworks. Proficient in SQL optimization and Python scripting, with a collaborative approach to problem-solving in fast-paced environments.

Highest-signal resume keywords

DatabricksPySparkSQL OptimizationDelta LakeAirflow

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data EngineeringCloud ProcessingDistributed ProcessingCDCSchema EvolutionPythonSQLData Quality FrameworksStreamingData Governance

Soft Skills

Proactive Problem-SolverCollaborativePragmaticIndependent WorkerAdaptable

Tools & Technologies

AirflowUnity CatalogGreat ExpectationsKafkaKinesis

Industry Keywords

CI/CDData IngestionData TransformationData IntegrityLegacy Datasets

Tech Stack

Tools & technologies

AirflowAWSCloudKafkaPySparkPythonSQLUnity

About the role

Key responsibilities & impact

Design and implement robust, scalable data ingestion and transformation pipelines using Databricks, PySpark, and distributed processing. Utilize Airflow (or similar) for reliable workflow orchestration.
Implement Delta Lake principles, focusing on CDC and schema evolution. Establish and integrate data quality frameworks (e.g., Great Expectations) within CI/CD pipelines for data integrity.
Develop and optimize complex SQL and Python scripts. Integrate diverse data sources (APIs, S3 files, etc.) and handle both structured and unstructured data.
Support the implementation of data governance and cataloguing solutions (e.g., Unity Catalog). Proactively investigate and improve inconsistent legacy datasets.
Guide and manage AI agents for code generation, ensuring quality and stability. Work pragmatically and collaboratively to drive technical solutions.

Requirements

What you’ll need

5+ years of professional experience in data engineering, focused on cloud and distributed processing environments.
Strong experience with Databricks, PySpark, distributed processing, and Delta Lake. Deep knowledge of CDC and schema evolution.
Expert SQL optimization and Python skills. Hands-on experience with Airflow (or similar).
Familiarity with relevant AWS components. Good understanding of CI/CD for data workflows and implementing data quality frameworks.
Knowledge of streaming (Kafka/Kinesis) and exposure to Unity Catalog or similar governance tools.
Proactive problem-solver; comfortable working with complex, inconsistent data without needing explicit specs.
Able to work independently in a fast-paced rebuild environment.
Collaborative, pragmatic, thrives on rapid iteration.
Skilled at directing AI agents to code while maintaining high quality standards.

Benefits

Comp & perks

Remote Office – Flexible hybrid form of working
Parking Space – We provide free parking spots
Fun Office Space – We offer a game zone and a relaxation area
Health Insurance – Additional private health insurance, including a dental care plan
Personal Development – Company-sponsored training budget to further develop your skills
Employee Referral Program – Receive a bonus for referring a friend
Holidays – Enjoy an extra 5 days after your 1st and 5th year
Social Events – We love to celebrate our success together
Family Insurance – Add insurance to a family member
Offering sports cards – 100% sponsored by the company