Data Engineer, Databricks

Guidehouse

Databricks Data Engineer supporting AI & Data team to design and maintain scalable data solutions. Collaborate on projects involving large-scale data pipelines and advanced analytics.

Posted 7/28/2026full-timeRemote • Colorado, District of Columbia, Illinois, Massachusetts, New York, Texas, Virginia • 🇺🇸 United StatesMid-LevelSenior💰 $113,000 - $188,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and maintaining scalable data pipelines using Databricks, PySpark, and SQL, while ensuring data quality and governance. Proficient in collaborating with cross-functional teams to translate business requirements into effective data engineering solutions.

Highest-signal resume keywords

Data Pipeline DevelopmentPython ProgrammingDatabricks PlatformData Quality ValidationCloud Data Engineering

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonPySparkSQLData ModelingData WarehousingETL/ELT WorkflowsData ProcessingData GovernancePerformance TuningData Validation

Soft Skills

CommunicationCollaborationProblem-SolvingTroubleshootingTeamwork

Tools & Technologies

DatabricksSparkDelta LakeUnity CatalogCloud Storage Services

Industry Keywords

Data EngineeringData ArchitectureCloud Data PlatformData QualityData Governance

Tech Stack

Tools & technologies

CloudETLPySparkPythonSparkSQLUnity

About the role

Key responsibilities & impact

Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, Delta Lake, and related cloud-native data engineering tools
Develop and support batch and streaming data workflows, including ingestion, transformation, validation, and publishing of curated data products
Write, optimize, and maintain Python, PySpark, and SQL code for data processing, orchestration, and performance tuning
Work with large-scale datasets using Databricks, Spark, Delta Lake, Unity Catalog, and cloud storage services
Troubleshoot and resolve data pipeline failures, performance issues, data quality issues, and workflow bottlenecks
Translate business and technical requirements into data engineering designs, processing scripts, and job orchestration workflows
Perform data validation, quality checks, code reviews, and issue resolution to support accurate and reliable data products
Collaborate with cross-functional teams including solution architects, data scientists, analysts, DevOps engineers, and client stakeholders
Communicate technical concepts, delivery impacts, and data engineering considerations to technical and non-technical audiences
Document pipelines, datasets, data models, workflows, and engineering decisions to support maintainability, transparency, and reuse
Follow data governance, security, lineage, and compliance standards within the Databricks platform

Requirements

What you’ll need

Bachelor’s degree in computer science, engineering, mathematics, statistics, or another relevant field
3-8 years of relevant experience in data engineering, data architecture, or cloud data platform implementation
Strong experience with Python, PySpark, and SQL for data transformation, pipeline development, and data processing
Experience with Databricks, Spark, Delta Lake, Unity Catalog, or similar cloud-native data platforms
Experience developing batch or streaming data pipelines, ETL/ELT workflows, and reusable data assets
Experience with data modeling, data warehousing, data quality validation, and large-scale data processing concepts
Ability to troubleshoot technical issues, communicate engineering recommendations clearly, and work effectively in team-based delivery environments
Experience applying data governance, access control, lineage, and security practices within cloud data platforms

Benefits

Comp & perks

Comprehensive total rewards package
Competitive compensation
Flexible benefits package