Data Engineer, Databricks

ICF

Senior Databricks SME enabling scalable data solutions for federal clients. Supporting complex data projects within a collaborative cross-functional team environment.

Posted 6/17/2026full-timeRemote • Virginia • 🇺🇸 United StatesMid-LevelSenior💰 $98,614 - $167,644 per yearWebsite

Tech Stack

Tools & technologies

ApacheAzureCloudDistributed SystemsETLRDBMSSparkTableauTerraform

About the role

Key responsibilities & impact

Enable secure, scalable, and efficient data exchange between federal client and external data sharing partners using Databricks Delta Sharing.
Support the design and development of data pipelines and ETL routines in Azure Cloud environment for many source system types including RDBMS, API, and unstructured data using CDC, incremental, and batch loading techniques.
Conduct data profiling, transformation, and quality assurance on structured, semi-structured, and unstructured data.
Identify underlying issues and translate them into technical requirements.
Assist in building and optimizing data lakes, feature stores, and data warehouse structures to support analytics and machine learning.
Prepare, structure, and validate data for data science and MLOps workflows, ensuring it meets the quality and format requirements for modeling.
Help monitor and maintain the flow of data across BI dashboards, analytics environments, and machine learning pipelines.
Engage directly with clients and stakeholders to understand data needs and translate them into scalable solutions.
Collaborate with UX designers, business analysts, developers, and end users to define data and reporting requirements
Work with external data partners to determine their data product needs and work within the Databricks platform to enable rapid prototyping and extensible use cases
Meet with government employees at executive levels, platform stakeholders, and vendor partners.
Work within Agile teams to support iterative development, backlog grooming, and sprint-based delivery.
Provide mentorship to junior resources.

Requirements

What you’ll need

Bachelor’s degree
5+ years in data engineering, data security practices, data platforms, and analytics
U.S. Citizenship required due to federal contract requirements.
Ability to obtain and maintain a federal public trust clearance or equivalent client-required background investigation.
Candidate must reside in the U.S., be authorized to work in the U.S., and all work must be performed in the U.S.
Candidate must have lived in the U.S. for three (3) full years out of the last five (5) years
3+ years Databricks Platform Expertise – SME Level Proficiency including: Databricks, Delta Lake, and Delta Sharing
Deep experience with distributed computing using Apache Spark
Knowledge of Spark runtime internals and optimization
Ability to design and deploy performant end-to-end data architectures
4+ years of ETL Pipeline Development building robust, scalable data pipelines
Databricks certifications - Professional or specialty certifications
Hands-on experience with Azure services such as Synapse, Data Factory, or Databricks.
Familiarity with data visualization tools such as Tableau, Power BI, or similar.
Machine Learning and Analytical Skills including: MLOps - Working knowledge of ML deployment and operations
Data Science Methodologies - Statistical analysis, modeling, and interpretation
Big Data Technologies - Experience beyond Spark with distributed systems
Experience with deployment pipelines, including Git-based version control and CI/CD pipelines and DevOps practices using Terraform for IaC.

Benefits

Comp & perks

Reasonable Accommodations are available, including, but not limited to, for disabled veterans, individuals with disabilities, and individuals with sincerely held religious beliefs, in all phases of the application and employment process.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringdata security practicesdata platformsanalyticsDatabricksDelta LakeDelta SharingApache SparkETL pipeline developmentMLOps

Soft Skills

mentorshipcollaborationcommunicationproblem-solvingclient engagement

Certifications

Databricks Professional certificationDatabricks specialty certification