Senior Data Engineer

Imagine Believe Realize, LLC.

Senior Data Engineer building secure Databricks, AWS, Kafka, and Palantir data platforms for IBR’s U.S. Navy mission programs.

Posted 9/11/2026full-timeRemote • 🇺🇸 United StatesSenior💰 $115,000 - $165,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing secure data engineering solutions, including data architectures, data lakes, and ETL/ELT pipelines. Proficient in utilizing tools such as Databricks, Apache Spark, and AWS for data processing and integration.

Highest-signal resume keywords

Data EngineeringETL/ELT Pipeline DevelopmentApache SparkDatabricksAWS Cloud Environment

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data ArchitectureData ModelingData QualityData IntegrationPythonSQLKafkaPySparkDelta LakeData Processing

Soft Skills

Team CollaborationCommunication

Tools & Technologies

DatabricksApache SparkKafkaAWS S3AWS LambdaKinesisDynamoDBPalantir FoundryGitJenkins

Certifications & Qualifications

CompTIA Security+CompTIA Network+

Industry Keywords

Data HubData LakeData WarehouseData Quality GatesData Lifecycle PoliciesCloud MonitoringCI/CD PipelinesTechnical DocumentationData Mapping ToolsEnterprise Data Applications

Tech Stack

Tools & technologies

ApacheAWSCloudDockerDynamoDBETLJenkinsKafkaPySparkPythonSparkSQL

About the role

Key responsibilities & impact

Design and implement secure, next-generation data engineering solutions supporting critical missions
Collaborate with technical leaders, business stakeholders, and multidisciplinary teams
Support implementation of the DON MAHRS program Data Strategy and High-Level Total Force Data Hub implementation plan
Develop, implement, integrate, and sustain data architectures, data hubs, data lakes, and data warehouses
Design and develop data models, data structures, and data acquisition processes
Develop engineering and implementation plans for data hubs, acquisition, modeling, integration, and related activities
Research and evaluate data sources to identify authoritative and reliable sources for data hub development
Plan, create, and maintain data architectures aligned with business requirements
Design, develop, maintain, and optimize ETL/ELT pipelines and transformation processes
Develop and maintain batch and streaming pipelines using Spark, Python, Databricks, Palantir Foundry, Kafka, and related tools
Develop data acquisition processes to ingest, transform, validate, and integrate enterprise data
Implement incremental loading, late-arriving data, processing-window, data-freshness, and lifecycle approaches
Automate manual data processes and improve workflow efficiency, reliability, and scalability
Develop, maintain, and optimize solutions within Databricks and enterprise data lakes
Configure, monitor, manage, and document Databricks clusters according to DON policies and security requirements
Develop and maintain Spark solutions using PySpark, Spark SQL, DataFrames, and Data Sets
Implement and maintain Delta Lake and Delta Live Tables solutions
Optimize data processing, storage, and query performance
Support Palantir Foundry ontologies, ETL/ELT pipelines, applications, and data-driven user interfaces
Integrate Palantir Foundry with enterprise data engineering and analytics environments
Develop and maintain streaming solutions using Kafka, Kafka Streams, and ksqlDB
Configure and manage Kafka topics and Schema Registry
Develop Python data-processing applications and AWS Lambda functions
Support processing with AWS S3, Kinesis, Lambda, and DynamoDB
Monitor, troubleshoot, and optimize cloud-based processing solutions
Develop data quality controls, validation processes, and data quality gates
Develop and execute data-driven and unit testing for Spark, Python, and other processing solutions
Establish data lifecycle policies covering retention, backup, recovery, and data management
Monitor and troubleshoot pipelines and processing jobs
Improve performance, reliability, scalability, and maintainability

Requirements

What you’ll need

5+ years of professional data engineering related experience
5+ years of IT experience focusing on enterprise data engineering, including data modeling, data quality, data mapping tools, and technical documentation
Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, or an IT-related field
Must be eligible to obtain and maintain a Secret security clearance
United States citizenship required to obtain this type of security clearance
Candidates must reside in the continental United States
Professional experience in data architecture, data engineering, data hub, data lake, and/or data warehouse development
Experience designing, developing, implementing, and supporting enterprise-scale data engineering solutions
Experience with data modeling, data mapping, data quality, data integration, and data management
Experience supporting large-scale, high-performance enterprise data applications
Experience developing and supporting ETL/ELT data pipelines and data transformation processes
Hands-on experience with Databricks required
Experience with Python and SQL for data engineering and data processing
Experience with Apache Spark and/or PySpark, including Spark SQL, DataFrames, and/or Data Sets preferred
Experience supporting data integration, migration, transformation, data warehouse, data hub, and/or data lake/Delta Lake implementations
Experience with batch and streaming data processing architectures
Experience with Kafka and/or other distributed streaming technologies
Experience developing and supporting Kafka Streams
Experience working in an AWS cloud environment, including S3, Lambda, Kinesis, and/or DynamoDB
Experience with Git-based version control, including branching, merging, pull requests, and repository management
Experience supporting CI/CD pipelines and cloud monitoring with Jenkins, Docker, and/or CloudWatch preferred
Experience with Palantir Foundry preferred
Experience with the Jupiter data and analytics environment preferred but not required
Ability to debug, troubleshoot, design, and implement solutions to complex technical issues
Ability to thrive in a team-based environment
Experience briefing technology solutions to technology partners, stakeholders, team members, and senior-level management
Strong written and verbal communication
Active CompTIA Security+ or CompTIA Network+ certification preferred; if selected, must obtain CompTIA Security+ before supporting the program

Benefits

Comp & perks

Nationwide medical, dental, and vision insurance
3 weeks of Paid Time Off
11 Paid Federal Holidays
401k matching
Life Insurance
Short-Term Disability at no cost to employees
Long-Term Disability at no cost to employees
Supplemental insurance options
Flexible spending accounts
Dependent Care spending accounts
Wellness incentives
Reimbursement for professional development and certifications
Training assistance opportunities to support career growth and progression
Flexible scheduling options
Hybrid and Remote work opportunities to support work-life balance