Data Engineer

• Develop, maintain, troubleshoot, and enhance ATIS data products, datasets, pipelines, applications, dashboards, and related capabilities within Army Vantage/Palantir Foundry
• Design, build, and sustain production-quality data pipelines and transformations using Python, SQL, PySpark, and native Foundry capabilities
• Develop and maintain Foundry ontology objects, relationships, and data models
• Integrate authoritative ATIS data with Vantage/Foundry and other Army enterprise data sources while maintaining data quality, lineage, and access controls
• Diagnose and resolve end-to-end data quality and integration issues across source data, interfaces, transformations, pipelines, and products
• Develop Foundry-native dashboards, applications, reports, and other data products
• Implement data-quality validation, monitoring, scheduling, error handling, and production reliability practices
• Apply version control, peer review, testing, documentation, and CI/CD where supported
• Collaborate with product owners, engineers, architects, analysts, and users to translate mission needs into technical solutions
• Document solutions, share knowledge, and build team capability
• Operate as a self-directed member of the Agile team and drive technical problems toward resolution

Data Engineer building Army ATIS data products in Palantir Foundry for LMI’s federal technology solutions

Data Engineer – Army Vantage, Palantir Foundry

Data Engineer building Army ATIS data products, pipelines, and dashboards in Palantir Foundry. Supporting LMI’s secure federal mission technology and U.S. Army analytics.

• Design and implement secure, next-generation data engineering solutions supporting critical missions
• Collaborate with technical leaders, business stakeholders, and multidisciplinary teams
• Support implementation of the DON MAHRS program Data Strategy and High-Level Total Force Data Hub implementation plan
• Develop, implement, integrate, and sustain data architectures, data hubs, data lakes, and data warehouses
• Design and develop data models, data structures, and data acquisition processes
• Develop engineering and implementation plans for data hubs, acquisition, modeling, integration, and related activities
• Research and evaluate data sources to identify authoritative and reliable sources for data hub development
• Plan, create, and maintain data architectures aligned with business requirements
• Design, develop, maintain, and optimize ETL/ELT pipelines and transformation processes
• Develop and maintain batch and streaming pipelines using Spark, Python, Databricks, Palantir Foundry, Kafka, and related tools
• Develop data acquisition processes to ingest, transform, validate, and integrate enterprise data
• Implement incremental loading, late-arriving data, processing-window, data-freshness, and lifecycle approaches
• Automate manual data processes and improve workflow efficiency, reliability, and scalability
• Develop, maintain, and optimize solutions within Databricks and enterprise data lakes
• Configure, monitor, manage, and document Databricks clusters according to DON policies and security requirements
• Develop and maintain Spark solutions using PySpark, Spark SQL, DataFrames, and Data Sets
• Implement and maintain Delta Lake and Delta Live Tables solutions
• Optimize data processing, storage, and query performance
• Support Palantir Foundry ontologies, ETL/ELT pipelines, applications, and data-driven user interfaces
• Integrate Palantir Foundry with enterprise data engineering and analytics environments
• Develop and maintain streaming solutions using Kafka, Kafka Streams, and ksqlDB
• Configure and manage Kafka topics and Schema Registry
• Develop Python data-processing applications and AWS Lambda functions
• Support processing with AWS S3, Kinesis, Lambda, and DynamoDB
• Monitor, troubleshoot, and optimize cloud-based processing solutions
• Develop data quality controls, validation processes, and data quality gates
• Develop and execute data-driven and unit testing for Spark, Python, and other processing solutions
• Establish data lifecycle policies covering retention, backup, recovery, and data management
• Monitor and troubleshoot pipelines and processing jobs
• Improve performance, reliability, scalability, and maintainability

Senior Data Engineer building secure cloud data platforms for IBR’s U.S. Navy mission programs

Senior Data Engineer

Senior Data Engineer building secure Databricks, AWS, Kafka, and Palantir data platforms for IBR’s U.S. Navy mission programs. Supporting enterprise data architecture, pipelines, quality, and integration.

• Provide data engineering expertise in developing, implementing, integrating, and sustaining data architectures, data hubs, data lakes, and data warehouse solutions
• Support design and development of data models, data structures, and data acquisition processes for the HC/HR Data Domain
• Develop engineering and implementation plans for data hubs, acquisition, modeling, integration, and related data engineering activities
• Research and evaluate enterprise data sources for authoritative and reliable data hub development
• Plan, create, and maintain data architectures aligned with business requirements
• Design, develop, maintain, and optimize ETL/ELT pipelines and data transformation processes
• Develop and maintain batch and streaming pipelines using Spark, Python, Databricks, Palantir Foundry, Kafka, and related tools
• Develop data acquisition processes to ingest, transform, validate, and integrate diverse enterprise data
• Implement incremental loading, late-arriving data, processing-window, data-freshness, and lifecycle strategies
• Automate manual processes and improve workflow efficiency, reliability, and scalability
• Develop, maintain, and optimize Databricks and enterprise data-lake solutions
• Configure, monitor, manage, and document Databricks clusters according to DON policies, standards, and security requirements
• Develop and maintain Spark-based processing solutions using PySpark, Spark SQL, Data Frames, and Data Sets
• Implement and maintain Delta Lake and Delta Live Tables solutions
• Optimize data processing, storage, and query performance
• Support Palantir Foundry ontologies, ETL/ELT pipelines, applications, user interfaces, and enterprise integrations
• Develop and maintain Kafka streaming solutions, topics, Schema Registry, Kafka Streams, and ksqlDB components
• Develop Python data-processing applications and AWS Lambda functions
• Support data integration and processing with AWS S3, Kinesis, Lambda, and DynamoDB
• Monitor, troubleshoot, and optimize cloud-based data-processing solutions
• Develop data-quality controls, validation processes, and quality gates
• Develop and execute data-driven and unit testing for Spark, Python, and other processing solutions
• Establish data lifecycle policies covering retention, backup, recovery, and data management
• Monitor and troubleshoot pipelines and processing jobs, resolving quality, performance, and integration issues
• Improve data-processing performance, reliability, scalability, and maintainability

Data Engineer building enterprise data platforms for Sparibis professional solutions firm

Data Engineer building enterprise data lakes, pipelines, and cloud solutions for Sparibis, a professional solutions firm. Supporting Databricks, Palantir Foundry, Kafka, Spark, and AWS environments.

• Develop Python data pipelines that build business data objects supporting applications
• Perform programming, project management, and development assignments, including complex systems work
• Identify and resolve complex application and technical problems
• Interpret design requirements for engineering implementation
• Build and deploy CI/CD pipelines
• Implement source-to-target mapping as pipeline code
• Perform unit and integration testing
• Maintain high software-quality standards through good team practices and habits
• Identify and encourage areas for team growth and improvement
• Perform other duties assigned by Caterpillar management

Data Engineer building Python pipelines and cloud data solutions for Caterpillar’s industrial technology business

Data Engineer building Python data pipelines, CI/CD workflows, and cloud data solutions for Caterpillar’s industrial technology business. Developing tested applications with AWS, databases, and modern software practices.

• Analyze, design, develop, and support SAP HANA solutions
• Lead the design and development of data products for fast-paced special projects
• Build reusable data-platform components and share architecture and best practices
• Partner with business stakeholders, analysts, and data scientists to translate strategic questions into data products and pipelines
• Gather reporting requirements and build BI solutions supporting business decisions and efficient operations
• Develop and enhance HANA Calculation views
• Use Business Objects and PowerBI to develop interactive data visualizations
• Validate HANA views and reports and resolve issues before production deployment
• Perform design, testing, and metadata documentation
• Develop and provide demos and end-user training
• Identify reporting gaps and opportunities to expand SAP HANA and S/4 Embedded Analytics solutions

Principal Data Engineer building SAP HANA reporting solutions for Northrop Grumman aerospace programs

Principal Data Engineer

Principal Data Engineer developing SAP HANA, Business Objects, and PowerBI reporting solutions for Northrop Grumman. Supporting executive-sponsored data initiatives and business decision-making.

• Develop Python data pipelines that build business data objects supporting applications
• Perform programming, project management, and development assignments without close supervision
• Handle complex systems work and application/technical problem identification and resolution
• Interpret design requirements for engineering implementation
• Build and deploy CI/CD pipelines
• Implement source-to-target mapping as pipeline code
• Perform unit and integration testing
• Maintain high standards of software quality by establishing good practices and habits
• Identify and encourage areas for team growth and improvement
• Perform other duties assigned by Caterpillar management

Data Engineer building Python data pipelines, CI/CD workflows, and AWS-backed data solutions for Caterpillar’s industrial technology business. Onsite in Chicago or Peoria.

• Act as a technical leader and Subject Matter Expert in data pipeline automation and workflow orchestration
• Design, implement, and maintain reliable data solutions using Airflow, dbt, and Google Cloud data products
• Manage a large portfolio of dbt models using macros and DRY patterns
• Advocate for test-driven development and assist QA in developing a reliable continuous integration and delivery process
• Monitor, troubleshoot, and optimize data pipeline and workflow performance
• Architect scalable solutions and reusable patterns
• Provide implementation, configuration, and deployment documentation
• Proactively address issues and implement innovative solutions
• Participate in agile ceremonies, including daily standups and sprint planning
• Mentor other engineers and foster technical excellence
• Stay current with industry trends and technologies
• Ensure data security, governance, compliance, and privacy standards
• Report to the Director of IT

Senior data engineer building automated pipelines for Wpromote, a digital marketing agency

Senior Software Engineer II, Data Pipelines

Senior data engineer building Airflow and dbt pipelines for Wpromote, a digital marketing agency. Optimizing cloud data systems, automation, governance, and delivery.

• Expand and optimize enterprise-wide data and data pipeline architectures
• Collect data from sources across the company
• Develop visualization and reporting strategies
• Implement data pipelines and acquire, transform, and optimize data sets
• Prepare data sets for consumption by internal and external customers
• Collaborate with developers, architects, analysts, data scientists, and business-facing users
• Ensure architectures and strategies remain consistent across projects
• Improve data architecture and visualization strategies for next-generation products and data initiatives
• Build stakeholder relationships and deliver valuable data solutions
• Design and implement process improvements focused on automation, data delivery optimization, self-service, and scalability
• Create and maintain data pipeline architectures
• Assemble large, complex data sets meeting business requirements
• Optimize data acquisition and transformation using SQL and big data technologies
• Enable enterprise-wide visualization and reporting
• Implement self-service and scalable visualization and reporting strategies
• Ensure data security across implementations, data centers, cloud providers, and client machines
• Mentor, train, and inspire team members
• Lead innovation activities and evaluate emerging technologies

Senior Data Engineer building enterprise data pipelines for Insperity’s HR solutions

Senior Data Engineer building Snowflake and dbt data pipelines for Insperity’s scalable HR solutions. Optimizing enterprise analytics, security, visualization, and Customer 360 data architecture.

• Define and maintain data engineering patterns, pipeline standards, and architectural guardrails
• Lead design of scalable data processing frameworks and transformation models for claims, encounters, risk adjustment, and member data
• Maintain accountability for platform reliability and data quality outcomes
• Apply machine learning and automation techniques in production data engineering environments
• Implement and continuously improve high-performance cloud-based data pipelines for real-time and batch workflows
• Serve as the technical subject matter expert for data engineering
• Mentor engineers through architecture reviews, design sessions, and internal documentation
• Evaluate emerging data technologies and make build-versus-buy recommendations
• Contribute to the multi-year data platform engineering roadmap
• Apply Medicare Advantage, CMS, HIPAA, HEDIS, and risk adjustment expertise to deliver audit-ready, compliant healthcare data solutions
• Provide informal technical leadership and subject matter guidance to Data Engineers and Senior Data Engineers
• Perform essential physical functions including talking, hearing, standing, walking, sitting, handling objects or controls, reaching, and occasionally lifting up to 10 pounds

Principal Data Engineer building compliant healthcare data platforms for Alignment Health

Principal Data Engineer setting standards and scaling cloud data platforms for Alignment Health’s Medicare Advantage healthcare ecosystem. Improving pipelines, data quality, compliance, and AI/ML integration.

• Lead end-to-end data requirements and stakeholder analysis
• Drive data quality improvement across claims, membership, member, and provider data
• Direct triage, analysis, and resolution of Data Ops issues affecting production environments and customer-facing outputs
• Serve as subject matter authority for EDI and healthcare data standards, including X12 834/837 transactions
• Evaluate internal and external data sources for quality, coverage, and suitability for analytical, operational, and regulatory use cases
• Apply knowledge of cloud data platforms, streaming architectures, automation, and data quality as a service
• Partner with data engineering to enable scalable, modern analytics infrastructure
• Build team capability through mentorship, coaching, and change leadership
• Interview, hire, train, assign, direct, and appraise employees; reward and discipline employees; address complaints and resolve problems

Director leading healthcare data engineering analysis for Alignment Health’s Medicare Advantage platform

Director, Data Engineering

Director leading data requirements, quality, and operations for Alignment Health’s Medicare Advantage platform. Guiding analytics teams and healthcare interoperability across claims, membership, and provider data.

• Supervise junior members of the data engineering team, including guiding, planning, and reviewing the team's work
• Create and maintain optimal data pipeline architecture
• Assemble large, complex data sets that meet functional and non-functional business requirements
• Extend the machine learning platform by designing tools that interface with cloud services and the existing codebase
• Build infrastructure for extracting, transforming, and loading data from a wide variety of sources using SQL, Python, and AWS
• Build analytics tools that provide actionable insights into key business performance metrics
• Support the needs of the analytics team
• Create data-handling tools for analytics and data science team members
• Interface with the Development team to create cross-team solutions
• Optimize or redesign the company’s data architecture for next-generation products and data initiatives
• Mentor and guide junior team members

Data Engineer building cloud data pipelines for NinjaHoldings’ digital finance products

Data Engineer building SQL, Python, and AWS pipelines for NinjaHoldings’ digital banking, lending, and analytics products. Extending machine-learning infrastructure and mentoring junior engineers.

• Design, develop, maintain, and enhance business applications that support project delivery and corporate operations.
• Modernize and support Excel/VBA-based tools through integration with SQL databases, Python applications, APIs, and other technologies.
• Develop and maintain database solutions, ETL processes, and system integrations that improve data quality and operational efficiency.
• Partner with business users to gather requirements, analyze workflows, and deliver effective technical solutions.
• Create and support reporting, dashboard, and data-management solutions.
• Perform testing, troubleshooting, and quality assurance activities to ensure reliable system performance.
• Develop and maintain technical documentation and end-user support materials.
• Apply secure software development practices and contribute to continuous improvement initiatives.

Construction Data Engineer modernizing data systems for a construction management consultancy

Construction Data Engineer

Construction Data Engineer modernizing Excel/VBA, SQL, Python, and ETL systems for a construction management consultancy. Improving automation, reporting, integrations, and data-driven project operations.

• Lead architecture and control frameworks for PNC’s on-premises D&A technology ecosystem
• Design data movement controls, reconciliation pipelines, and semantic abstraction layers across relational warehouses, legacy Hadoop, and on-premises Lakehouse environments
• Define standards for high availability, enterprise security, and audited data movement
• Architect automated enterprise-wide data validation, row-count verification, checksum validation, threshold alerting, and schema-evolution tracking
• Define standards bridging Control-M batch/legacy workflows with Apache Airflow DAG orchestration
• Architect and maintain Apache Polaris as a centralized REST catalog for Apache Iceberg tables
• Enforce multi-engine catalog governance, role-based access control, and credential scoping
• Implement and tune Trino across bare-metal or private Kubernetes clusters to federate queries across relational databases, Hadoop, and Iceberg storage
• Architect graph database integrations for entity relationships, data lineage, and semantic ontologies
• Maintain operational metadata, structural tracking, and end-to-end data lineage for regulatory compliance
• Provide thought leadership for control architecture evolution within the DnA Data Ops ecosystem
• Drive strategic application architecture roadmaps and prioritize architectural and technical work
• Partner with Product Owners to prioritize and deliver business priorities through monthly releases
• Participate in business requirements gathering and understand client requirements
• Create, present, and obtain signoff for design documents
• Support enterprise architecture and perform proofs of concept for new requirements
• Ensure compliance with architecture and technology standards and manage exceptions and vulnerabilities
• Design, develop, and optimize scalable data warehouses and ETL architecture for BI applications
• Analyze business intelligence data and recommend data warehouse growth and integration
• Capture requirements with IT and business customers for data warehouse architecture
• Provide technical guidance and ensure solutions align with architecture frameworks
• Lead and consult on data warehouse implementation and integration while ensuring quality and integrity standards

Data Architect designing on-premises Hadoop, Lakehouse, ETL, and AI data platforms for PNC

Data Architect – Hadoop, ML, AI, ETL

Data Architect designing PNC’s on-premises Hadoop, Lakehouse, ETL, and AI data platforms. Leading governance, lineage, reconciliation, and enterprise data warehouse architecture.

• Own the strategy, architecture, and roadmap for the Policy and Data Platform
• Build and scale a multi-team engineering organization
• Develop managers and senior engineers
• Translate customer security policy needs into durable technical systems
• Partner with product and security teams to align platform capabilities with customer needs
• Set engineering standards and operational practices that improve platform reliability
• Build security policy systems that protect customers' environments
• Enable teams across Cisco to integrate with the platform
• Strengthen signals used to recommend new policies and expand enforcement points

Engineering leader scaling Cisco’s security policy and data platform

Senior Director of Engineering – Policy and Data Platform

Engineering leader owning Cisco’s security policy and data platform strategy. Building multi-team organizations and reliable systems protecting customer environments.

• Compose, build, and maintain scalable data pipelines extracting data from casino source systems
• Apply complex transformations and load optimized datasets for analytics and downstream applications
• Write production-quality Python and Node.js code
• Develop and maintain data models representing business domains and supporting analytical use cases
• Use ETL orchestration tools to schedule, monitor, and manage production workflows
• Optimize queries and data processing logic for large volumes of transactional data
• Ensure data integrity through validation, testing, and observability practices
• Investigate and resolve data quality issues, pipeline failures, and performance bottlenecks
• Collaborate with data scientists, analysts, and product teams to understand requirements and deliver solutions
• Document pipelines, data models, and technical decisions

Data Engineer building scalable casino data pipelines for Aristocrat’s entertainment and gaming business

Data Engineer building scalable pipelines and data models from casino systems. Supporting Aristocrat’s mobile and casino games through reliable analytics data infrastructure.

• Collaborate across Agile teams to design, develop, test, implement, and support technical solutions using full-stack development tools and technologies
• Lead a team of developers experienced in machine learning, distributed microservices, and full-stack systems
• Use Java, Scala, Python, open-source RDBMS and NoSQL databases, and cloud-based data warehousing services such as Redshift and Snowflake
• Stay current with technology trends, experiment with and learn new technologies, participate in internal and external technology communities, and mentor engineering community members
• Collaborate with digital product managers to deliver robust cloud-based solutions that create powerful experiences for millions of Americans
• Perform unit tests and conduct code reviews to ensure rigorously designed, elegant, and performance-tuned code
• Build scalable platforms delivering omnichannel messages across owned and paid Adtech channels

Senior Lead Data Engineer building cloud data platforms for Capital One’s personalized marketing systems

Senior Lead Data Engineer – Java, Python, Scala, AWS

Senior Lead Data Engineer building Java, Python, and Scala cloud data platforms for Capital One’s personalized marketing systems. Leading developers and delivering scalable omnichannel messaging technology.

• Lead and manage a team of data engineers delivering enterprise-grade data products, data marts, and AI-ready data assets
• Lead the strategic design and delivery of scalable data pipelines and architectures supporting commerce analytics, machine learning, and AI workloads
• Partner with Ecommerce, Product, Growth/Marketing, Finance, and Data Science stakeholders to enable AI use cases, feature stores, and model-ready commerce datasets
• Champion data quality, completeness, consistency, lineage, customer identity resolution, and trusted order/product/customer domains
• Lead development and implementation of semantic layers standardizing business definitions
• Structure and document enterprise data to support LLMs, knowledge graphs, personalization, forecasting, and downstream AI applications
• Own platform reliability, including SLAs, monitoring, observability, and incident management for ecommerce analytics and AI data pipelines
• Lead implementation and enforcement of data contracts and schema governance
• Lead adoption of lakehouse, real-time streaming, batch processing, feature stores, data monitoring, and observability patterns
• Guide platform optimization for performance, scalability, freshness, and cost
• Collaborate with governance teams to ensure data is discoverable, explainable, and compliant
• Mentor and develop engineering talent in data engineering and AI data readiness principles

Senior data engineering leader building AI-ready commerce platforms for Universal Music Group

Senior Manager, Data Engineering

Senior data engineering leader building AI-ready commerce data platforms for Universal Music Group. Leading pipelines, semantic layers, governance, and ML/AI data products.

• Develop technical enablement content, including presentations, solution guides, white papers, workshops, labs, and training materials
• Design, build, and maintain product demonstrations showcasing customer challenges, technical capabilities, and business outcomes
• Support field sales, systems engineering, and channel teams with technical expertise and customer-facing content
• Deliver presentations at customer meetings, internal enablement sessions, partner events, and industry conferences
• Collaborate with Product Management, Engineering, Marketing, and regional field organizations on product launches and strategic initiatives
• Stay current on emerging technologies and trends in data center networking, automation, AI, and cloud infrastructure
• Translate complex technical concepts into clear messaging for technical and executive audiences
• Help customers, partners, and field teams position and adopt HPE Apstra

Technical marketing engineer advancing HPE Apstra data center automation adoption across U.S. customers

Senior Technical Marketing Engineer – Data Center Networking

Technical marketing engineer creating demos, content, and enablement for HPE Apstra data center automation. Supporting customers, partners, sales, and field teams across the U.S.

• Develop technical enablement content, including presentations, solution guides, white papers, workshops, labs, and training materials
• Design, build, and maintain product demonstrations showcasing customer challenges, technical capabilities, and business outcomes
• Support field sales, systems engineering, and channel teams with technical expertise and customer-facing content
• Deliver presentations at customer meetings, internal enablement sessions, partner events, and industry conferences
• Collaborate with Product Management, Engineering, Marketing, and regional field organizations on product launches and strategic initiatives
• Stay current on emerging technologies and trends in data center networking, automation, AI, and cloud infrastructure
• Translate complex technical concepts into messaging for technical and executive audiences
• Help shape product positioning and technical storytelling
• Influence product adoption, customer success, and automated data center operations

Technical Marketing Engineer promoting HPE Apstra data center automation across U.S. markets

Senior Technical Marketing Engineer promoting HPE Apstra, HPE’s data center automation platform. Creating technical content, demonstrations, presentations, and field enablement for U.S. customers and partners.

• Develop technical enablement content, including presentations, solution guides, white papers, workshops, labs, and training materials
• Design, build, and maintain product demonstrations showcasing customer challenges, technical capabilities, and business outcomes
• Support field sales, systems engineering, and channel teams with technical expertise and customer-facing content
• Deliver presentations at customer meetings, internal enablement sessions, partner events, and industry conferences
• Collaborate with Product Management, Engineering, Marketing, and regional field organizations on product launches and strategic initiatives
• Stay current on emerging technologies and trends in data center networking, automation, AI, and cloud infrastructure
• Translate complex technical concepts into clear messaging for technical and executive audiences
• Help customers, partners, and field teams position and adopt HPE Apstra
• Influence product adoption, customer success, and the future of automated data center operations

Technical marketing engineer advancing HPE Apstra data center automation across U.S. customers

Senior Technical Marketing Engineer – Data Center Networking, Apstra

Technical marketing engineer creating demos and enablement content for HPE Apstra, HPE’s intent-based data center automation platform. Supporting sales, partners, and customers remotely across the United States.

Data Engineer jobs in United States

See all jobs on JobTailor

Data Engineer – Army Vantage, Palantir Foundry

Senior Data Engineer

Data Engineer

Data Engineer

Principal Data Engineer

Data Engineer

Senior Software Engineer II, Data Pipelines

Senior Data Engineer

Principal Data Engineer

Director, Data Engineering

Data Engineer

Construction Data Engineer

Data Architect – Hadoop, ML, AI, ETL

Senior Director of Engineering – Policy and Data Platform

Data Engineer

Senior Lead Data Engineer – Java, Python, Scala, AWS

Senior Manager, Data Engineering

Senior Technical Marketing Engineer – Data Center Networking

Senior Technical Marketing Engineer – Data Center Networking

Senior Technical Marketing Engineer – Data Center Networking, Apstra