Data Engineer, Databricks

Horizon Industries, Limited

Databricks Data Engineer leading data pipeline implementations for clients at Horizon Industries, a management consulting firm. Collaborating with a dynamic team while focusing on performance and reliability in data engineering.

Posted 6/24/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSeniorWebsite

Tech Stack

Tools & technologies

Amazon RedshiftAWSAzureCloudDynamoDBEC2ETLGoGoogle Cloud PlatformHadoopKafkaNoSQLPythonScalaSparkSQL

About the role

Key responsibilities & impact

Build end-to-end implementation of multiple ETL/ELT pipelines, demonstrating efficient data transformation and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability
Assess and understand the ETL jobs, workflows, BI tools, and reports
Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products
Experience in crafting database / data warehouse solutions in the cloud (Preferably AWS, Azure, Alternatively GCP)
Key must have skill sets – Python, SQL, Databricks, AWS Data Services
Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores
Experience manipulating, processing, and extracting value from large, disconnected datasets
Experience manipulating structured and unstructured data for analysis
Experience with data modeling tools and processes
Experience aggregating and transforming data from multiple datasets to create data products
Support an Agile software development lifecycle

Requirements

What you’ll need

Ability to hold a position of public trust with the US government.
S. in Computer Science or equivalent
4+ years' experience in data engineering and big data. At least 2 years' of professional services experience interacting directly with clients.
Big data tools: Hadoop, Spark, Kafka, etc.
Relational SQL and NoSQL databases and experience working with relational databases
AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena
Data Platforms: Databricks
Data streaming systems: Batch,Kafka, Storm, Spark-Streaming, etc.
Languages: Python, R, Scala, Go
Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
Extensive knowledge of data warehousing concepts and hands-on experience deploying pipelines using Databricks **a must
Data modeling and database design skills and knowledge of version control
Excellent verbal and written communication skills
Experience architecting scalable and fault-tolerant data solutions across Azure, AWS, and Databricks
Databricks Data Engineer Professional certification a plus.
Preference for candidates with Databricks Professional certifications

Benefits

Comp & perks

A comprehensive benefits package including healthcare (medical, dental, vision and disability)
a 401k program where you are 100% vested from day one with an employer match after 90 days.
an Educational Assistance program.
a Student Loan Repayment Program
Gym Reimbursement Program.
Paid Time off
Dynamics, passionate, multi-disciplinary team of creative minds to work with, and many more.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonSQLDatabricksHadoopSparkKafkaAWS Data Servicesdata modelingdata warehousingbig data

Soft Skills

communicationclient interactionproblem-solvingteam collaborationadaptabilityanalytical thinkingattention to detailorganizational skillspublic trustAgile methodology

Certifications

Databricks Data Engineer Professional