Senior Data Engineer
Abacus Insights
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇺🇸 United States
Visit company websiteJob Level
Senior
Tech Stack
AWSCloudETLPySparkPythonSQLTerraform
About the role
- Partner with clients and implementation teams to understand data distribution requirements.
- Design and develop data pipelines integrating with Databricks and Snowflake, ensuring accuracy and integrity.
- Lead architecture and implementation of solutions for health plan clients, optimizing cloud-based technologies.
- Translate business requirements into technical solutions in collaboration with cross-functional teams.
- Implement robust data security measures and ensure compliance with healthcare regulations such as HIPAA.
- Provide technical leadership, mentor junior engineers, and stay current with industry trends.
- Optimize data processes for efficiency, review architectures regularly, and maintain detailed documentation.
- Conduct training sessions for team members and clients to enhance understanding and utilization of data solutions.
Requirements
- 5+ years of experience in data engineering.
- Bachelor’s degree in Computer Science, Computer Engineering, or related IT field.
- Basic knowledge of U.S. healthcare data (e.g., medical claims, member eligibility).
- Strong communication skills with both technical and non-technical stakeholders.
- Proficiency in Python and SQL.
- Experience building and managing ETL pipelines using Databricks workflows or similar orchestration frameworks.
- Familiarity with structured and semi-structured data, and processing using PySpark.
- Working knowledge of AWS services (S3, SQS) or equivalent cloud services.
- Experience with Terraform and GitLab CI/CD.
- Skills in query tuning and performance optimization in SQL and/or SparkSQL.
- Familiarity with data warehousing solutions such as Snowflake or similar platforms.
Benefits
- Unlimited paid time off – recharge when you need it
- Work from anywhere – flexibility to fit your life
- Comprehensive health coverage – multiple plan options to choose from
- Equity for every employee – share in our success
- Growth-focused environment – your development matters here
- Home office setup allowance – one-time support to get you started
- Monthly cell phone allowance – stay connected with ease
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data engineeringPythonSQLETL pipelinesDatabricksPySparkAWSTerraformGitLab CI/CDquery tuning
Soft skills
communicationtechnical leadershipmentoringcollaborationtraining