Lead Data Engineer
Centene Corporation
SQL
Senior Data Engineer – Databricks
Centene Corporation
full-time
Posted on:
Location Type: Remote
Location: Remote • Florida, Montana, New York, Washington • 🇺🇸 United States
Visit company websiteSalary
💰 $85,300 - $158,100 per year
Job Level
Senior
Tech Stack
AWSAzureCloudGoogle Cloud PlatformPySparkPythonSQLUnity
About the role
- Develops and operationalizes data pipelines to make data available for consumption (reports and advanced analytics)
- Engage with the DevSecOps Engineer during continuous integration and continuous deployment
- Provides guidance to data engineers in the design, development, implementation, testing, documenting, and operating of large-scale, high-volume, high-performance data structures for business intelligence analytics
- Designs, develops, and maintains real-time processing applications and real-time data pipelines
- Ensure quality of technical solutions as data moves across Centene’s environments
- Develops, constructs, tests, and maintains architectures using advanced programming language and tools
- Drives ways to improve data reliability, efficiency, and quality and deploys a solution; use data to discover tasks that can be automated
- Performs other duties as assigned
- Complies with all policies and standards
Requirements
- A Bachelor's degree in a quantitative or business field (e.g., statistics, mathematics, engineering, computer science)
- Requires 4 – 6 years of related experience.
- Experienced in implementations using Databricks Lakehouse Medallion architecture and Unity Catalog (for data governance, lineage tracking, and access control)
- Skilled in building and optimizing complex data pipelines using PySpark, SQL and/or Python on Databricks (or other comparable modern cloud DBMS preferably on AWS)
- Proficient in handling relational, flat files and hierarchical (JSON, XML) data formats on cloud platforms (AWS, Azure, GCP)
- Skilled in deploying production-grade pipelines with Databricks workflows (leveraging CI/CD – Gitlab, DAB etc.)
- Experienced in data pipeline optimization strategies, development of automated monitoring
- Familiarity with Databricks API integrations and Connector tools for BI platforms
Benefits
- competitive pay
- health insurance
- 401K and stock purchase plans
- tuition reimbursement
- paid time off plus holidays
- flexible approach to work with remote, hybrid, field or office work schedules
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data pipelinesdata structuresreal-time processing applicationsPySparkSQLPythondata pipeline optimizationautomated monitoringdata governancedata lineage tracking
Soft skills
guidancecollaborationproblem-solvingquality assuranceefficiency improvement
