Senior Core Platform Engineer
Trumid
AWSAzureCloudDockerGoogle Cloud PlatformJenkinsKubernetesTerraform
Lead Data Platform Engineer
LivantaLLC
full-time
Posted on:
Location Type: Remote
Location: Remote • Maryland, North Carolina, Virginia • 🇺🇸 United States
Visit company websiteJob Level
Senior
Tech Stack
AirflowAWSSparkUnity
About the role
- Design, build, and operate Commence’s enterprise data platform.
- Establish standards, pipelines, and controls for analytics, AI/ML, and compliance use cases.
- Administer and manage Databricks workspaces, clusters, and security configuration.
- Design and implement scalable ingestion pipelines (batch and streaming) into Delta Lake.
- Establish CI/CD practices for data pipelines and implement cost monitoring/optimization.
- Ensure alignment with FedRAMP, HIPAA, and FISMA security and compliance requirements.
- Partner with DevOps for infrastructure-as-code and automation practices.
- Collaborate with stakeholders to identify platform gaps and design long-term solutions.
- Provide technical leadership and management of data engineers.
- Collaborate with Analytics and Data Science leads to deliver trusted, reusable data assets.
- Automate infrastructure and monitor key systems.
Requirements
- 7+ years of experience in data engineering, platform architecture, or related roles.
- Hands-on expertise with using and administering Databricks, Spark, Delta Lake, and orchestration tools (Airflow, Databricks Workflows, etc.).
- Demonstrated experience in implementing Data Lakes/ Lakehouse architecture using Databricks and AWS technologies.
- Familiarity with modern data frameworks like Data Fabric/ Mesh is a plus.
- Strong knowledge of AWS services (incl. GovCloud), IAM, VPC, and network security.
- Experience with Unity Catalog or equivalent governance/cataloging tools.
- Familiarity with regulated environments (HIPAA, FedRAMP, FISMA).
- Demonstrated ability to design secure, cost-effective, and scalable data platforms.
- Strong problem-solving skills, attention to detail, and a passion for continuous learning.
- Experience working in a healthcare or regulated environment.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data engineeringplatform architectureDatabricksSparkDelta LakeCI/CDdata pipelinesinfrastructure-as-codeautomationdata lakes
Soft skills
technical leadershipproblem-solvingattention to detailcollaborationcontinuous learning
