Principal Data Engineer

Medical Guardian

Principal Data Engineer building and leading data engineering foundation for Medical Guardian. Focusing on real-time decisioning, operational applications, analytics, and data services.

Posted 7/6/2026full-timePhiladelphia • Pennsylvania • 🇺🇸 United StatesLeadWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and operating production-grade data pipelines and platforms using Azure and Databricks, with a strong focus on real-time streaming and IoT applications. Proven ability to lead and mentor data engineering teams while ensuring high standards of data quality, observability, and operational excellence.

Highest-signal resume keywords

Data Pipeline Design and OptimizationAzure Cloud ServicesDatabricks and Delta LakeReal-Time Streaming SolutionsTeam Leadership and Mentoring

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonPySparkSpark SQLSQLETL/ELT WorkflowsData QualityObservabilityCI/CDProduction OperationsData Architecture

Soft Skills

Ownership MindsetTechnical CommunicationProblem-SolvingMentoringCollaboration

Tools & Technologies

Azure Event HubsAzure Stream AnalyticsDatabricks WorkflowsMicroservicesAPIs

Industry Keywords

Medallion ArchitectureIoTTelemetryStreaming DataEvent-Driven Pipelines

Tech Stack

Tools & technologies

AzureCloudDistributed SystemsETLIoTMicroservicesPySparkPythonSparkSQL

About the role

Key responsibilities & impact

Design, build, optimize, and operate production-grade batch and streaming data pipelines on Azure and Databricks, with a primary focus on real-time IoT and telemetry use cases within a Medallion architecture.
Develop ETL/ELT workflows to ingest, transform, validate, and serve large volumes of structured, semi-structured, unstructured, and streaming data.
Build and maintain reliable data products, data services, APIs, and microservices that support operational applications, analytics, software engineering, and ML/AI teams.
Use Python, PySpark, Spark SQL, SQL, Delta Lake, Databricks Workflows, CI/CD, and related tools to build maintainable, testable, and observable data systems.
Troubleshoot complex production pipeline issues across Databricks, Azure, streaming systems, APIs, and source systems, including root cause analysis, corrective action, and prevention planning.
Move quickly from rough business need to prototype, pilot, and production-ready data capability while maintaining appropriate engineering discipline.
Lead the design and delivery of real-time streaming ingestion and processing patterns for connected medical device telemetry, event data, and operational data feeds.
Implement streaming solutions using Azure Event Hubs, Azure Stream Analytics, Databricks, Delta Lake, and related Azure integration patterns.
Define reliability, latency, quality, observability, and supportability expectations for production streaming systems.
Manage, mentor, and develop data engineers, providing clear expectations, technical guidance, prioritization support, feedback, and accountability.

Requirements

What you’ll need

10+ years of professional experience in data engineering, software engineering, data platform engineering, distributed systems, analytics engineering, or related technical fields.
7+ years of hands-on experience designing, building, optimizing, and operating production data pipelines, data platforms, or data services.
5+ years of hands-on experience with modern cloud data platforms, including Databricks, Spark, Delta Lake, SQL, Python/PySpark, and production pipeline orchestration.
3+ years of experience leading, managing, mentoring, or providing technical direction to data engineers or related technical teams.
Strong experience with Azure cloud services for data engineering, streaming, integration, storage, security, and production operations.
Experience designing and operating real-time streaming, event-driven, or near-real-time data pipelines in production or business-critical environments.
Experience applying DevOps, CI/CD, testing, version control, code review, documentation, and automation practices to data engineering workloads.
Experience building data services, APIs, microservices, or reusable consumption patterns for downstream applications, analytics, ML/AI, or operational workflows.
Strong understanding of data quality, observability, monitoring, lineage, reliability, cost optimization, privacy, and production support for data systems.
Experience translating ambiguous business needs into technical designs, architecture recommendations, delivery plans, and measurable outcomes.
Ability to explain data architecture, pipeline behavior, tradeoffs, assumptions, risks, and limitations to both technical and non-technical stakeholders.
Strong ownership mindset and ability to drive work forward independently in a fast-moving, evolving environment.

Benefits

Comp & perks

Health Care Plan (Medical, Dental & Vision)
Paid Time Off (Vacation, Sick Time Off & Holidays)
Company Paid Short Term Disability and Life Insurance
Retirement Plan (401k) with Company Match