Lead BI Engineer

CRG Solutions

Lead BI Engineer responsible for driving data strategy and building BI solutions at CRG Solutions. Collaborating with cross-functional stakeholders to deliver actionable insights.

Posted 7/10/2026full-timeRemote • 🇺🇸 United StatesSeniorWebsite

Tech Stack

Tools & technologies

ApacheAWSAzureCloudGoogle Cloud PlatformKafkaPythonSparkSQL

About the role

Key responsibilities & impact

Architect, build, and optimize distributed data pipelines using Apache Spark in a high‑volume, mission‑critical environment.
Design and maintain enterprise Lakehouse architecture with Delta Lake, ensuring ACID compliance, lineage, auditability, and data governance.
Develop automated ingestion frameworks (batch, streaming, and event‑driven) across multiple cloud services and integration points.
Enable machine‑learning workflows by preparing feature‑ready datasets and establishing reproducible ML deployment patterns.
Lead platform‑wide data quality, access control, and cataloging frameworks.
Implement advanced cost‑optimization, cluster tuning, and performance engineering strategies.
Collaborate with Finance, BI, Operations, and ML teams to translate complex business needs into scalable data solutions.
Own production reliability, troubleshooting, and root‑cause analysis for data and ML pipelines.

Requirements

What you’ll need

7+ years of experience in advanced data engineering with distributed compute technologies.
Expert‑level Spark engineering (performance tuning, cluster configuration, partition strategies, optimization of large datasets).
Hands‑on experience with Lakehouse architectures including ACID transactions, schema evolution, and governance frameworks.
Deep proficiency in Python and SQL for large‑scale data transformation.
Experience supporting machine‑learning pipelines or model operationalization.
Proven experience architecting cloud‑native data platforms (Azure, AWS, or GCP).
Strong background integrating diverse, complex data sources at enterprise scale.
Demonstrated ability to own mission‑critical production systems.
Experience with distributed streaming frameworks (Kafka, Event Hubs, or similar).
Experience building or supporting ML platforms, feature stores, or experiment‑tracking systems.
Background in data security, compliance controls, or audit‑ready governance.
Experience automating data operations with CI/CD and infrastructure‑as‑code.

Benefits

Comp & perks

Flexible work arrangements

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Data Pipeline ArchitecturePerformance TuningData TransformationMachine Learning SupportCI/CD AutomationInfrastructure-as-CodeACID ComplianceData GovernanceCluster ConfigurationPartition Strategies