Solutions Architect – Databricks

EXL

Solutions Architect designing and building scalable data platforms with Databricks Lakehouse. Leading client engagements and driving Databricks adoption across enterprise data ecosystems.

Posted 5/27/2026full-timeNew York City • New York • 🇺🇸 United StatesMid-LevelSenior💰 $155,000 per yearWebsite

Tech Stack

Tools & technologies

AirflowCloudDistributed SystemsETLJavaKafkaPySparkPythonScalaSparkSQL

About the role

Key responsibilities & impact

Partner with client stakeholders to define data platform strategy and establish Databricks Lakehouse architecture as the standard.
Design scalable, secure, and high-performance data architectures across batch and real-time processing.
Build and present reference architectures, solution blueprints, and demos to drive adoption and technical buy-in.
Translate complex business requirements into robust technical data solutions.
Design and implement scalable data pipelines using Databricks (PySpark, SQL).
Build and optimize data models, data marts, and medallion architecture layers (Bronze/Silver/Gold).
Develop and manage ETL/ELT pipelines, including CDC, incremental processing, and performance tuning.
Ensure data quality, observability, monitoring, and alerting across production workloads.
Work with large-scale distributed data systems (Spark, Kafka, etc.).
Lead and mentor a team of data engineers across multiple workstreams.
Conduct code reviews, enforce best practices, and create reusable frameworks/patterns.
Drive end-to-end solution delivery, including architecture, development, and production deployment.
Collaborate with cross-functional teams including analytics, BI, data science, and cloud engineering.
Manage stakeholder communication and provide technical thought leadership.

Requirements

What you’ll need

6-10 years of experience in data engineering, data architecture, or analytics engineering
Strong experience with Databricks ecosystem (Spark, SQL, PySpark, workflows)
Expertise in Big Data Technologies (Spark, Kafka, distributed systems)
Data Warehousing & Modeling (OLTP/OLAP, dimensional modeling, medallion architecture)
ETL/ELT pipeline development and orchestration (Airflow or similar tools)
Advanced proficiency in SQL and Python (Scala/Java/R is a plus)
Experience designing and delivering enterprise-grade data architectures
Strong understanding of performance tuning, query optimization, and large-scale data processing
Proven ability to translate business needs into scalable technical solutions
Experience leading teams and working in client-facing environments
Excellent communication, problem-solving, and stakeholder management skills

Benefits

Comp & perks

Health insurance
Retirement plans
Paid time off
Flexible work arrangements
Professional development
Bonuses
Performance incentives

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

DatabricksPySparkSQLSparkKafkaETLELTdata modelingperformance tuningdata architecture

Soft Skills

communicationproblem-solvingstakeholder managementleadershipmentoringcollaborationtechnical thought leadership