Lead Databricks Architect
Nimble Gravity
full-time
Posted on:
Location Type: Remote
Location: United States
Visit company websiteExplore more
Job Level
About the role
- Act as the senior technical authority for enterprise data architecture across client engagements.
- Design and evolve cloud-native, best-of-breed data platforms for our clients, supporting analytics, reporting, automation, and AI at enterprise scale, with a strong focus on Databricks as a core platform.
- Define target-state architectures for data ingestion, transformation, storage, semantic layers, and consumption across federated domains.
- Establish and enforce architectural standards for scalability, performance, security, resiliency, and cost efficiency, including platform-level governance and operational best practices.
- Architect and guide the implementation of modern data pipelines using tools such as Databricks and dbt.
- Design ELT and streaming patterns that are testable, observable, and resilient.
- Define data modeling standards (analytical, dimensional, and semantic) that enable trusted self-service analytics.
- Partner with client data engineering teams to ensure consistent application of patterns and reuse of shared platform capabilities, including effective use and management of Databricks environments.
- Design AI-ready data foundations, including semantic layers, contextual metadata, and reusable data contracts.
- Enable machine learning and generative AI use cases by ensuring data is well-modeled, discoverable, governed, and explainable.
- Collaborate with client Data Science and Analytics teams to support feature stores, experimentation environments, and reusable data products.
- Partner with client Data Governance stakeholders to define standards for data quality, metadata management, lineage, and stewardship.
- Ensure architectural designs support regulatory and compliance requirements (e.g., SOX, SEC, FINRA, GDPR, DORA).
- Promote reuse, interoperability, and consistent definitions of critical enterprise data across client environments, including alignment with platform governance models.
- Own and administer Databricks workspaces across client environments, including configuration, access controls, and governance (Unity Catalog, IAM, RBAC).
- Manage cluster configurations, job orchestration, and cost optimization to ensure efficient and reliable platform operations.
- Monitor platform performance, troubleshoot issues, and ensure SLAs for data pipelines and workloads.
- Establish best practices for CI/CD, environment management, and operational support within Databricks.
- Partner with client teams to operationalize and scale Databricks as a core enterprise data platform.
Requirements
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or related fields.
- Cloud or data architecture certifications (AWS, data management, analytics) preferred.
- 10+ years of experience in data architecture, data engineering, or enterprise analytics.
- Proven experience designing and operating modern, cloud-native data platforms.
- Hands-on expertise with AWS-based data ecosystems and modern analytics stacks.
- Strong experience building data platforms that support AI, machine learning, and advanced analytics.
- Experience operating in federated data models and complex enterprise environments.
- Strong hands-on experience with Databricks, including workspace setup, configuration, administration, and platform optimization; experience with AWS setup and administration is strongly preferred.
Benefits
- Nimble Gravity is an Equal Opportunity Employer and considers applicants for employment without regard to race, color, religion, sex, orientation, national origin, age, disability, genetics or any other basis forbidden under federal, state, or local law.
- This position is open only to candidates who are legally authorized to work in the United States. We do not provide visa sponsorship for this role.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data architecturedata engineeringcloud-native data platformsdata ingestiondata transformationdata modelingELT patternsstreaming patternsmachine learninggenerative AI
Soft Skills
collaborationleadershipcommunicationproblem-solvingorganizational skills
Certifications
Bachelor's degreeMaster's degreeAWS certificationdata management certificationanalytics certification