Interface Systems

Data Engineer - Senior Manager

Interface Systems

full-time

Posted on:

Origin:  • 🇺🇸 United States

Visit company website
✨ AI Apply
Manual Apply

Salary

💰 $130,000 - $150,000 per year

Job Level

Senior

Tech Stack

ApacheAzureCloudDockerETLKubernetesPySparkPythonScalaSparkSQLTerraform

About the role

  • Data Pipeline Development: Build and maintain efficient, secure, and reliable ELT pipelines using Azure Synapse Analytics and Apache Spark.
  • Data Modeling and Warehousing: Develop and optimize data models (e.g., star schemas, snowflake schemas) for data warehousing and analytics, ensuring performance, scalability, and maintainability within Azure Synapse analytics.
  • Data Governance and Security: Implement data governance frameworks, including data quality, lineage, and cataloging, using Azure Purview or similar tools. Ensure compliance with security standards and data privacy regulations (e.g., GDPR, CCPA).
  • Dev Ops: Take ownership of existing Azure Dev Ops pipelines , source code repositories and branching strategies and produce new artefacts when required . Be responsible for managing all Azure EDW resources using infrastructure as code tooling .
  • Performance Tuning : Monitor and optimize the performance of Azure Synapse Analytics, Data Lake , Spark jobs and on-premise SQL Servers including query optimization, partitioning, and indexing strategies.
  • Integration and Collaboration : Collaborate with cross-functional teams (e.g., business analysts, report and application developers) to integrate Azure data solutions with other systems, including APIs, third-party tools, and business applications. Provide guidance and mentorship for other less experienced team members.
  • Innovation and Strategy : Stay abreast of emerging Azure data technologies, Apache Spark advancements, and industry trends, recommending and implementing innovative solutions to enhance the data platform.

Requirements

  • Bachelor's degree in computer science, Data Science, Information Technology, or a related field (or equivalent work experience).
  • 7+ years of experience in data architecture, data engineering, or related roles, with at least 3 years focused on Azure cloud platforms.
  • Extensive hands-on experience with Azure Synapse Analytics, Azure Data Lake (Gen2), SQL Server and Apache Spark.
  • Proven track record of designing and implementing large-scale data solutions in a cloud environment.
  • Strong background in Dev Ops culture with a deep understanding of source control and infrastructure as code.
  • Microsoft certifications in Azure data technologies (e.g., Microsoft Certified: Azure Data Engineer Associate, Azure Solutions Architect Expert).
  • Familiarity with Power BI or other visualization tools for reporting and dashboard development.
  • Experience with containerization and orchestration tools like Docker and Kubernetes in Azure (e.g., AKS).