Senior Data Engineer, Databricks

Avanade

Databricks Data Engineer developing Azure data pipelines and PySpark transformations for Avanade’s banking and analytics initiatives. Collaborating with Data Scientists on secure, governed datasets and optimizing Spark workloads.

Posted 8/17/2026full-timeBrisbane • 🇦🇺 AustraliaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and optimizing data ingestion pipelines using Azure Data Factory and Databricks, with a strong focus on data governance and compliance in financial services. Proficient in data processing with PySpark and Python, ensuring high-quality data delivery and collaboration with cross-functional teams.

Highest-signal resume keywords

Azure DatabricksApache SparkData GovernanceETL/ELT ProcessesCI/CD Pipelines

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Ingestion PipelinesPySparkAzure Data FactorySQLNoSQLData ModelingData WarehousingPythonScalaTerraform

Soft Skills

Analytical SkillsProblem-Solving SkillsCommunication SkillsStakeholder ManagementCollaboration

Tools & Technologies

Azure SynapseAzure Blob StorageDelta LakeAzure DevOpsGit

Certifications & Qualifications

Databricks Certified Data Engineer AssociateDatabricks Certified Data Engineer Professional

Industry Keywords

Data GovernanceData SecurityComplianceBanking RegulationsFinancial Services

Tech Stack

Tools & technologies

ApacheAzureETLNoSQLPySparkPythonScalaSparkSQLTerraform

About the role

Key responsibilities & impact

Design, develop, and optimize scalable data ingestion pipelines using Azure Data Factory, Databricks, and Apache Spark
Build and maintain Databricks notebooks using PySpark for data preparation, transformation, and enrichment
Collaborate with Data Scientists to understand data requirements and deliver clean, structured datasets
Implement data quality checks, validation rules, and monitoring mechanisms
Integrate structured, semi-structured, and unstructured data sources
Ensure data security, governance, and compliance with banking regulations
Optimize Spark jobs for performance and cost-efficiency in Azure
Participate in code reviews, design discussions, and agile ceremonies
Document data pipelines, workflows, and technical decisions

Requirements

What you’ll need

At least 2+ years of hands-on experience in Azure Databricks
Strong proficiency in Apache Spark, especially PySpark
Experience with Azure Data Factory, Azure Synapse, Azure Blob Storage, and Delta Lake
Solid understanding of ETL/ELT processes, data modeling, and data warehousing concepts
Proficiency in Python and/or Scala for data processing
Experience with CI/CD pipelines using Azure DevOps, Git, and Terraform preferred
Familiarity with SQL, NoSQL, and data lake architectures
Knowledge of data governance, security, and compliance in financial services
Strong analytical and problem-solving skills
Excellent communication and stakeholder management abilities
Ability to work collaboratively in cross-functional teams
Agile mindset and experience in Scrum/Agile environments
Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field preferred
Databricks Certified Data Engineer Associate/Professional certification is a plus
Experience in banking or financial services is highly desirable
Australian citizenship required
Must be eligible to obtain security clearance

Benefits

Comp & perks

Commitment to employee growth, well-being, and success
Career path opportunities aligned with skills and interests
Freedom to be your authentic self