Commonwealth Bank

Staff Data Engineer – AWS Cloud, AI

Commonwealth Bank

full-time

Posted on:

Location Type: Hybrid

Location: Sydney • Australia

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Lead

Tech Stack

Amazon RedshiftAWSCloudETLPySparkPythonScalaSparkSQLTerraform

About the role

  • Design and build next-gen data platforms and pipelines using Python, PySpark, or Scala.
  • Integrate data from diverse sources into centralised lakes and cloud platforms.
  • Create technology blueprints and engineering roadmaps for long-term transformation.
  • Ensure data security, governance, and compliance.
  • Deliver end-to-end solutions that meet business needs and minimise risk.
  • Build high-quality, scalable data products from scratch.
  • Communicate effectively with stakeholders and contribute to team-wide engineering practices.
  • Define and evolve shared semantic models and metric definitions to ensure consistent meaning and reusable data across products and domains.
  • Drive adoption of catalogues, glossaries, and stewardship practices that promote discoverability, accountability, and self-service.
  • Embed policy-as-code for data access, classification, and retention ensuring compliance with internal and regulatory standards.
  • Implement automated testing, observability, and end-to-end lineage to improve data trust, traceability, and incident resolution speed.

Requirements

  • AWS Certified Data Engineer Associate with demonstrable experience in Data Engineering.
  • Strong expertise in Python, PySpark, SQL, Spark, Scala, and distributed data processing.
  • Experience in AWS data platforms (Glue, Lambda, Redshift, Athena, Kinesis, EMR).
  • Experience with AWS DataZone, Lake Formation, IAM, encryption.
  • Experience in using AI Powered coding Assistants like Cline, Roo and Aider.
  • Exposure to AWS Marketplace, API Gateway, and data monetization strategies.
  • Experience with Terraform, CloudFormation, GitOps, and serverless frameworks.
  • Ability to fine-tune ETL jobs, query performance, and cost efficiencies.
  • Knowledge of data product thinking and data mesh principles.
Benefits
  • Flexible working arrangements
  • Opportunities for professional development
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
PythonPySparkSQLSparkScaladata engineeringETLautomated testingdata processingdata product thinking
Soft Skills
effective communicationstakeholder engagementteam collaborationproblem-solvingaccountabilityself-service promotiondata governancerisk minimizationincident resolutiondata stewardship
Certifications
AWS Certified Data Engineer Associate