Deeplight AI

AWS Glue Data Engineer

Deeplight AI

full-time

Posted on:

Location Type: Hybrid

Location: Dubai • United Arab Emirates

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AWSETLPySparkTerraform

About the role

  • ***Your responsibilities as the AWS Glue Data Engineer will include:***
  • - **Data Ingestion Development**
  • - Building and implementing AWS Glue jobs for Bronze layer ingestion using defined standards and templates.
  • - Implementing correct loading methods based on source requirements (CDC, full load, delta, snapshot).
  • - Designing and executing historical loading mechanisms to bring legacy data into the Lakehouse.
  • - **Performance Optimisation**
  • - Optimising Glue job performance (DPU allocation, parallelization, partitioning) according to best practices.
  • - Collaborating with platform teams to ensure tooling and optimization alignment.
  • - **Migration & Automation**
  • - Aggressively migrating source tables to Bronze layer, initially using manual approaches with standards/templates, later leveraging AI-enabled acceleration.
  • - Ensuring jobs are version-controlled and production deployment is automated via Git and Terraform.
  • - **Governance & Monitoring**
  • - Implementing source system connectivity into CDP in collaboration with source system owners.
  • - Ensuring jobs comply with data contracts and are properly monitored.
  • - Preparing documentation and handover to operational support teams.
  • - **Collaboration**
  • - Working closely with Data Architect for ingestion patterns and standards.
  • - Coordinating with Data Assurance Lead to apply quality checks across all jobs.
  • - Partnering with platform engineers for tooling and optimisation.

Requirements

  • ***You will have experience in:***
  • - AWS Glue, PySpark, and ETL pipeline development;
  • - substantial knowledge of Lakehouse architecture and Medallion design principles;
  • - familiarity with CDC, delta loads, and historical data ingestion strategies; and;
  • - 5+ years experience in data engineering roles, with hands-on experience in AWS Glue.
  • ***You should also have knowledge of:***
  • - AWS services: Glue, S3, Athena, Lambda;
  • - Git, Terraform for CI/CD automation;
  • - data quality frameworks (e.g., Soda Core);
  • - identifying ways to automate their work / repetitive tasks;
  • - working in a fast-paced environment and deliver aggressive migration targets;
  • - collaborating and communication with different stakeholder levels; and;
  • - working with Jira and agile way of working.
Benefits
  • **Benefits & Growth Opportunities:**
  • · Competitive salary and performance bonuses
  • · Comprehensive health insurance
  • · Professional development and certification support
  • · Opportunity to work on cutting-edge AI projects
  • · Flexible working arrangements
  • · Career advancement opportunities in a rapidly growing AI company
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
AWS GluePySparkETL pipeline developmentLakehouse architectureMedallion design principlesCDCdelta loadshistorical data ingestiondata quality frameworksautomation
Soft Skills
collaborationcommunicationstakeholder managementperformance optimizationproblem-solvingadaptabilityattention to detailtime managementteamworkleadership