Compass

Senior Spark Data Engineer

Compass

full-time

Posted on:

Location Type: Remote

Location: Brazil

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AWSLinuxPythonSparkSQLTerraform

About the role

  • Analyze legacy SAS routines;
  • Rewrite and adapt SAS programs to technologies within the data ecosystem;
  • Execute necessary tests, identify inconsistencies, and implement corrections;
  • Participate in functional and technical validation, comparing results between the legacy SAS and the new implementation.

Requirements

  • Strong experience with Python for data processing;
  • Hands-on experience with Spark for processing large volumes of data;
  • Proficiency in SQL for queries, transformations, and data validations.
  • Preferred: Experience with AWS services (S3, Athena, Glue, Lambda, and Step Functions), batch job orchestration with Control-M, code versioning with Git/GitHub, working in Linux environments, using Terraform for infrastructure as code (IaC), and experience in agile environments, including tools such as Agile Cockpit.

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
SASPythonSparkSQLAWSControl-MGitLinuxTerraformAgile