EXL

Senior Data Engineer

EXL

full-time

Posted on:

Location Type: Remote

Location: Mexico

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowAWSCloudDistributed SystemsHadoopJenkinsPythonShell ScriptingSpark

About the role

  • Build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging.
  • Use expertise in solving big data problems and apply design thinking, coding and analytical skills to develop data pipelines that support marketing data products.
  • Integrate services directly with Lending Club’s products and with external vendors.
  • Work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., and cloud technologies across AWS stack.
  • Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
  • Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
  • Support operations to manage the production environment and help in resolving production issues with RCA.
  • Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Requirements

  • 9+ years of experience and a bachelor’s degree in computer science or a related field, or equivalent work experience
  • In-depth working experience of distributed systems Hadoop, Spark, Hive, DBT and Airflow/Dagster
  • At least 5 years of solid production quality coding experience in data pipeline implementation in Python
  • Experience working with public cloud platforms, preferably AWS
  • Experience working with Databricks and/or Snowflake
  • Experience in Git, JIRA, Jenkins, shell scripting
  • Familiarity with Agile methodology, test-driven development, source control management and test automation
  • Experience supporting and working with cross-functional teams in a dynamic environment
  • Excellent collaborative problem solving and communication skills
  • Track record of building fast, reliable, and high-quality data pipelines
  • Experience building Data pipelines for Digital Marketing will be a big plus
Benefits
  • Health insurance
  • Retirement plans
  • Paid time off
  • Flexible work arrangements
  • Professional development
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
data pipeline implementationbig data problem solvingPythonHadoopSparkDBTAirflowdata quality monitoringtest-driven developmentautomation
Soft Skills
collaborative problem solvingcommunicationcross-functional teamworkdesign thinkinganalytical skillsadaptabilityleadershiporganizational skillscreativityattention to detail