B2

Senior/Mid-level Data Engineer

B2

contract

Posted on:

Location Type: Remote

Location: Brazil

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AWSDynamoDBETLPySparkPythonSparkSQLTerraform

About the role

  • Develop and maintain scalable data pipelines (batch and streaming)
  • Ensure efficient ingestion, transformation, and provisioning of data
  • Model data to support analytics and AI applications
  • Integrate multiple data sources (APIs, event streams, and internal systems)
  • Build and optimize jobs using Spark / PySpark
  • Orchestrate pipelines using Step Functions
  • Work with event-driven architecture
  • Organize and structure data in Data Lake / Lakehouse environments
  • Implement data partitioning and optimization strategies
  • Ensure pipeline quality, reliability, and observability
  • Optimize cost and performance in AWS environments
  • Collaborate with data teams (analysts and data scientists) to deliver solutions
  • Support technical decisions related to data architecture and engineering

Requirements

  • Experience building data pipelines (batch and streaming)
  • Data processing in a lakehouse architecture
  • Data modeling for analytical consumption
  • Integrating data across multiple sources (APIs, events, internal systems)
  • Advanced Python (ETL, automation, and API integration)
  • Advanced SQL for data manipulation and optimization
  • Spark / PySpark for developing and optimizing jobs
  • Experience with AWS (S3, Glue, Lambda, Step Functions)
  • Knowledge of EMR (strong plus)
  • Experience with messaging and event services (EventBridge, SQS, SNS)
  • DynamoDB (data modeling and access)
  • Pipeline orchestration with Step Functions
  • Event-driven architecture
  • Experience with Data Lake / Lakehouse (Parquet, raw/curated/analytics layers)
  • Data partitioning strategies
  • Best practices for data quality and pipeline observability
  • Performance and cost optimization in data environments
  • Version control with Git
  • Infrastructure as code with Terraform (intermediate level)
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
data pipelinesdata processingdata modelingPythonSQLSparkPySparkAWSDynamoDBTerraform
Soft Skills
collaborationtechnical decision making