Senior Data Engineer
Deroyque IT
full-time
Posted on:
Location Type: Remote
Location: Remote • Brasil
Visit company websiteJob Level
Senior
Tech Stack
AzurePySparkScalaSQL
About the role
- Develop, build and maintain efficient, scalable data pipelines (ingestion, processing, cleansing and quality assurance).
- Perform data modeling (conceptual, logical and physical) to support various business needs.
- Implement and ensure the stability of processes and pipelines in QA and Production environments.
- Create automated health-check processes and develop modular, scalable pipelines.
- Actively collaborate with cross-functional teams in an agile environment, using strong communication skills.
- Troubleshoot complex problems and operate in dynamic, high-performance environments.
Requirements
- Minimum of 5 years of proven experience in Data Engineering.
- Hands-on and essential experience with Databricks.
- Solid experience with the Azure platform (Data Factory, Synapse Analytics, Data Lake Storage, etc.).
- Deep knowledge of modern data storage (Data Lake, Delta Lake, Lakehouse).
- Proficiency in developing pipelines using PySpark and/or Scala.
- Experience with the Medallion architecture.
- Experience handling transactional/relational data and advanced SQL skills.
- Knowledge and practice in performance and cost optimization in PySpark and SQL.
- Bachelor’s degree in Computer Science, Engineering, Information Systems or related fields.
- English: intermediate to advanced (technical reading and professional communication).
- Strong communication skills and experience working in collaborative, agile teams.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data engineeringdata modelingdata pipelinesPySparkScalaSQLperformance optimizationcost optimizationMedallion architecturedata storage
Soft skills
communication skillscollaborationtroubleshootingagile environmentproblem-solving