CI&T

Mid-level Data Engineer, Azure/Databricks

CI&T

full-time

Posted on:

Location Type: Hybrid

Location: Brazil

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AzureCloudETLNoSQLPySparkPythonSQL

About the role

  • Design and manage data pipelines using Azure Data Factory.
  • Develop and maintain notebooks, workflows, and Delta Tables in Databricks.
  • Implement data processing solutions with Python and PySpark.
  • Build stored procedures, views, and functions in SQL Server (on-premises and in the cloud with Azure SQL).
  • Work with relational and non-relational databases, ensuring efficiency and integrity in data storage and retrieval.
  • Model data using widely recognized industry approaches, primarily transactional but also including dimensional structures such as star and snowflake schemas.
  • Ensure the quality, scalability, and security of implemented data solutions.
  • Collaborate with multidisciplinary teams (product, development, and analytics) to meet organizational needs.
  • Develop and maintain CI/CD processes for continuous integration and delivery in data pipelines.

Requirements

  • Strong experience with Azure Data Services and Databricks.
  • Proficiency in Python, PySpark, and SQL for data manipulation and transformation.
  • Knowledge of CI/CD and versioning with GitHub.
  • Experience in data modeling (conceptual, logical, and physical).
  • Background in cloud migration projects.
  • Experience in ETL/ELT and integrating large volumes of data.
  • Experience deploying AI models and using tools such as Azure ML.
  • Understanding of DevOps practices applied to the data environment.
Benefits
  • Health and dental insurance
  • Meal and food allowance
  • Childcare assistance
  • Extended parental leave
  • Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass
  • Profit Sharing (PLR)
  • Life insurance
  • Continuous learning platform (CI&T University)
  • Discount club
  • Free online platform dedicated to promoting physical and mental health and well-being
  • Pregnancy and responsible parenthood course
  • Partnerships with online course platforms
  • Language learning platform
  • And many others

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data pipelinesAzure Data FactoryDatabricksPythonPySparkSQL ServerETLELTdata modelingAI models
Soft skills
collaborationcommunicationorganizational skills