Minsait

Data Engineer – Senior

Minsait

full-time

Posted on:

Location Type: Hybrid

Location: Goiânia • Brasil

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowApacheETLPython

About the role

  • Develop and maintain data pipelines in Big Data environments (batch and, when necessary, near real-time);
  • Implement data ingestion, transformation and load processes (ETL/ELT);
  • Work on data modeling in distributed environments (Hive and Impala);
  • Manage and optimize queries over large volumes of data;
  • Orchestrate data workflows using Apache Airflow;
  • Develop scripts and applications in Python for data processing;
  • Use notebooks (Jupyter and Hue) for exploration, development and testing;
  • Ensure data quality, integrity and governance;
  • Implement best practices for versioning, documentation and observability;
  • Support the construction of the semantic layer for consumption by BI tools;
  • Develop APIs for data ingestion and extraction;
  • Create RESTful services for integration between internal and external systems;
  • Implement validation, transformation and data control routines in APIs;
  • Ensure performance and scalability when consuming large volumes of data;
  • Maintain and evolve existing APIs, ensuring stability and compatibility;
  • Implement access control and security (authentication and authorization);
  • Monitor and log executions for traceability and auditing;
  • Document APIs using Swagger/OpenAPI.

Requirements

  • Bachelor’s degree completed (or equivalent);
  • Solid experience in data engineering within Big Data and Data Lake environments;
  • Advanced knowledge of Python for data processing;
  • Experience with distributed data modeling (Hive and Impala);
  • Experience orchestrating pipelines using Apache Airflow;
  • Experience building RESTful APIs and integrating systems;
  • Knowledge of data governance, quality and observability practices;
  • Experience with code versioning and technical documentation;
  • Knowledge of analytical layer architectures (RAW, TRT, RFD / Bronze, Silver, Gold);
  • Ability to optimize queries over large volumes of data;
  • Capacity to solve complex problems;
  • Emotional intelligence, teamwork, leadership and proactivity.
Benefits
  • Company-subsidized Health Plan for the employee;
  • Option to include dependents in the Health Plan with payroll deduction;
  • Dental Assistance (optional);
  • Option to include dependents in the Dental Assistance with payroll deduction;
  • Meal Allowance or Food Voucher;
  • Transportation Voucher (optional);
  • Impact & Care - Personal Guidance Program offering emotional support and confidential counseling in psychological, legal, financial, social and pet-related matters at no cost for the employee and legal dependents;
  • Gympass - Wellhub (access to over 700 gyms across Brazil with plans starting at R$ 29.90 deducted from payroll);
  • Option to include dependents in Gympass - Wellhub (up to 3 dependents - paid via credit card);
  • Access to Udemy through our intranet;
  • Partnerships with major consumer brands;
  • Agreement with SESC for employee and dependents;
  • Discounts agreements with educational institutions (undergraduate and graduate) and language/certification schools;
  • Group life insurance.
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
data pipelinesETLELTdata modelingApache AirflowPythonAPIsRESTful servicesdata governancedata quality
Soft Skills
problem solvingemotional intelligenceteamworkleadershipproactivity
Certifications
Bachelor’s degree