FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesAirflowApacheAzureETLPySparkSQLUnity
About the role
Key responsibilities & impact- Pipeline Development: Implement ELT/ETL pipelines for ingesting, transforming, and delivering data from diverse sources, including spreadsheets, SharePoint, and relational databases, following architecture standards;
- Data Platform Construction: Build and organize the layers of the Data Lake/Lakehouse (Bronze, Silver and Gold / Medallion Architecture) using optimized formats such as Delta Lake and Parquet;
- Transformation and Modeling: Develop data transformations using PySpark and advanced SQL, implementing architecture-defined models in the platform's analytical layers;
- Ingestion Strategies: Implement loading strategies (full load or incremental) appropriate to the volume and criticality of each data domain — focusing on efficiency for small-data contexts;
- Data Quality: Implement automated data quality tests and checks within pipelines, ensuring consistency, integrity, and traceability across layers;
- Cataloging and Governance: Support the cataloging and documentation of data assets with emphasis on lineage and classification, following governance guidelines defined by the Architect, establishing a unified data catalog as a central element of platform management;
- Observability: Ensure observability of pipelines through logs, alerts, and proactive monitoring;
- Documentation: Document models, transformations, and business rules applied in pipelines, ensuring traceability and maintainability of the solution.
Requirements
What you’ll need- Proven experience in Data Engineering in cloud environments;
- Strong experience with PySpark for developing pipelines and data transformations on distributed platforms;
- Experience with advanced SQL for queries, transformations, and analytical modeling;
- Knowledge of Microsoft Azure (Azure Data Factory, ADLS Gen2 and/or Microsoft Fabric);
- Experience with data formats such as Delta Lake, Parquet, or ORC;
- Experience with Data Lake, Lakehouse, and Medallion Architecture;
- Experience with data cataloging and lineage, including Unity Catalog (Databricks);
- Preferred / Nice to Have:
- Experience with Databricks and/or Snowflake as a data processing platform;
- Familiarity with Microsoft Fabric (Lakehouses, Notebooks, Pipelines or Dataflows);
- Experience with Microsoft Purview for data cataloging and lineage;
- Knowledge of dbt for transformations and data quality testing;
- Experience orchestrating pipelines with Apache Airflow or Prefect;
- DP-700 certification or equivalent;
- Development best practices: version control with Git, automated testing, documentation, and CI/CD;
- Familiarity with Gen AI and autonomous agents applied to data.
Benefits
Comp & perks- Health and dental insurance;
- Meal and food allowance;
- Childcare assistance;
- Extended parental leave;
- Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) / TotalPass;
- Profit Sharing (PLR);
- Life insurance;
- Continuous learning platform (CI&T University);
- Discount club;
- Free online platform dedicated to promoting physical and mental health and wellbeing;
- Expectant parent and responsible parenthood course;
- Partnerships with online course platforms;
- Language learning platform;
- And many others
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
Data EngineeringPySparkSQLDelta LakeParquetData LakeLakehouseMedallion Architecturedata catalogingdata quality testing
Soft Skills
documentationobservabilitydata governance
Certifications
DP-700