CI&T

Data Developer, Mid-level

CI&T

full-time

Posted on:

Location Type: Remote

Location: Brazil

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AirflowApacheBigQueryCloudETLGoogle Cloud PlatformKafkaPythonSparkSQLTerraform

About the role

  • Migration Execution (Refactoring & Modernization): Analyze and migrate legacy notebooks and pipelines (Spark/Databricks). This includes refactoring existing logic to the new architecture as well as fully rewriting processes (modernization) into SQL/Dataform or Dataflow.
  • Building ELT/ETL Pipelines: Develop and maintain data transformations using BigQuery and Dataform (SQL) to create Trusted/Silver and Gold layers, ensuring data quality, deduplication, and standardization.
  • Data Ingestion (Batch & Streaming): Implement ingestion patterns using Dataflow (Apache Beam) for event consumption (Kafka/Event Hubs) and Datastream for CDC from transactional databases. Work with data persistence in the Raw layer using Iceberg tables managed via BigLake.
  • Automation and Infrastructure as Code (IaC): Use Terraform to provision data resources (datasets, tables, views) and manage pipelines via CI/CD (GitHub Actions), following an Ingestion Factory model and domain-separated repositories.
  • Quality and Governance: Implement data quality tests (assertions in Dataform) and ensure data cataloging and lineage using Dataplex and Analytics Hub for secure sharing across domains.

Requirements

  • Strong SQL experience: Ability to write complex, high-performance queries, preferably in the Google BigQuery dialect.
  • Google Cloud Platform (GCP) knowledge: Hands-on experience with services such as BigQuery, Cloud Storage (GCS), Dataflow, and Cloud Composer (Airflow).
  • Data Engineering (Python/Spark): Experience with data processing using Python and Apache Spark (to understand legacy Databricks code and work with Dataproc when necessary).
  • Data Architecture concepts: Understanding of Data Lakehouse, data modeling, partitioning, and file formats (Parquet, Avro, Iceberg).
  • Versioning and CI/CD: Experience with Git and automated deployment pipelines.
  • Previous experience with Dataform or dbt for orchestrating SQL transformations.
  • Knowledge of Terraform for Infrastructure as Code (IaC).
  • Familiarity with event-driven architectures (Kafka or Event Hubs) and stream processing.
  • Understanding of Databricks (to facilitate reading and migrating legacy code).
  • Knowledge of data governance (Dataplex, IAM) and security (VPC Service Controls).
Benefits
  • Health and dental insurance;
  • Food and meal allowance;
  • Childcare assistance;
  • Extended parental leave;
  • Partnerships with gyms and health & wellness providers via Wellhub (Gympass) TotalPass;
  • Profit Sharing (PLR);
  • Life insurance;
  • Continuous learning platform (CI&T University);
  • Discount club;
  • Free online platform dedicated to promoting physical and mental health and wellness;
  • Pregnancy and responsible parenthood course;
  • Partnerships with online course platforms;
  • Language learning platform;
  • And many more
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
SQLBigQueryDataformDataflowApache BeamTerraformPythonApache SparkGitCI/CD