CI&T

Senior Data Developer, Databricks

CI&T

full-time

Posted on:

Location Type: Remote

Location: Remote • 🇧🇷 Brazil

Visit company website
✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowApacheAWSAzureCloudETLGoogle Cloud PlatformHadoopKafkaNoSQLPythonScalaSparkSQLTableau

About the role

  • Create robust, scalable data pipelines using Databricks, Apache Spark, and SQL to transform and process large datasets efficiently.
  • Collaborate with data architects to design data models and architecture that support data analytics and machine learning applications.
  • Monitor and optimize the performance of existing data pipelines and workflows to ensure high throughput and low latency.
  • Work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical specifications.
  • Implement data quality checks and governance practices to ensure data consistency, accuracy, and compliance.
  • Maintain comprehensive documentation for data pipelines and processes, and contribute to the establishment of best practices within the team.
  • Provide guidance and mentorship to junior team members, fostering a culture of learning and collaboration.

Requirements

  • 5+ years of experience in data engineering or a related field, with a focus on data pipelines and ETL processes.
  • Excellent communication in English and Portuguese, with the ability to work effectively in a team-oriented environment and engage with clients.
  • Proficiency in Databricks, Apache Spark, SQL, and Python (or Scala).
  • Experience with cloud platforms such as Azure, AWS, or GCP, particularly related to data storage and processing solutions.
  • Experience with additional big data technologies such as Hadoop, Kafka, or Airflow.
  • Strong understanding of data modeling concepts and experience with relational and NoSQL databases.
  • Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data issues.
  • Bachelor’s degree in Computer Science, Information Technology, or a related field.
  • Nice to Have:
  • Databricks certification (e.g., Databricks Certified Developer) or other relevant certifications.
  • Familiarity with machine learning frameworks and concepts, and experience integrating data engineering with machine learning workflows.
  • Knowledge of data visualization tools (e.g., Tableau, Power BI) to help communicate insights effectively.
  • Experience working in Agile development environments and familiarity with tools such as JIRA or Confluence.
Benefits
  • Health and dental insurance
  • Meal and food allowance
  • Childcare assistance
  • Extended paternity leave
  • Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass;
  • Profit Sharing and Results Participation (PLR);
  • Life insurance
  • Continuous learning platform (CI&T University);
  • Discount club
  • Free online platform dedicated to physical, mental, and overall well-being
  • Pregnancy and responsible parenting course
  • Partnerships with online learning platforms
  • Language learning platform
  • And many more!

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data engineeringdata pipelinesETL processesDatabricksApache SparkSQLPythonHadoopKafkaAirflow
Soft skills
communicationteam-orientedanalytical skillsproblem-solvingmentorshipcollaboration
Certifications
Databricks Certified Developer