Data Engineering Specialist I
Inmetrics
ApacheAzureHadoopHBaseJavaJenkinsPySparkPythonSpark
Senior Data Engineering Analyst
Inmetrics
full-time
Posted on:
Location Type: Hybrid
Location: São Paulo • 🇧🇷 Brazil
Visit company websiteJob Level
Senior
Tech Stack
ApacheAzureHadoopHBaseJavaJenkinsPostgresPySparkPythonSpark
About the role
- The Data Engineer will work with the Cards team's Data Engineering group to create data pipelines for ingesting and provisioning card-domain data into Santander Brazil's corporate data lake. The person will work within an agile team on a strategic project for the area and must have experience with Databricks and PySpark.
Requirements
- Mandatory Knowledge
- Databricks proficiency: Experience working with Apache Spark on Databricks, including creating and optimizing data pipelines.
- Experience with PySpark, Python and Kedro: Strong programming skills in PySpark and Python and experience with Kedro to develop, debug, and maintain data transformation code.
- Batch and streaming data processing: Knowledge of batch and streaming (messaging) data processing, with the ability to design, implement, and maintain data processing pipelines.
- DevOps knowledge: Familiarity with using Jenkins for continuous integration and delivery (CI/CD), as well as automation of deployment tasks and pipeline management.
- Git: Proficiency with Git for source code version control and effective collaboration in development teams.
- Agile methods: Understanding of agile principles and practices, such as Kanban and Scrum, for effective collaboration and project management.
- Orchestration (e.g., Control‑M or other): Knowledge of workflow orchestration tools, important for scheduling and controlling workflows.
- Microsoft Azure knowledge: Experience with key Microsoft Azure data services, including Azure Databricks, Azure Data Factory, and Azure Storage Accounts.
- Desired Knowledge
- Knowledge in Azure: Experience with core services such as Aurora PostgreSQL, CloudWatch, Lambda, and S3.
- Experience with On‑Premises Environments (Cloudera): Preferred.
- Previous experience with the Cloudera platform or other on‑premises big data solutions, including Hadoop, HBase, and Hive.
- Object‑oriented development knowledge: Familiarity with Java is very helpful (not required to code, but to interpret).
- Optional certifications: AZ‑900 (Microsoft Azure Fundamentals) and DP‑900 (Microsoft Azure Data Fundamentals) are preferred and demonstrate solid knowledge of the Azure platform and data concepts.
Benefits
- Bradesco Health Plan (30% copayment)
- Bradesco Dental Plan (no contribution)
- Life Insurance
- Wellhub (Gympass)
- Childcare allowance
- Allowance for children with special needs
- Payroll-deductible loan (Consigned credit)
- Private pension
- Pet plan
- SESC benefits
- Conexa telemedicine
- Cost assistance
- Meal / Food allowance
- Multi-benefit card
- Medical plan upgrade
- We are a Citizen Company: extended maternity and paternity leave
- INMaterna Program: support program for pregnant employees
- Birth kit and the book "It Happened When I Was Born"
- Professional development: courses available through the internal university
- 100% remote or hybrid, depending on project applicability.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
DatabricksPySparkPythonKedrobatch data processingstreaming data processingDevOpsJenkinsGitworkflow orchestration
Soft skills
agile principlescollaborationproject management
Certifications
AZ-900DP-900
