
Data Engineer
Paschoalotto
full-time
Posted on:
Location Type: Remote
Location: Brazil
Visit company websiteExplore more
About the role
- Design, develop, and maintain efficient and reliable data pipelines.
- Create integrations between heterogeneous systems and data stores (structured and unstructured).
- Perform cleansing, transformation, and modeling of large volumes of data.
- Implement event-driven solutions and real-time processing.
- Ensure data quality, integrity, and availability.
- Work closely with data analysts, software engineers, and architects to build scalable and secure solutions.
- Automate tasks and promote observability across ingestion, processing, and data delivery processes.
- Support data architecture and governance definition.
- Work with distributed processing systems such as DataProc, DataFlow, Databricks, Spark, and Hadoop;
- Integrate with APIs and external data sources;
- Use MLOps tools such as Kubeflow, MLflow, AutoML, and Azure ML;
Requirements
- Solid experience in data engineering and building ETL/ELT pipelines;
- Knowledge of messaging systems and real-time data integration;
- Familiarity with MLOps practices and tools;
- Knowledge of relational and analytical databases;
- Ability to handle large volumes of data in complex environments.
- Python: For ETL scripts, analysis, or automations.
- C# and .NET Core (desirable): Experience developing data services.
- Develop solutions using languages such as Python, SQL, and .NET.
- SQL Server / PostgreSQL: Proficiency in queries, modeling, and tuning.
- NoSQL: Knowledge of MongoDB and Redis is a plus.
- RabbitMQ and/or Kafka: Experience with asynchronous processing, events, and high-performance systems.
- Build and maintain automated data pipelines (batch or streaming).
- Experience with orchestration tools (e.g., Apache Airflow, Azure Data Factory, Luigi, etc.).
- Experience with Git, Azure DevOps, or similar tools for versioning and CI/CD.
- Automation of pipeline deployments and monitoring.
- Experience with distributed architectures and data replication between different systems.
- Implementation of encryption, anonymization, and access control for sensitive data.
Benefits
- Meal allowance
- Transportation allowance
- Perks club: An exclusive benefits club for our Paschoalovers, providing access to discounts at various partner establishments
- Wellz Care: Online therapy platform offering up to 12 sessions at no cost to Paschoalovers
- Wellhub: Well-being platform with discounted gym and physical activity options
- Day off: All employees receive a day off on their birthday
- Maternity and paternity leave: 6 months of maternity leave and 20 days of paternity leave
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data engineeringETLELTdata cleansingdata transformationdata modelingPythonSQLC#.NET Core
Soft skills
collaborationproblem-solvingcommunicationautomationobservability