Senior Data Engineer
GVW Group
full-time
Posted on:
Location Type: Hybrid
Location: Durbanville • South Africa
Visit company websiteExplore more
Job Level
About the role
- Serve as the subject matter expert for data and systems.
- Develops and maintains scalable data pipelines and build new API integrations to support continuing increases in data volume and complexity.
- Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increase data accessibility and foster data-driven decision making across the organization.
- Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
- Develop end-to-end Data pipelines encompassing the full lifecycle from data ingestion, data transformation and Data Marts in our Enterprise Data Warehouse.
- Collaborate closely with Data Scientists to accelerate development of ML and AI solutions ready for production.
- Setup CI/CD/CT pipelines
- Contributes to engineering wiki, and documents work.
- Performs data analysis required to troubleshoot data related issues and assists in the resolution of data issues.
- Works closely with cross-functional teams of frontend and backend engineers, product managers, and analysts to enhance data models and support advanced BI and analytics.
- Defines company data assets (data models), ETL jobs to populate data models.
- Designs data integrations and data quality frameworks.
- Designs and evaluates open source and vendor tools for data lineage.
- Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.
- Mentor junior data engineers, lead code reviews, and promote best practices and skill development.
Requirements
- BSc or MSc degree in Computer Science or a related technical field
- 5+ years of Python development experience
- 5+ years of MS SQL experience
- 5+ years of experience with Warehouse Architecture, schema design and dimensional data modeling.
- Ability in managing and communicating data warehouse plans to internal clients
- Experience designing, building, and maintaining data processing systems on multiple platforms both Cloud (Azure, AWS, MS Fabric) and On-Premises (MS SQL Server, SSIS) is advantageous
- Good experience in Apache Spark.
- Experience debugging and reasoning about production issues is essential.
- Experience presenting demos and training of technical, non-technical and analytical resources.
Benefits
- Hybrid work flexibility
- Professional development opportunities
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
PythonMS SQLETLdata modelingdata pipeline developmentdata transformationdata quality frameworksApache SparkCI/CDAPI integrations
Soft Skills
communicationcollaborationmentoringtroubleshootingdocumentationpresentationleadershipproblem-solvingdata-driven decision makingcross-functional teamwork
Certifications
BSc in Computer ScienceMSc in Computer Science