Data Engineer
Ocean Technologies Group
full-time
Posted on:
Location Type: Remote
Location: India
Visit company websiteExplore more
About the role
- Create and maintain optimal data pipeline architecture from multiple data sources.
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
- Ensure robust data validation and verification processes to ensure the correctness of data for reporting.
- Develop and maintain data warehousing approaches, including the modelling and pipelining of data for storage and retrieval.
- Keeping data separated and secure as per best practice guidelines
- Ensure that the data taken from the information sources is accurate, liaising with other business functions when necessary
- Proactively keep up to date with the latest cloud data technology
- Provide support, documentation and training to other applicable team members as and when required to ensure that overall objectives are met
Requirements
- Experience with building and maintaining ETL/ELT pipelines as part of an automated workflow on high-volume, high-dimensionality data from varying sources
- Good understanding on how data management, cleansing and query optimisation influences pipeline, and data model design
- Good understanding of dimensional modelling and slowly changing dimensions
- Experience using cloud technologies including Azure, AWS (preferred)
- Experience with scripting languages (e.g. Python, PySpark)
- Demonstrably strong problem-solving skills to help design and implement solutions to data problems.
- Experience of CICD technologies and source control e.g. Liquibase, Jenkins
- Understand and apply infrastructure knowledge to develop and maintain an efficient environment.
- Modelling datasets in a way applicable for use in different visualisation tools (e.g. Power BI, Apache Superset)
- Strong stakeholder management skills
- Highly numerate and logical
- Ability to clearly communicate and present outputs to stakeholders.
- Experience of working in fast paced environments to strict deadlines.
- Team player and can communicate effectively to technical and non-technical colleagues
- Passionate about continuous improvement, collaboration and knowledge sharing at all levels
- The ability to manage time, prioritise tasks, self-review your work and produce deliverables of a high quality under tight client deadlines in time pressured environments
- It would be useful if you’ve got experience with some of these technologies- AWS Glue, AWS RDS, AWS Aurora, AWS S3, Amazon Athena, Amazon EMR, PostgreSQL, Python, Bitbucket, Liquibase, Jenkins, Lucid chart
- Experience in developing deliverables in visualisation tools such as Tableau, PowerBI or open-source BI tools such as Apache Superset (Desirable)
Benefits
- - A highly competitive salary
- - A discretionary annual performance bonus
- - Statutory benefits including enhanced Private Medical Insurance
- - A “remote first” working environment where we fully support remote working
- - Internal mobility options - we post all vacancies on our internal job board and encourage all Oceaneers to make their next move within OneOcean
- - A culture of continuous development and growth
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
ETLELTdata managementdata cleansingquery optimisationdimensional modellingscripting languagesproblem-solvingdata modellingdata validation
Soft Skills
stakeholder managementcommunicationteam playertime managementprioritisationcollaborationcontinuous improvementlogical thinkingself-reviewadaptability