ML Data Engineer
Smart Working
AWSAzureCloudDynamoDBETLGoogle Cloud PlatformNoSQLNumpyPandasPostgresPythonScikit-Learn
Senior Data Engineer
NowVertical Group Inc.
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇮🇳 India
Visit company websiteJob Level
Senior
Tech Stack
AzureBigQueryCloudETLGoogle Cloud PlatformMicroservicesNoSQLPythonSDLCSQL
About the role
- Be a key member of our Technology Consulting team helping our clients in addressing their business challenges by participating in designing and developing their customer data analytics solutions.
- Analyse business/functional requirements, review the cloud-based solution architecture for ETL projects and develop the technical cloud-based architecture and design
- Work on the data from latest channels and avenues of customer interaction (e.g. cloud based/mobile solutions/social media/web logs along with the traditional data sources like CRM, Marketing automation) to derive customer insight and analytics.
- Design, build and test the cloud-based integration solution in collaboration with onsite/offshore team members. Deploy the solution on a high-availability shared service cloud-based production environment.
- Perform code reviews and ensure that the solution is aligned to pre-defined architectural standards, guidelines, best practices, and meet the quality standards.
- Understand and comply with the established software development life cycle methodology.
- Proactively identify opportunities for process improvements and implement the same.
- Establish and enhance technical guidelines and best practices for cloud integration development team.
Requirements
- Expertise of data integration methodologies with experience on different cloud-based data integration technologies such as DBT / Azure Data Factory / Google Cloud Data Fusion and Databricks utilising techniques on data pipelines, APIs, Microservices within GCP and/or Azure.
- Experience of programming languages (Python) pertaining to data engineering.
- Experience working with cloud-based SQL/NoSQL databases, data warehouses and data lakes like Snowflake, Azure Data Lake Storage (ADLS), Google Cloud Storage (GCS), Azure Synapse Analytics, and Google BigQuery.
- Expertise on GCP and/or Azure platforms and components including Google BigQuery, Cloud Functions, Cloud Dataflow, Cloud Storage, Azure Synapse Analytics, Azure Functions, Azure Data Factory, and Azure Data Lake Storage (ADLS).
- A good understanding of Data Governance/Data Management/Data Quality processes.
- Proficiency with data modelling techniques.
- Experience or knowledge of DevOps and DataOps utilising Git related platforms.
- The ability to work on fast-paced projects and deliver independently for any critical Consulting role.
- Relevant Certifications on GCP and/or Azure specifically **Microsoft Certified: Azure Data Engineer Associate or Google Cloud Certified - Professional Data Engineer**
Benefits
- You will have the opportunity to work in a successful, fast-moving mission-driven company
- You will have autonomy and growth opportunities
- You will be provided with stock options to invest in company’s growth
- You will collaborate with other brilliant minds
- You will always be surrounded by new challenges from clients, consultants and technology
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data integration methodologiescloud-based data integration technologiesDBTAzure Data FactoryGoogle Cloud Data FusionDatabricksPythonSQL databasesNoSQL databasesdata modelling techniques
Soft skills
process improvementsindependent deliverycollaborationcode reviewscompliance with standards
Certifications
Microsoft Certified: Azure Data Engineer AssociateGoogle Cloud Certified - Professional Data Engineer
