Tech Stack
Amazon RedshiftAWSAzureBigQueryCloudFirewallsGoogle Cloud PlatformJavaKubernetesNoSQLPythonScalaSparkSQL
About the role
- Mentorship and solution support to client-facing Data Engineers across key accounts: review work, establish adoption of best practices and frameworks, and coordinate onsite/offshore leadership updates.
- Work with DevOps and MLE teams to onboard and productionize Data Science models at scale.
- Implement and manage production processes for data ingestion, transformation, coding utilities, storage, reporting, and other data integration points.
- Engage key client stakeholders to review and provide solutions for existing or new architecture requirements.
- Recommend strategies to improve resiliency, security, and optimize costs.
- Analyse, architect, design, and actively develop analytics frameworks, data lakes, and other cloud-based data solutions.
- Design and develop scalable data ingestion frameworks to transform a variety of datasets, capture metadata and lineage, and implement data quality.
Requirements
- Bachelor’s degree in Computer Science/or any relevant experience in Information systems.
- 10 years of experience in the IT field with hands-on experience in system administration and automation
- 3-5 years of strong hands-on experience with AWS(or another cloud) in building a Data and Analytics platform on the cloud.
- 1+ years of experience with deploying services on Kubernetes on-prem or on-cloud
- 3-5 years of strong hands-on expertise on Cloud (AWS/Azure/GCP) with strong experience in working with modern data and analytics frameworks, big data, and DevOps techniques, and is proficient with programming languages (like Python, Java, Scala).
- 1-2 yrs experience with Cloud data warehouse (Redshift, Snowflake, BigQuery)
- Solid understanding of building Data Lakes on Cloud environments - Data Integration, Data Pipeline, Data compute, ML Models and Storage, etc.
- Solid knowledge of and experience with Big data and Stream processing frameworks - Spark, Flink, or other
- Knowledge of delta lake or similar technology
- Exposure to operationalizing ML models @ scale on Cloud
- Solid understanding of DevSecOps and cloud security concepts and how to mitigate security risks in the cloud
- Must understand the Scrum Agile methodology and have worked on a Scrum team.
- Knowledge of SQL, non-relational (NoSQL) databases, and Object Stores
- Good experience working on an onsite/offshore model
- Knowledge of networking, firewalls, load balancers, etc.
- AWS Professional Architect Certification is preferred
- Exceptional communication skills and the ability to communicate appropriately with executives and technical teams.