Tech Stack
Amazon RedshiftAWSETLJavaKafkaLinuxPythonScalaSparkSQL
About the role
- Developing ETL pipelines in and out of data warehouse using combination of Java/ Scala/ Python Spark jobs for data transformation and aggregation
- Writing SQL queries against Snowflake.
- Provide production support for Data Warehouse issues such data load problems, transformation translation problems
- Develop production grade real time or batch data integrations between systems
- Real time data processing of events from Kafka using streaming processing into data warehouse
- Design and build data pipelines of medium to high complexity
- Translate requirements for BI and Reporting to Database design and reporting design
- Design and build machine learning pipelines of medium to high complexity
- Deploy production grade data pipelines, data infrastructure and data artifacts as code.
Requirements
- Minimum BSc or BTech / B . E in Computer Science, Engineering, or related discipline.
- Relevant professional qualification such as AWS Certified Big Data, SnowPro Core certification, other Data Engineer certifications
- Strong development hands-on background in creating Snow pipe, and complex data transformations and manipulations using Snow Pipe, Snow SQL
- Hands-on experience with Snowflake external tables concepts, Staging, Snow scheduler & performance tuning.
- Good understanding of Snowflake Time travel concepts and zero-copy cloning, Network policies, clustering, and tasks
- 5 + year experience working in an enterprise big data environment
- Deep knowledge of Spark , Kafka and data warehouse such as snowflake, Hive, Redshift etc
- Hands-on experience in development, deployment and operation of data technologies and platforms
- Deep knowledge of SQL
- OS knowledge particularly Linux
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Professional development opportunities
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
JavaScalaPythonSparkSQLETLMachine LearningData IntegrationData TransformationData Aggregation
Certifications
AWS Certified Big DataSnowPro Core certificationData Engineer certifications