Salary
💰 $150,000 - $200,000 per year
Tech Stack
AWSCassandraCloudGoMySQLNeo4jPythonSQL
About the role
- Design, implement, and maintain the organization’s data infrastructure, ensuring it meets business requirements and technical standards.
- Deploy data pipelines to AWS infrastructure such as EKS, ECS, Lambdas and AWS Glue.
- Develop and deploy data pipelines to clean and transform data to support other engineering teams, analytics and AI applications.
- Extract and deploy reusable features to Feature stores such as Feast or equivalent.
- Evaluate and select appropriate database technologies, tools, and platforms, both on-premises and in the cloud.
- Monitor data systems and troubleshoot issues related to data quality, performance, and integrity.
- Work closely with Product, Engineering, and Analytics teams to understand and cater to their data needs.
- Define and document data workflows, pipelines, and transformation processes for clear understanding and knowledge sharing.
- Ensure student data privacy and security, and adhere to PII compliance best practices.
Requirements
- 3+ years of experience as a data engineer, and 8+ years of software engineering experience (including data engineering)
- Proven experience as a Data Engineer or in a similar role with strong data modeling, architecture, and design skills.
- Strong understanding of data engineering principles including infrastructure deployment, governance and security.
- Experience with MySQL, Snowflake, Cassandra and familiarity with Graph databases (Neptune or Neo4J)
- Proficiency in SQL, Python, (Golang)
- Proficient with AWS offerings such as AWS Glue, EKS, ECS and Lambda
- Excellent communication skills and ability to articulate complex technical concepts to non-technical stakeholders
- Strong understanding of PII compliance and best practices in data handling and storage
- Strong problem-solving skills, with a knack for optimizing performance and ensuring data integrity and accuracy.