Data Engineer

• Design, develop, and optimize next-generation data pipelines and data platforms using PySpark
• Work with large-scale datasets and solve complex data challenges
• Feature development, data quality checks, and deploying/integrating ML models with backend services and the Tide platform
• Identify, diagnose, and resolve complex performance bottlenecks in PySpark jobs and Spark clusters using Spark UI and query plans
• Lead design and implementation of scalable, fault-tolerant ETL/ELT pipelines for batch and potential real-time processing
• Collaborate with data scientists, analysts, and product teams to design efficient data models (star/snowflake schemas, SCDs)
• Implement data quality checks, monitoring, and alerting to ensure accuracy and reliability of data assets
• Contribute to data architecture strategy and evaluate new technologies and best practices
• Promote engineering best practices: code quality, testing, documentation, version control; participate in code reviews
• Mentor junior data engineers and foster continuous learning
• Work closely with cross-functional teams including software engineers, data scientists, product managers, and business stakeholders

Lead Data Engineer, PySpark

Data Engineer

Senior Data Engineer

Vice President – Data Engineering, Analytics

Senior Azure/Fabric Data Engineer

AWS Data Pipeline Engineer