Lead Data Engineer
Ness Digital Engineering
full-time
Posted on:
Location Type: Hybrid
Location: New York City • Massachusetts • New York • United States
Visit company websiteExplore more
Job Level
About the role
- Lead the data architecture and engineering strategy, ensuring alignment with business and technology roadmaps.
- Design and implement data models, pipelines, and data integration frameworks across multiple platforms.
- Partner with stakeholders to translate business requirements into scalable data solutions.
- Performance Optimization: Optimize data pipelines for performance, scalability, and reliability, including query tuning and resource management within Snowflake.
- Drive adoption of best practices in data engineering design patterns and modern cloud architectures.
- Data Quality Assurance: Implement and monitor data validation procedures to ensure data accuracy and consistency across systems.
- Collaboration and Communication: Work closely with project managers, data architects, and business analysts to align project milestones and deliverables with business goals.
- Mentor and guide data engineering teams (onsite & offshore) to deliver high-quality outcomes.
- Ensure compliance with data governance, security, and privacy standards.
- Documentation: Create and maintain detailed documentation of data pipelines, data flow diagrams, and transformation logic.
- Issue Resolution: Troubleshoot and resolve issues related to data pipelines, including job failures and performance bottlenecks.
Requirements
- Bachelor’s degree in Computer Science, Information Technology, or a related field
- 8+ years of experience in data engineering with a strong focus on Data Architecture, Data Modelling (Conceptual, Logical, Physical), ELT processes and data pipeline development.
- Hands-on experience with Snowflake cloud data platform, including data sharing, secure views, and performance optimization.
- Proficiency in SQL and familiarity with data integration and ETL/ELT tools.
- AWS Technologies (Glue, EMR)
- Python for data engineering workflows
- Strong understanding of data engineering design patterns
- Strong problem-solving skills and the ability to work independently to meet deadlines.
- Excellent communication skills for effectively interacting with technical and non-technical stakeholders.
Benefits
- Flexible remote options
- Access to development resources
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data architecturedata modelingELT processesdata pipeline developmentSQLSnowflakeAWS GlueAWS EMRPythondata engineering design patterns
Soft Skills
problem-solvingcommunicationcollaborationmentoringstakeholder engagementindependent workattention to detaildocumentationissue resolutionperformance optimization
Certifications
Bachelor’s degree in Computer ScienceBachelor’s degree in Information Technology