Senior Data Engineer
Knowledge Services
contract
Posted on:
Location Type: Remote
Location: United States
Visit company websiteExplore more
Job Level
About the role
- Develop efficient and scalable data extraction methodologies to retrieve data from diverse sources, such as databases, APIs, web scraping, flat files, and streaming platforms.
- Design and implement robust data loading processes to efficiently ingest and integrate data into the latest data warehousing technology, ensuring data quality and consistency.
- Develop and maintain staging processes to facilitate the organization and transformation of raw data into structured formats, preparing it for downstream analysis and reporting.
- Implement data quality checks and validation processes to identify and address data anomalies, inconsistencies, and integrity issues.
- Identify and resolve performance bottlenecks in data extraction and loading processes, optimizing overall system performance and data availability.
- Ensure adherence to data security and privacy standards throughout the data extraction and warehousing processes, implementing appropriate access controls and encryption mechanisms.
- Create and maintain comprehensive documentation of data extraction and warehousing processes, including data flow diagrams, data dictionaries, and process workflows.
- Mentor and support junior data engineers, providing guidance on best practices, technical design, and professional development to elevate overall team capability and performance.
- Collaborate with cross-functional teams, including data scientists, data analysts, software engineers, and business stakeholders, to understand their data requirements and provide efficient data engineering solutions.
- Stay updated with the latest advancements in data engineering, data warehousing, and cloud technologies, and proactively propose innovative solutions to enhance data extraction and warehousing capabilities.
Requirements
- Minimum of 5 years’ experience in data engineering, with a strong focus on data extraction and cloud-based warehousing; a combination of years of experience and relevant advanced technology proficiency will also be considered.
- Proficiency with Snowflake and data integration tools like Fivetran.
- Advanced SQL skills and experience with ETL/ELT frameworks.
- Experience with scripting languages such as Python for data processing and automation.
- Solid understanding of data modeling and relational database design.
- Strong communication skills and the ability to collaborate with technical and non-technical stakeholders.
- Strong analytical and problem-solving skills, with the ability to identify and resolve complex data engineering challenges.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data extractiondata loadingdata transformationdata quality checksdata validationperformance optimizationdata securitydata warehousingSQLdata modeling
Soft skills
communicationcollaborationanalytical skillsproblem-solvingmentoring