
Senior Data Engineer
EMC Insurance Companies
full-time
Posted on:
Location Type: Hybrid
Location: Des Moines • Iowa • United States
Visit company websiteExplore more
Salary
💰 $103,954 - $164,309 per year
Job Level
Tech Stack
About the role
- Develops, test, and maintains prioritized engineering (design/build/test) work as a member of a data analytics product pipeline architecture and infrastructure team
- Ensures delivery of solutions for organization of data, such as transformation, conformation, integration, and aggregation of data
- Partners with the Data Platform Operations (DPOs), Information Security, Network, and Operations teams to design and implement new data pipelines for new business initiatives as related to data and analytics
- Serves as a technical expert for the Data Engineering team
- Ensures best practices and internal processes are followed when solutions are designed and implemented
- Ensures the delivery of solutions for acquisition of data, such as landing, staging, rapid ingestion, and historical data extraction
- Creates processes framework, design and implementation of large-scale data conversion and data ingestion on various cloud and on-premise platforms using most optimal processing and programming techniques
- Investigates and analyzes feasibility of data integration prototypes and program requirements
- Develops specifications and plans the full range of programming actions to produce data integration components
- Develops and maintains highly complex automated ETL/ELT pipeline architecture that cleans, transforms, and aggregates data from disparate sources using architecturally approved tools, programming languages, and services to deliver a data mesh approach
- Develops and maintains highly complex automated ETL/ELT monitoring and alarming solutions using architecturally approved languages and services
- Monitors, troubleshoots, and optimizes data pipelines for quality, performance, and reliability creating a single source of truth providing business customers with highly reliable and quality data
- Builds highly complex unit tests to enable test driven development and automation to drive code reliability
- Develops, maintains, tests, and deploys code with architecturally approved toolsets using Dev Ops practices, ETL/ELT pipelines, data catalogs and data queries
- Serves as a technical resource and supports team members on EMC’s data integration projects and helps them learn new technologies
- Leads evaluation of new technologies including performing POCs and presenting results to others, with a goal of providing technical recommendations
- Balances and designs data analytics product solutions based on cost and performance to meet business requirements
- Guides business customers to develop, troubleshoot, and optimize complex SQL and ETL/ELT solutions to solve reporting, metrics, and analytics problems.
Requirements
- Bachelor’s degree, preferably in a computer related field or equivalent relevant experience
- Eight years of data integration development experience, or related experience
- Excellent SQL Knowledge and experience working with relational databases
- Excellent analytics skills to work with unstructured datasets preferred
- Excellent knowledge of applicable programming languages such as Python
- Strong knowledge of cloud services such as RDS, S3, SNS, CloudWatch and Lambda
- Advanced knowledge of DBT/Informatica or other ETL/ELT tools preferred
- Excellent knowledge of Snowflake
- Excellent knowledge of data modeling
- Excellent ability to build processes supporting data transformation, data structures, metadata, dependency and workload management
- Excellent knowledge of orchestration or scheduling tools
- Excellent ability to manipulate, process and extract value from large, disconnected datasets
- Advanced knowledge of DevOps and/or DataOps practices, CI/CD pipelines, and containerization technologies preferred
- Excellent verbal and written communication skills
- Strong attention to detail, organizational and multi-tasking skills required with the ability to adapt to changing priorities
- Ability to maintain confidentiality.
Benefits
- Health insurance
- 401(k) matching
- Paid time off
- Volunteer time off annually
- Tuition reimbursement
- Wellness initiatives to improve team member well-being and reduce health insurance costs
- Flexibility to dress for your day and opportunities for alternative work arrangements
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
SQLPythonETLELTdata modelingdata transformationdata integrationDevOpsDataOpsCI/CD
Soft Skills
communicationattention to detailorganizational skillsmulti-taskingadaptability