Senior Data Engineer
CGWS - COME GROW WITH US
full-time
Posted on:
Location Type: Hybrid
Location: Utah • Utah • United States
Visit company websiteExplore more
Job Level
About the role
- Collaborate with stakeholders to make effective use of core data assets
- Development and maintenance of data integrations written in Python
- Load both streaming and batched data in Spark with Pyspark and lakehouse libraries
- Engineer lakehouse models to support defined data patterns and use cases
- Build unit, integration, quality, and performance tests
- Participate in peer code reviews, code approvals, and pull requests
- Monitor, operate, and optimize data integrations, pipelines, and distributed applications
- Documentation of data pipelines, cloud infrastructure, and standard operating procedures
- Work within an IT managed AWS account and VPC to stand up and maintain data platform development, staging, and production environments
- Express data platform cloud infrastructure, services, and configuration as code
- Automate continuous upgrades and testing of data platform infrastructure and services
- Help ensure appropriate data privacy and security
- Identify, recommend, and implement opportunities for improvement in efficiency, resilience, scale, security, and performance
Requirements
- Python development abilities, skills, and experience
- Expert level SQL coding abilities and experience
- Ability to quickly learn new development frameworks
- Ability to develop, scale, and tune Spark and PySpark applications
- Understanding of data lake, lakehouse, and data warehouse systems, and related technologies
- Knowledge and understanding of data formats, data patterns, models, and methodologies
- Ability to work with cloud infrastructure including resource scaling, S3, RDS, IAM, security groups, AMIs, cloudwatch, cloudtrail, and secrets manager
- Experience with CI/CD in a data engineering environment
- Experience managing access controls in a data engineering environment
- Git-based team coding workflows
Benefits
- Comprehensive health, life, and disability insurance
- Generous leave policies that include 4 weeks of vacation, 12 company holidays, parental leave, and volunteer time off so you can enjoy quality of life
- 401k plans with up to 6% company match
- $2000 Paid-Paid Vacation bonus
- EAP through Headspace
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
PythonSQLSparkPySparkdata integrationdata modelingunit testingintegration testingperformance testingCI/CD
Soft Skills
collaborationproblem-solvingcommunicationpeer code reviewdocumentationefficiency improvementsecurity awareness