Senior Data Engineer
Vytwo Technologies Inc
full-time
Posted on:
Location Type: Hybrid
Location: Minneapolis • Minnesota • United States
Visit company websiteExplore more
Job Level
About the role
- Design, develop, and maintain scalable data pipelines using Apache Spark (PySpark and/or Scala)
- Build and optimize data workflows on Databricks , including Delta Lake, notebooks, and scheduled jobs
- Ingest, transform, and curate large-scale structured and semi-structured datasets
- Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
- Implement data quality checks, monitoring, and error handling
- Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
- Support batch processing and, where applicable, streaming data pipelines
- Follow best practices for testing, documentation, security, and version control
Requirements
- 5+ years of experience in Data Engineering or related roles
- Strong hands-on experience with Apache Spark (PySpark or Scala)
- Proven experience working in Databricks environments
- Strong SQL skills and experience with relational and analytical databases
- Experience building and maintaining ETL/ELT pipelines at scale
- Familiarity with modern data lake architectures (Delta Lake preferred)
- Experience with Git-based version control
Benefits
- Flexible work from home options
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
Apache SparkPySparkScalaDatabricksDelta LakeSQLETLELTdata pipelinesdata quality checks
Soft Skills
collaborationcommunicationproblem-solvingattention to detailanalytical thinking