Data Engineer – Purchasing Power
Progressive Leasing
full-time
Posted on:
Location Type: Hybrid
Location: Atlanta • United States
Visit company websiteExplore more
About the role
- Data Pipeline Development: Develop and maintain streaming and batch data pipelines using Kafka, Spark, and related technologies
- Implement ingestion pipelines using Kafka Connect and CDC patterns
- Build transformation and enrichment processes using Spark and PySpark
- Ensure reliable data delivery to downstream platforms
- Streaming Platform Support: Support Kafka topic configuration, partitioning strategies, and consumer group management
- Troubleshoot streaming pipeline issues and optimize performance
- Help maintain data integrity and reliability across distributed systems
- Data Processing & Transformation: Implement business logic and transformations within Spark processing frameworks
- Develop and maintain Airflow DAGs (or equivalent orchestration workflows)
- Ensure data quality through validation, monitoring, and alerting practices
- Platform Operations: Monitor pipeline performance, failures, and data latency
- Assist with observability and monitoring using tools such as Elastic and Kibana
- Support incident resolution and root‑cause analysis for data pipeline issues
Requirements
- 3–5+ years of experience in data engineering or distributed data systems
- Hands‑on experience building and maintaining pipelines using Spark / PySpark
- Experience working with Kafka or other streaming platforms
- Strong SQL skills and experience with data transformation
- Experience with data orchestration tools such as Apache Airflow
- Familiarity with AWS‑based data environments
- Strong troubleshooting and debugging skills for data pipelines
- Curiosity, adaptability, and comfort working in evolving technical environments
Benefits
- Hybrid work model (Atlanta‑based)
- Comprehensive benefits: medical, dental, vision, company‑paid Basic Life & AD&D
- 401(k) Retirement Plan
- Flexible Paid Time Off
- Career Development opportunities
- Employee Purchase Program
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data pipeline developmentstreaming data pipelinesbatch data pipelinesKafkaSparkPySparkSQLdata transformationApache AirflowCDC patterns
Soft Skills
troubleshootingdebuggingcuriosityadaptability