Senior Data Engineer

CCC Intelligent Solutions

Senior Data Engineer developing data pipeline applications for CCC's cloud-based insurance platform. Involves mentoring teams and working with various data technologies.

Posted 6/12/2026full-timeRemote • Illinois • 🇺🇸 United StatesSenior💰 $108,077 - $125,282 per yearWebsite

Tech Stack

Tools & technologies

AirflowAmazon RedshiftApacheAWSCloudETLHadoopHBaseHDFSKafkaPySparkPythonSparkSQLSubversionTerraformUnixYarn

About the role

Key responsibilities & impact

Senior Data Engineers for various and unanticipated worksites throughout the U.S. (HQ: Chicago, IL).
Develop large scale end to end data pipeline applications, covering multiple data sources spread across data center and AWS cloud.
Use developed software applications to locate and analyze source data; create data flows to extract, profile, and store ingested data; define and build data cleansing and imputation; map to a common data model; transform to satisfy business rules and statistical computations; and validate data content.
Produce software data building blocks, data models, and data flows, such as dimensional data, data feeds, dashboard reporting, and data science research and exploration.
Produce automated software tests of data flow components and for data content quality.
Automate orchestration and error handling for use by production operation teams.
Provide technical expertise to diagnose errors from production support teams.
Guide junior team members in performance tuning applications in distributed computing environments.
Perform root cause analysis on all data and processes and identify opportunities for improvement.
Develop metadata-driven and fully parameterized data processing tools.
Mentor junior engineers.

Requirements

What you’ll need

Master’s degree in Computer Science, Computer Engineering, Management Information Systems or related field plus 2 years of experience in software development/data processing or analysis required.
Hands-on experience with: Programming using Python & PySpark; Hadoop; HDFS, map-reduce, YARN, AWS EMR, Redshift, Terraform; Hive, HBase, parquet, ORC, Spark SQL, Sqoop, Apache Hudi; Orchestrating ETL pipelines involving data sourcing, transformations & publishing using Apache Airflow; Performance tuning applications in distributed computing environments; Designing & developing data pipeline applications with Apache Kafka; Advanced SQL for data profiling & data validation; Unix commands & scripting; performing root cause analysis on internal & external data & processes to identify opportunities for improvement; JIRA, Gitlab, Subversion; Development of metadata-driven & fully parameterized data processing tools; AWS.

Benefits

Comp & perks

401K Match
Paid time off
Annual Incentive Plan
Performance Bonus
Comprehensive health insurance
Adoption Assistance
Tuition Reimbursement
Wellness Programs
Stock Purchase Plan options
Employee Resource Groups

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonPySparkHadoopHDFSmap-reduceYARNAWS EMRRedshiftApache KafkaAdvanced SQL

Soft Skills

mentoringperformance tuningroot cause analysistechnical expertiseguiding junior team members

Certifications

Master’s degree in Computer ScienceMaster’s degree in Computer EngineeringMaster’s degree in Management Information Systems