ICF

Senior Data Engineer

ICF

full-time

Posted on:

Location Type: Remote

Location: Remote • Virginia • 🇺🇸 United States

Visit company website
✨ AI Apply
Apply

Salary

💰 $89,203 - $151,646 per year

Job Level

Senior

Tech Stack

AirflowAWSCloudPySparkPythonSparkSQL

About the role

  • Design, develop, and maintain scalable data pipelines using Spark, Hive, and Airflow
  • Develop and deploy data processing workflows on the Databricks platform
  • Develop API services to facilitate data access and integration
  • Create interactive data visualizations and reports using AWS QuickSight
  • Build required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies
  • Monitor and optimize the performance of data infrastructure and processes
  • Develop data quality and validation jobs
  • Assemble large, complex sets of data that meet non-functional and functional business requirements
  • Write unit and integration tests for all data processing code
  • Work with DevOps engineers on CI, CD, and IaC
  • Read specs and translate them into code and design documents
  • Perform code reviews and develop processes for improving code quality
  • Improve data availability and timeliness by implementing more frequent refreshes, tiered data storage, and optimizations of existing datasets
  • Maintain security and privacy for data at rest and while in transit

Requirements

  • Bachelor's degree in computer science, engineering or related field
  • 7+ years of hands-on software or data development experience
  • 4+ years of data pipeline experience using Python, PySpark and cloud technologies
  • 2 years working in Spark and Hive or similar large data environments
  • Candidate must be able to obtain and maintain a Public Trust clearance
  • Candidate must reside in the US, be authorized to work in the US, and work must be performed in the US
  • Must have lived in the US 3 full years out of the last 5 years
Benefits
  • Health insurance
  • 401(k) matching
  • Paid time off
  • Flexible work hours
  • Professional development opportunities

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data pipelinesSparkHiveAirflowDatabricksAPI servicesAWS QuickSightSQLPythonPySpark
Soft skills
code reviewscommunicationproblem-solvingcollaborationattention to detail
Certifications
Bachelor's degree in computer sciencePublic Trust clearance