Design and develop scalable, resilient offline indexing pipelines for Slack's search infrastructure
Work with product engineering on new features for a growing user base
Optimize data processing pipelines for performance, cost-efficiency, and reliability
Collaborate with team members to ensure high availability and data freshness across distributed systems
Actively own indexing pipeline features and improve surrounding data infrastructure
Monitor and optimize pipeline performance while addressing data quality issues
Mentor other engineers and review code
Improve engineering standards, tooling, and processes

Requirements

7+ years of experience in building web applications and operating large-scale distributed systems
Experienced with distributed data processing frameworks such as Spark, Hadoop, Map/Reduce, or EMR
Deep experience with workflow orchestration tools like Airflow
Proficient in languages commonly used for data engineering (e.g., Python, Java, Scala)
Track record of developing high-availability, performant data pipelines across multiple clusters and regions
Strong communication skills for explaining complex technical concepts to diverse teams
Computer science fundamentals: data structures, algorithms, programming languages, distributed systems, and information retrieval
Bachelor's degree in Computer Science, Engineering or related field, or equivalent

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data processing pipelinesdistributed systemsoffline indexingdata engineeringworkflow orchestrationdata structuresalgorithmsPythonJavaScala

Soft Skills

communicationmentoringcollaborationproblem-solvingleadership