Design and optimize high-performance data pipelines for distributed training and storage (using tools like Arrow, DuckDB, LanceDB, BigQuery, vector databases).
Focus on low-level optimizations (latency, throughput, reliability, GPU usage).
Build monitoring and visualization tools for tracking data quality, pipeline performance, and experiments.
Optimize distributed AI workloads for reliability, latency, and efficiency.
Scope and supervise projects so that interns, PhD students, and post-docs can contribute and collaborate effectively.
Support recruiting efforts and help shape the growth of the infrastructure team.

Requirements

5+ years of backend or infrastructure engineering experience
Strong Python programming skills (bonus points for lower-level languages)
Experience with distributed systems and cloud platforms (AWS, GCP, Azure)
Hands-on experience with containerization (Docker, Kubernetes) and infrastructure as code (Terraform)
Experience building or supporting ML/AI infrastructure in production
Experience with high-performance data tools (DuckDB, Apache Spark, Delta Lake)
GPU orchestration and large-scale model training experience
Familiarity with ML platforms (SageMaker, Vertex AI) and frameworks (PyTorch, JAX)
Experience mentoring junior engineers, interns, or researchers and breaking down complex projects into manageable tasks
Experience participating in technical hiring processes and evaluating candidates

Benefits

📊 Check your resume score for this job Improve your chances of getting an interview by checking your resume score before you apply. Check Resume Score

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Pythondistributed systemscloud platformsDockerKubernetesTerraformDuckDBApache SparkDelta LakeGPU orchestration

Soft Skills

mentoringproject supervisioncollaborationtechnical hiringevaluating candidates