Cohere

Site Reliability Engineer – Inference Infrastructure

Cohere

full-time

Posted on:

Location Type: Remote

Location: Canada

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AWSAzureCloudDistributed SystemsGoGoogle Cloud PlatformKubernetesLinux

About the role

  • Build self-service systems that automate managing, deploying and operating services.
  • This includes our custom Kubernetes operators that support language model deployments.
  • Automate environment observability and resilience. Enable all developers to troubleshoot and resolve problems.
  • Take steps required to ensure we hit defined SLOs, including participation in an on-call rotation.
  • Build strong relationships with internal developers and influence the Infrastructure team’s roadmap based on their feedback.
  • Develop our team through knowledge sharing and an active review process.

Requirements

  • 5+ years of engineering experience running production infrastructure at a large scale
  • Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
  • Experience with Kubernetes dev and production coding and support
  • Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
  • Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
  • Experience in compute/storage/network resource and cost management
  • Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
  • The grit and adaptability to solve complex technical challenges that evolve day to day
  • Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
  • Strong understanding or working experience with distributed systems.
  • Experience in Golang, C++ or other languages designed for high-performance scalable servers).
Benefits
  • An open and inclusive culture and work environment
  • Work closely with a team on the cutting edge of AI research
  • Weekly lunch stipend, in-office lunches & snacks
  • Full health and dental benefits, including a separate budget to take care of your mental health
  • 100% Parental Leave top-up for up to 6 months
  • Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
  • Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
  • 6 weeks of vacation (30 working days!)

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
KubernetesGolangC++LinuxGCPAzureAWSOCIdistributed systemsGPU workloads
Soft skills
collaborationtroubleshootinginfluenceknowledge sharingadaptabilityproblem-solvingcommunicationteamworkrelationship buildingreview process