
Staff Software Engineer, Inference Infrastructure
Cohere
full-time
Posted on:
Location Type: Hybrid
Location: San Francisco • California • United States
Visit company websiteExplore more
Job Level
About the role
- Join a mission to scale intelligence and serve humanity by building AI systems
- Work closely with technical teams to deploy optimized NLP models to production
- Interface with customers to create customized deployments
Requirements
- 5+ years of engineering experience running production infrastructure at a large scale
- Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
- Experience with Kubernetes dev and production coding and support
- Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
- Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
- Experience in compute/storage/network resource and cost management
- Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
- The grit and adaptability to solve complex technical challenges that evolve day to day
- Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
- Strong understanding or working experience with distributed systems.
- Experience in Golang, C++ or other languages designed for high-performance scalable servers).
Benefits
- An open and inclusive culture and work environment
- Work closely with a team on the cutting edge of AI research
- Weekly lunch stipend, in-office lunches & snacks
- Full health and dental benefits, including a separate budget to take care of your mental health
- 100% Parental Leave top-up for up to 6 months
- Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- 6 weeks of vacation (30 working days!)
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
NLP modelsKubernetesGCPAzureAWSOCILinux-based computing environmentsGolangC++distributed systems
Soft skills
collaborationtroubleshootingadaptabilityproblem-solvingteamwork