Featherless AI

Software Engineer – API Gateway

Featherless AI

full-time

Posted on:

Location Type: Remote

Location: Canada

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

CloudDNSKubernetesMongoDBNode.jsPythonRedis

About the role

  • support and evolve the API gateway to our inference cloud, which is responsible for
  • authentication and inference to all models
  • subscription management and subscription entitlement (e.g. context-length, concurrency limits)
  • providing the necessary API surface for applications and builders
  • undertake feature development and bug fixes to keep up with clients, resolve user issues, and onboard new models
  • improve the reliability of the existing API (increasing instrumentation and monitoring, right-sizing infrastructure)
  • respond to availability incidents
  • triage and resolve issues of inference quality and reliability
  • manage the infrastructure on which our gateway runs

Requirements

  • first-hand experience of the user’s we’re building for (familiarity with popular open LLMs, common clients, and experience building with LLM)
  • experience with the technologies and paradigms of the web (REST, websockets, DNS, networking, opentelemetry)
  • experience with significant components of our stack (k8s, node, mikro-orm, fastify, redis, mongodb, python, elastic cloud, cloudflare, sentry, otel)
  • ability to debug complex issues across a wide stack and build instrumentation as necessary
  • desire to work collaboratively as part of a skilled team
  • Alignment with team and company values, including
  • bias to action
  • responsiveness to users (bug-fixes over features)
  • instinct to iterate
  • subscribing to that done means proven by usage data
Benefits
  • This team operates on Eastern Time
  • Remote preference to hire in Toronto, Canada

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
API developmentbug fixingfeature developmentdebugginginstrumentationmonitoringinfrastructure managementsubscription managementinference qualityreliability improvement
Soft skills
collaborationresponsivenessbias to actioniterationuser focus