Staff Software Engineer, AI Runtime
Apollo GraphQL
ApolloCloudDistributed Systems
Job Level
Lead
Tech Stack
ApolloAWSCloudDistributed SystemsPython
About the role
- End-to-End Agentic Systems:
- Autonomous AI Agents: Architect and lead the development of multi-agent systems capable of long-horizon planning, reasoning, and API orchestration.
- Workflow Automation: Build reusable agentic components that integrate deeply into sales and marketing processes.
- LLM Platformization: Own and evolve our in-house platform for scalable, low-latency, and cost-efficient LLM and agent deployments.
- AI Assistants and Search:
- Conversational AI & UI: Lead design of interfaces powered by natural language understanding and retrieval-augmented generation (RAG).
- Semantic & Personalized Search: Build embedding-based, intent-aware search and personalization systems tuned to business user needs.
- Email Intelligence: Drive innovation in personalized outreach generation using context-aware generation pipelines.
- Production-Grade Applied AI:
- Latency & Cost Optimization: Tune inference pipelines, caching layers, and model selection logic for high-scale, cost-aware performance.
- Evaluation at Scale: Define and drive robust offline and online testing methodologies (A/B, sandboxing, human evals) across agents and LLM flows.
- Feedback Loops: Architect human-in-the-loop systems and telemetry to improve accuracy, UX, and explainability over time.
Requirements
- 10+ years of software engineering experience, with at least 3 years in applied LLM or agentic AI systems (2023–present).
- Proven success in deploying LLM-powered products used by real users at scale, not just prototypes or internal tools.
- Deep backend & systems engineering expertise with Python, distributed systems, and scalable APIs.
- Familiarity with LangChain, LlamaIndex, or similar orchestration frameworks.
- Experience with RAG pipelines, vector DBs, embedding models, and semantic search tuning.
- Experience managing performance across cloud providers (e.g., AWS Bedrock, OpenAI, Anthropic, etc.).
- Demonstrated experience building multi-step agents, planning workflows, chaining reasoning steps, and integrating APIs with agent memory/state.
- Comfort with advanced prompting strategies, few-shot and chain-of-thought reasoning, and embedding retrieval setups.
- Strong understanding of AI system evaluation, human ratings, A/B experimentation, and feedback loop pipelines.
- Experience designing safety-aware, reliable LLM systems in production environments.
- Experience owning logging, monitoring, and observability for live AI systems.
- Principal-Level Ownership: You thrive in an ambiguous environment, define company wide roadmaps, drive the most important Engineering decisions, and mentor others. You lead from the front.
- AI-Native Mentality: You leverage AI to ship faster and smarter, and champion automation across engineering workflows.
- Applied Focus: You prioritize impact over novelty. You’re deeply pragmatic in your application of AI research to product features.
