Technical Customer Success Manager – AI Tuning
HeyMilo AI
full-time
Posted on:
Location Type: Hybrid
Location: Toronto • Canada
Visit company websiteExplore more
Tech Stack
About the role
- Work closely with customers and customer success to define and track success metrics
- Monitor performance of deployed AI agents and identify gaps
- Tune prompts, workflows, and agent behavior to improve outcomes
- Iterate on agent logic, tool usage, and context management
- Debug issues across prompts, integrations, and system behavior
- Translate real-world usage into product and system improvements
Requirements
- Strong technical background with ability to code when needed
- Deep experience with prompt engineering and LLM behavior
- Strong understanding of agent systems (tool calling, context management, orchestration)
- Heavy user of AI tools (Cursor, Claude, Codex, etc.)
- Ability to debug ambiguous issues across AI and system layers
- Strong ownership and communication skills
- Comfortable working under pressure and tight timelines
- Able to own outcomes, not just tasks
- Experience implementing agentic workflows or AI agents in enterprise environments
- Programming fundamentals + Python/JS experience are a huge plus
Benefits
- Direct ownership over how AI performs in the real world
- Work on high-impact problems tied to customer outcomes
- High visibility across product, engineering, and customer teams
- Fast-paced, execution-driven environment
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
prompt engineeringLLM behavioragent systemstool callingcontext managementorchestrationdebuggingPythonJavaScriptprogramming fundamentals
Soft Skills
communication skillsownershipworking under pressuretime management