Senior Software Engineer II – AI Engineering

Smartsheet

Senior AI Engineer building production LLM agents and evaluation systems for Smartsheet’s enterprise work-management platform. Improving SmartAssist quality across cloud infrastructure and full-stack applications.

Posted 9/3/2026full-timeRemote • 🇺🇸 United StatesSenior💰 $175,000 - $245,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and building LLM-powered agents and workflows, with a strong focus on prompt engineering, RAG architectures, and cloud-based AI systems. Proven ability to mentor engineers and establish best practices for AI development and model governance.

Highest-signal resume keywords

LLM Production ExperiencePrompt EngineeringRAG Architecture KnowledgePython ProgrammingCloud AI Systems Management

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software EngineeringPrompt EngineeringRAG ArchitectureLLM Evaluation FrameworksAgent System DesignPythonData EngineeringAI Development Best PracticesModel GovernanceQuality Improvement Methodology

Soft Skills

Cross-Functional CollaborationComplex CommunicationJudgment in Ambiguity

Tools & Technologies

AWSGCPAzureDatabricksDelta TablesMLflowCI-Integrated Evaluation PipelinesMulti-Agent Orchestration Frameworks

Certifications & Qualifications

BS in Computer ScienceMS in Computer Science

Industry Keywords

Applied AILLMOpsModel EvaluationQuality AssuranceAI Systems

Tech Stack

Tools & technologies

AWSAzureCloudGoogle Cloud PlatformPython

About the role

Key responsibilities & impact

Design and build production-grade LLM-powered agents and workflows within Smartsheet
Make architecture decisions on system design, data pipelines, and deployment strategies
Develop and optimize prompts, RAG pipelines, and agent reasoning patterns
Build evaluation frameworks to measure accuracy, hallucination rates, and performance across model versions
Implement and manage end-to-end AI systems on AWS, GCP, or Azure
Monitor and optimize cloud-based AI systems and respond to incidents
Collaborate with Engineering, Product, and cross-functional teams to translate business requirements into technical AI solutions
Mentor and guide engineers on applied ML best practices, modern AI development patterns, and production considerations
Establish best practices and standards for AI development, model governance, and responsible AI
Evaluate emerging AI/LLM tools, frameworks, and techniques
Contribute to full-stack applications, both legacy and greenfield
Deliver measurable, validated quality improvements on SmartAssist agents
Expand evaluation coverage and establish a repeatable quality improvement methodology

Requirements

What you’ll need

8+ years of software engineering experience, with at least 2 years working directly with LLMs in production
Deep, hands-on experience with prompt engineering and context engineering
Strong working knowledge of RAG architectures, including chunking strategies, embedding models, retrieval evaluation, and failure diagnosis
Experience building or extending LLM evaluation frameworks, including designing scorers and working with golden datasets
Fluency in agent system design and architectural tradeoffs
Strong Python skills
Comfortable working in data-heavy environments such as Databricks and Delta tables, or equivalent
Ability to communicate complex quality findings to technical and non-technical stakeholders
Strong cross-functional judgment
Bias for clarity in ambiguous situations
Legally eligible to work in the U.S. on an ongoing basis
BS or MS in Computer Science, a related field, or equivalent industry experience
Strong plus: experience with MLflow or similar experiment tracking platforms
Strong plus: familiarity with CI-integrated evaluation pipelines
Strong plus: experience with multi-agent orchestration frameworks
Strong plus: prior work in an Applied AI or LLMOps function within a product company

Benefits

Comp & perks

Employer subsidized medical/vision and dental coverage for full-time employees
401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay)
Monthly stipend to support your work and productivity
Flexible Time Away Program, plus Sick Time Off
US employees are automatically covered under Smartsheet-sponsored life insurance, short-term, and long-term disability plans
US employees receive 12 paid holidays per year
Up to 24 weeks of Parental Leave
Personal paid Volunteer Day to support our community
Opportunities for professional growth and development including access to Udemy online courses
Company Funded Perks, including a counseling membership, local retail discounts, and your own personal Smartsheet account
Teleworking options from any registered location in the U.S. (role specific)
Market competitive incentive opportunity