Senior Software Engineer – Applied AI

Mainstay

Senior Software Engineer working on AI-powered product experiences in real estate. Collaborating with teams to optimize models and build backend systems for production-ready features.

Posted 5/8/2026full-timeRemote • 🇺🇸 United StatesSeniorWebsite

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

Contribute to all phases of product development, from ideation and prototyping to implementation and iteration
Build and evolve both frontend product experiences and backend services that power them
Fine-tuning existing models and optimize their performance for Mainstay use cases
Design reliable, scalable backend services to support AI-powered features in production
Leverage AI to modernize and automate legacy workflows in the real estate industry
Evaluate and benchmark LLM performance across latency, cost, accuracy, and hallucination rates
Tune prompts, chain-of-thought logic, and implement retrieval strategies such as RAG and vector stores
Monitor performance, safety, and cost of LLM-driven features in production environments
Define and build useful LLM-powered experiences in partnership with cross-functional teams
Mentor and support the growth of other engineers on the team
Collaborate closely with product managers, designers, and operators to identify and solve customer pain points

Requirements

What you’ll need

5+ years of experience building and delivering production-grade software, ideally in fast-paced, high-growth startup environment
At least 3 years of hands-on experience developing and deploying production-ready AI features (LLM applications, fine-tuning, RAG systems, guardrails, evals, etc.)
Strong full-stack engineering skills with a focus on backend scalability and system reliability
Experience fine-tuning and deploying LLMs
Deep understanding of LLM evaluation methods, including prompt testing, retrieval precision, hallucination detection, and cost/performance tradeoffs
Experience building custom eval pipelines using techniques like model-as-a-judge or embedding similarity
Hands-on work with agentic frameworks (e.g. LangGraph, AutoGen, CrewAI) or custom agent systems
Clear, concise communicator - essential for our remote, highly collaborative team
Proactive and ownership-driven - comfortable navigating ambiguity and shifting priorities
Customer-focused mindset with a drive to build solutions that solve real problems
Experience with AI/ML tools or APIs; bonus if you’ve worked with Python-based RAG, voice agents, or AI task automation.

Benefits

Comp & perks

Trust is at the core of everything we do.
We build trust with everyone -- from candidates to employees to partners.
Providing reasonable accommodations throughout the recruitment process for candidates with disabilities, pregnancy, religious beliefs, or other reasons protected by applicable laws.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

production-grade softwareAI featuresLLM applicationsfine-tuningRAG systemsbackend scalabilitysystem reliabilityLLM evaluation methodscustom eval pipelinesagentic frameworks

Soft Skills

clear communicationproactiveownership-drivencustomer-focusedcollaborative