Senior AI Application Engineer

DISHER

Senior AI Application Engineer responsible for evolving RAG layer and agentic AI capabilities in a global supply chain solutions organization.

Posted 7/7/2026full-timeRemote • 🇺🇸 United StatesSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing AI systems, particularly in building and optimizing RAG pipelines and AI agents. Proficient in integrating these systems with enterprise tools and ensuring performance and compliance through robust evaluation frameworks.

Highest-signal resume keywords

RAG System DevelopmentHybrid Search OptimizationAI Agent DesignPrompt EngineeringAzure AI Search

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

RAG SystemsHybrid SearchEvaluation FrameworksAI AgentsLLMsEmbeddingsVector DatabasesInformation RetrievalPrompt EngineeringSystem Optimization

Soft Skills

Self-StarterOwnershipCollaboration

Tools & Technologies

Azure AI SearchElasticsearchAPIsCRM SystemsERP SystemsTicketing Systems

Industry Keywords

AI GovernanceGenerative AIAgentic WorkflowsPerformance OptimizationContext Retrieval

Tech Stack

Tools & technologies

AssemblyAzureElasticSearchERPFlash

About the role

Key responsibilities & impact

Own the end-to-end RAG pipeline, from data ingestion to response generation
Design and implement Azure AI Search indexing strategies, including schema design, scoring profiles, and performance optimization
Develop and refine chunking and metadata strategies to maximize retrieval relevance and context quality
Tune retrieval pipelines (hybrid search, vector search, filters, reranking) for accuracy, latency, and cost
Build and maintain prompt orchestration frameworks, including dynamic prompt assembly and context injection
Ensure strong grounding and citation mechanisms so model outputs are traceable and verifiable
Design and implement AI agents and agentic workflows, including tool use, planning, and multi-step task execution
Integrate agents with internal and external systems (APIs, databases, SaaS tools) to enable real-world actions
Define and implement evaluation frameworks (offline & online) to measure retrieval quality, agentic performance, and answer correctness
Design and enforce guardrails (safety, hallucination reduction, policy compliance) across generation and agentic behavior
Collaborate with product, data, and platform teams to integrate AI capabilities into applications
Monitor system performance and continuously iterate based on user feedback and metrics
Partner with business and technology leadership to identify, evaluate, and prioritize agentic AI opportunities across Flash's operations
Translate business problems into AI system designs - scoping what's feasible, defining the data requirements, and owning delivery
Build repeatable patterns and internal tooling that accelerate future AI development across the organization
Contribute to Flash's evolving AI governance and responsible use practices

Requirements

What you’ll need

Experience building or maintaining RAG systems in production environments
Experience with hybrid search (keyword + vector) and reranking models
Knowledge of evaluation frameworks (e.g. RAGAS, LLM-as-a-judge approaches)
Experience integrating agents with enterprise tools (CRM, ERP, ticketing systems, etc.)
Experience optimizing systems for scale, latency, and cost
Strong knowledge of search technologies (Azure AI Search, Elasticsearch, or similar)
Hands-on experience with LLMs, embeddings, and vector databases
Experience designing and deploying AI agents or agentic systems
Demonstrated ability to integrate agents with enterprise systems - ERP, CRM, ticketing, or similar operational platforms
Understanding of information retrieval concepts (ranking, relevance, recall/precision tradeoffs)
Experience with prompt engineering and LLM orchestration tools
Familiarity with evaluation techniques for generative AI and agent systems
Ability to translate ambiguous business problems into concrete AI system designs
Ability to balance experimentation with production reliability
Hands-on experience building AI agents and agentic workflows, including tool use, multi-step reasoning, and orchestration across APIs and services
Understand how to design systems that can plan, retrieve context, take actions, and operate within defined guardrails in production environments
Proven ability to build effective, reliable AI agents that complete real-world tasks safely and accurately
Ability to provide measurable improvements in retrieval relevance and system performance
Previous experience creating robust guardrails and evaluation systems that catch issues early
Experience building scalable, maintainable architecture integrated into core products
Self-starter that thrives on ownership
Must have the current and continuing right to work in the United States of America without restrictions or expirations

Benefits

Comp & perks

Supportive, collaborative environment
Opportunities for technical and personal advancement
Focus on work-life balance and professional development