Senior QA Operations Associate
User Interviews
PythonSQL
Salary
💰 $200,000 - $250,000 per year
Job Level
Lead
Tech Stack
PythonSQL
About the role
- Build and scale a multidisciplinary AI team (AI engineers, data scientists, applied researchers) and own the Zinnia AI vision
- Mentor team members in growing their career and skill set and enabling others to build with AI
- Drive AI strategy: Independently own and drive key AI initiatives across the business, proactively identifying areas for efficiency improvements and automation as well as product opportunities
- Advanced analytics & measurement: Move beyond basic dashboards and leverage inferential modeling, causal analysis, and experimental design to generate actionable insights
- Experimentation & testing: Design and implement A/B tests to measure the impact of AI deployments, optimizing key processes such as processing, fraud detection, customer interactions, and compliance
- Provide hands-on guidance of Gen AI based frameworks and architecture from simple knowledge retrieval to fully agentic systems
- Cross-functional collaboration: Work closely with stakeholders across Operations, Data Engineering, and Product to align AI initiatives with business needs
- Scalability & automation: Ship full end to end systems to production that can work standalone or part of our larger product platform builds
- Thought leadership & best practices: Drive AI best practices and keep up with advancements in the AI space. Work to drive a Federated AI strategy across Zinnia
- Lead the build vs buy strategy for AI to help us maximize partners where appropriate and building IP to drive a competitive advantage
- Communicate AI systems in plain language and work with teams on change management to maximize adoption
Requirements
- 15+ years of total technology experience with 5+ years of people management
- 7+ years of experience in data science, machine learning, artificial intelligence or software engineering, with a focus on artificial intelligence and machine learning
- Expertise in the AI focused tech stack including SQL and relational databases, Python, RAGs, knowledge graphs, learning systems, unstructured datasets and document stores, LLMOps and ML Ops tools (LangGraph, LangChain, Sagemaker), agentic based AI frameworks (LangGraph, CrewAI), multimodal AI (voice, text to speech, speech to text, OCR)
- Must be able to wrangle and clean datasets as well as join together disparate datasets
- Expertise of programmatic testing of machine learning and Gen AI based models for bias and performance
- Experience designing frameworks to provide demonstrable value of AI based systems
- Strong statistical knowledge of the inner workings of Gen AI based models and machine learning and deep learning models
- Experience in bringing AI based systems to production from scoping to testing to production monitoring
- Ability to work independently, take ownership of projects, and influence business decisions through data-driven recommendations
- Strong problem-solving skills and a proactive mindset to identify business opportunities using data
- Communication Skills: Independently prepares written and oral communication and can effectively deliver. Can explain how AI systems work and take feedback from SMEs on issues and performance requirements. Can train end users on how to utilize AI based systems.
- Knowledge of AI regulations in the insurance space and familiarity with ethical and responsible uses of AI as well as production monitoring of performance, explainability and ethics
