Leidos

Senior Test and Evaluation Engineer – Agentic AI

Leidos

full-time

Posted on:

Origin:  • 🇺🇸 United States

Visit company website
✨ AI Apply
Manual Apply

Salary

💰 $104,650 - $189,175 per year

Job Level

Senior

Tech Stack

NumpyPandasPythonScikit-Learn

About the role

  • At Leidos, contribute to AI solutions for defense, intelligence, healthcare, energy, and space exploration emphasizing Trusted Mission AI that is transparent, ethical, resilient, and accountable.
  • Work with multidisciplinary teams to transition AI research into operational environments where accuracy, security, and reliability are essential.
  • Develop and operationalize test and evaluation CONOPS, scalable Test and Model Evaluation plans for Agentic AI systems including process, techniques and tools.
  • Work with AI scientists, data scientists and data engineers to develop test procedures for positive and negative testing and evaluation and collect performance metrics.
  • Integrate testing tools and procedures with the CI/CD pipeline with MLOps and SecDevOps engineers.
  • Analyze existing processes and metrics to recommend improvements and collaborate with AI Governance to maintain visibility and explainability through testing.
  • Implement testing process in the AI system design, development and deployment life cycle and identify testing risks and limitations.

Requirements

  • Bachelor's degree in Computer Science, Data Science or related field and over 8 years of relevant experience, Masters with 6 years experience. Additional experience may be considered in lieu of degree.
  • Strong Python programming fundamentals
  • Experience with system and subsystem level test process and automation
  • Experience with creating user acceptance test scenarios
  • Experience with SecDevOps tooling and MLOps pipeline development
  • Experience with software test automation techniques
  • Experience with AI Performance and vulnerability assessment
  • AI model assurance evaluation
  • Experience applying and automating AI interpretability & explainability tools and methods
  • Experience with developing CONOPS and presentations
  • Good understanding of machine learning algorithms, tools and platforms
  • Self-starter with high intellectual curiosity
  • Great communication skills, able to explain model and test results to a non-technical audience
  • Proficient in data exploration techniques and tools
  • Preferred: Experience with data visualization libraries such as Plotly, Streamlit, and matplotlib
  • Preferred: Experience with AI/ML tools, such as common Python packages (e.g., scikit-learn, NumPy, Pandas) and Jupyter notebooks
  • Preferred: Experience with database administration and data repositories
  • Preferred: Experience with building LLM and other Generative AI applications
  • Ability and willingness to obtain a Top Secret security clearance