Pfizer

Director, AI Quality Engineering Lead

Pfizer

full-time

Posted on:

Location Type: Office

Location: Thessaloniki • 🇬🇷 Greece

Visit company website
✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AWSAzureCloudGrafanaPython

About the role

  • Ensure that AI systems across the enterprise are safe, fair, reliable, compliant, and observable
  • Design and operate enterprise evaluation frameworks (quality, bias/fairness, robustness, privacy)
  • Lead red-teaming and guardrail development
  • Encode policy and regulatory requirements as automated checks
  • Drive metric reporting and post‑incident learning across AI programs
  • Partner with Enterprise Architecture, Compliance, Legal/Privacy, Security, Data Governance, and product/engineering teams
  • Manage a team of QA Engineers and AI Engineers
  • Design and deliver robust, scalable solutions that demonstrate the value of AI across the enterprise in a safe and compliant way

Requirements

  • 7+ years in ML/AI/Platform/Quality roles, including 3+ years evaluating production systems.
  • Advanced degree in Computer Science, Engineering or related discipline.
  • Proven design & operation of evaluation harnesses and policy-as-code checks in CI/CD.
  • Hands-on Python and expertise with evaluation and safety tooling such as: OpenAI Evals, Ragas/LangSmith, DeepEval, Azure AI Content Safety / OpenAI Moderation, Presidio, Guardrails.ai, Nemo, MLFlow, Grafana, Kibana, Azure monitor Cloud & MLOps: Azure/AWS, CI/CD (GitHub Actions/Azure DevOps, feature stores, vector DBs (Azure AI Search, OpenSearch, Pinecone), testing in pipelines.
  • Excellent communication with executives and engineers
  • Experience leading an engineering team.
Benefits
  • Health insurance
  • Flexible working hours
  • Professional development opportunities

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
machine learningartificial intelligencequality assuranceevaluation frameworkspolicy-as-codePythonCI/CDMLOpsdata governancescalable solutions
Soft skills
communicationleadershipteam managementcollaborationproblem-solvingcritical thinkingorganizational skillspost-incident learningstakeholder engagementstrategic thinking
Certifications
advanced degree in Computer Scienceadvanced degree in Engineering