Director, AI Experimentation – Measurement

Pfizer

Director of AI Experimentation & Measurement driving AI experiment standards and governance for Pfizer. Leading proof definitions and evidence narratives across a portfolio of AI initiatives.

Posted 7/18/2026full-timeNew York City • New York, Pennsylvania • 🇺🇸 United StatesLead💰 $162,900 - $271,500 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in experiment design, causal inference, and measurement science, with a strong ability to synthesize complex data into actionable insights. Proven experience in leading evidence-based decision-making processes and communicating findings effectively to senior stakeholders.

Highest-signal resume keywords

Experiment DesignCausal InferencePower AnalysisAI FluencyStorytelling

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Measurement ScienceCausal CredibilityControlled ExperimentsMatched ControlsQuasi-Experimental MethodsPre-RegistrationData Leakage Risk ManagementBlinded ReviewBacktestingPower Calculations

Soft Skills

Systems ThinkingIndependenceExecutive Presence

Industry Keywords

Regulated-Industry MeasurementProvenanceAuditabilityResponsible-AI Evaluation Frameworks

About the role

Key responsibilities & impact

Own the portfolio-wide standard for what counts as proof—defined before work begins, not after results land.
Draft and socialize pre-registered success criteria for each initiative (primary KPIs, guardrails, minimum detectable effect, read windows).
Maintain a shared measurement playbook (matched controls, power calculations, blinded review, backtesting where appropriate).
Prioritize what the portfolio must prove next—and in what sequence evidence compounds.
Maintain a ranked learning agenda across initiatives and recommend the next proof point per initiative.
Review experiment designs: control arms, audience matching, read windows, confounders, data leakage risks.
Run power analysis and flag underpowered or ungradeable designs before launch.
Produce evidence read memos on a fixed cadence: what was tested, what was observed, what it means, what is recommended.
Synthesize across initiatives into one portfolio story—not a patchwork of disconnected scorecards.
Guard against failure modes that quietly invalidate results: moving goalposts; letting the people a system is meant to outperform grade it; acting before pre-registered proof exists.
Escalate when a read is not gradeable.
Run evidence read sessions at pre-registered dates; own continue / adjust / stop recommendations with rationale; track whether recommendations are acted on.

Requirements

What you’ll need

Bachelor's degree required with 8+ years of relevant experience spanning experimentation, measurement science, causal inference, or a quantitatively rigorous discipline — paired with strategy or product-facing work.
Systems thinking: sees across an entire portfolio, frames the questions that matter most, and designs how proof compounds over time rather than optimizing one test at a time.
Storytelling and executive presence: turns a rigorous, technical result into a clear, persuasive narrative that shapes senior decisions.
Experiment design and causal credibility: has designed and defended controlled experiments at enterprise scale: power analysis, matched controls, quasi-experimental methods, pre-registration.
Independence and spine: able to tell a senior stakeholder that a result does not clear the bar, and make it stick.
AI fluency: evaluates AI/ML and generative systems credibly: what a real performance signal looks like versus a plausible artifact.
Regulated-industry measurement experience (preferred).
Provenance, auditability, and responsible-AI evaluation frameworks (preferred).
Experience standing up an experimentation or evidence function from scratch (preferred).

Benefits

Comp & perks

401(k) plan with Pfizer Matching Contributions and additional Pfizer Retirement Savings Contribution
Paid vacation, holiday, and personal days
Paid caregiver/parental and medical leave
Health benefits including medical, prescription drug, dental, and vision coverage