Pathway

Artificial Intelligence

• You will design and execute rigorous benchmarks and define dataset standards.
• Collaborating closely with our R&D team, you will build the evaluation infrastructure that guides the evolution of Pathway’s post-transformer models.
• You Will
• - Proactively identify, prioritize, and curate relevant public and client-driven benchmarks across our target use cases and markets.
• - Evaluate candidate benchmarks for clarity, data quality, evaluation methodology, and fit with our model roadmap.
• - Run benchmarks with baseline models to validate setup, uncover edge cases, and de-risk R&D runs.
• - Hand off “benchmark-ready” packages to R&D (specs, data, evaluation scripts, expected metrics, constraints)
• - Maintain a shared vocabulary and documentation around benchmarks, datasets, and evaluation formats that GTM and R&D can both use.
• - Track and organize benchmark results, model leaderboards, and “what good looks like” for different customers and scenarios.
• - Contribute to demos and public-facing proof points based on benchmark outcomes.
• You will play a key role in defining and driving the benchmarking process for AI model evaluation. Your work will directly influence what we build, how we talk about it, and how customers and the market experience BDH.

AI Benchmark & Datasets Engineer – Researcher

• Proactively identify, prioritize, and curate relevant public and client-driven benchmarks across our target use cases and markets.
• Evaluate candidate benchmarks for clarity, data quality, evaluation methodology, and fit with our model roadmap.
• Run benchmarks with baseline models to validate setup, uncover edge cases, and de-risk R&D runs.
• Hand off “benchmark-ready” packages to R&D (specs, data, evaluation scripts, expected metrics, constraints)
• Maintain a shared vocabulary and documentation around benchmarks, datasets, and evaluation formats that GTM and R&D can both use.
• Track and organize benchmark results, model leaderboards, and “what good looks like” for different customers and scenarios.
• Contribute to demos and public‑facing proof points based on benchmark outcomes.
• You will play a key role in defining and driving the benchmarking process for AI model evaluation. Your work will directly influence what we build, how we talk about it, and how customers and the market experience BDH.

AI Benchmark & Datasets Engineer – Researcher

AI Benchmark & Datasets Engineer – Internship