Applied Data Scientist, Finance AI Evaluation, Datasets

InnoData

Applied Data Scientist at Innodata designing datasets for evaluating financial AI systems in high-stakes environments. Collaborating with cross-functional teams to ensure data quality and compliance.

Posted 7/27/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSenior💰 $150,000 - $175,000 per yearWebsite

Tech Stack

Tools & technologies

PandasPythonPyTorchScikit-LearnSQL

About the role

Key responsibilities & impact

Translate customer goals — such as improving financial reasoning, building an eval suite for earnings-call summarization, or evaluating an AML/fraud copilot — into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria.
Design training and evaluation datasets across the financial AI surface: financial QA, filings and earnings analysis, credit and underwriting, fraud/AML investigation, and compliance, among other financial workflows.
Foreground unstructured and multimodal financial data in dataset design — PDFs, scanned statements, tables, charts, and call transcripts — used by analysts, advisors, compliance reviewers, and operations teams.
Design datasets and evaluations for retrieval-augmented and source-grounded systems: evidence citation and faithfulness to source documents, data freshness, conflict resolution across sources, and failure modes caused by incomplete or incorrectly parsed context.
Evaluate agentic and workflow-integrated financial AI systems: tool use, retrieval, transaction boundaries, escalation behavior, and controls that prevent unsafe or unauthorized actions.
Develop evaluation methodology that goes beyond surface accuracy — numerical consistency, hallucination rates on high-risk claims, refusal and escalation appropriateness, robustness under ambiguity, and fairness across protected or sensitive customer segments.
Define sampling strategies, label schemas, and adjudication workflows with Language Data Scientists and finance SMEs; write annotation guidelines that make subjective finance-domain judgments explicit, calibratable, and auditable.
Build the statistical and ML tooling that makes large financial datasets trustworthy: stratified sampling across products, markets, and modalities; bias analysis; leakage detection; and distribution shift checks, among other reliability checks.
Build evaluation and dataset-quality evidence to support financial-services model risk management: assumptions, limitations, validation results, and residual risks, packaged as reproducible evidence.
Partner with the AI/ML Research Engineer to instrument datasets into training, evaluation, and monitoring pipelines — rubric-grounded LLM-as-judge prompts, regression suites, and continuous monitoring.
Own data quality end-to-end, from intake through delivery: PII handling, provenance tracking, versioning, and modality-specific QA checks.
Reason about financial workflow context: where AI outputs enter analyst, advisor, compliance, risk, or customer-facing workflows; what evidence a reviewer needs to trust them; and when uncertainty must be surfaced.
Support the Technical Solutions Architect during customer discovery and proposals: scoping dataset programs, sizing annotation effort, and explaining methodology to client stakeholders.
Stay current on the financial AI landscape: regulatory developments, benchmark releases, and emerging evaluation methodology for finance-domain models.
Contribute to Innodata internal IP: reusable taxonomies, evaluation rubrics, golden datasets, and methodology templates.

Requirements

What you’ll need

5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
Real working knowledge of financial data and workflows: financial statements, SEC filings, transaction data, and other common financial-services document types.
Hands-on experience with unstructured and multimodal financial data — some combination of PDFs, scanned documents, spreadsheets, charts, or call transcripts.
Familiarity with financial standards or protocols such as XBRL, ISO 20022, or GAAP/IFRS reporting concepts, etc. is strongly preferred.
Hands-on experience designing datasets for ML — not just consuming them. You have written annotation guidelines, sized cohorts, set quality thresholds, and shipped data that downstream teams could actually train, evaluate, or monitor on.
Familiarity with LLM-based and multimodal financial AI workflows: prompt design, rubric-based evaluation, RAG, LLM-as-judge methods, and the limitations of automated evaluation in high-stakes contexts.
Strong Python and SQL; comfort with pandas, scikit-learn, or equivalent; working familiarity with Hugging Face, PyTorch, or model APIs.
Statistical literacy: sampling design, inter-annotator agreement metrics (e.g., Cohen's kappa), confidence intervals, and the ability to push back when a number is being over-interpreted.
Solid grasp of financial services privacy, compliance, and governance: PII handling, GLBA or equivalent privacy regimes, MNPI sensitivity, and documentation fit for regulated AI programs.
Excellent collaboration skills — upstream with a Technical Solutions Architect, sideways with research scientists and engineers, and downstream with SME annotators and quality teams.
A bias toward financial workflow realism. You would rather build a smaller dataset that reflects what analysts, advisors, or customers actually see than a larger one that looks impressive on paper but fails in practice.
Degree in a relevant field — statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience. Formal finance credentials aren't required, but CFA, FRM, or MBA backgrounds, etc. are especially encouraged.
Experience designing evaluations for LLMs, VLMs, or multimodal models in financial reasoning, filings analysis, or fraud/AML contexts.
Experience with document AI, OCR/post-OCR quality, or table and chart extraction for complex financial documents.
Familiarity with agentic evaluation, AI observability, experiment tracking, or tools such as Weights & Biases or LangFuse.
Familiarity with model risk management frameworks, validation documentation, fairness/bias auditing, or consumer protection analysis.
Experience with multilingual or cross-border financial data, or published/open-source work in financial AI or model governance.

Benefits

Comp & perks

🌐 Worldwide ❌ Jobs You've Hidden ⭐️ Saved Jobs ✅ Applied Jobs ✉️ Email Alerts 👤 Account InnoData Website LinkedIn All Job Openings 2 - 10 employees Founded 2019 🤝 B2B 💼 Consulting 🌍 Social Impact B2B
Consulting
Social Impact InnoData is INNOvation DATA SCS, an Italian social cooperative based in Foggia that identifies itself as a provider of technological solutions. The company website (currently under maintenance) highlights "Soluzioni tecnologiche" (technological solutions) and emphasizes social impact ("Impatto sociale"). Contact details listed include Via Francesco Crispi 65, 71121 Foggia, Italy. Based on the available information, InnoData appears to operate at the intersection of technology and social impact, likely offering tech-focused services to other organizations. Applied Data Scientist, Finance AI Evaluation, Datasets Job not on LinkedIn 🔥 25 minutes ago 🇺🇸 United States – Remote 💵 $150k - $175k / year ⏰ Full Time 🟡 Mid-level 🟠 Senior 📊 Data Scientist Apply Now Find Hiring Managers Customize resume + cover letter Report problem ☆ Save ☑️ Mark as applied ❌ Hide 📋 Description
Translate customer goals — such as improving financial reasoning, building an eval suite for earnings-call summarization, or evaluating an AML/fraud copilot — into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria.
Design training and evaluation datasets across the financial AI surface: financial QA, filings and earnings analysis, credit and underwriting, fraud/AML investigation, and compliance, among other financial workflows.
Foreground unstructured and multimodal financial data in dataset design — PDFs, scanned statements, tables, charts, and call transcripts — used by analysts, advisors, compliance reviewers, and operations teams.
Design datasets and evaluations for retrieval-augmented and source-grounded systems: evidence citation and faithfulness to source documents, data freshness, conflict resolution across sources, and failure modes caused by incomplete or incorrectly parsed context.
Evaluate agentic and workflow-integrated financial AI systems: tool use, retrieval, transaction boundaries, escalation behavior, and controls that prevent unsafe or unauthorized actions.
Develop evaluation methodology that goes beyond surface accuracy — numerical consistency, hallucination rates on high-risk claims, refusal and escalation appropriateness, robustness under ambiguity, and fairness across protected or sensitive customer segments.
Define sampling strategies, label schemas, and adjudication workflows with Language Data Scientists and finance SMEs; write annotation guidelines that make subjective finance-domain judgments explicit, calibratable, and auditable.
Build the statistical and ML tooling that makes large financial datasets trustworthy: stratified sampling across products, markets, and modalities; bias analysis; leakage detection; and distribution shift checks, among other reliability checks.
Build evaluation and dataset-quality evidence to support financial-services model risk management: assumptions, limitations, validation results, and residual risks, packaged as reproducible evidence.
Partner with the AI/ML Research Engineer to instrument datasets into training, evaluation, and monitoring pipelines — rubric-grounded LLM-as-judge prompts, regression suites, and continuous monitoring.
Own data quality end-to-end, from intake through delivery: PII handling, provenance tracking, versioning, and modality-specific QA checks.
Reason about financial workflow context: where AI outputs enter analyst, advisor, compliance, risk, or customer-facing workflows; what evidence a reviewer needs to trust them; and when uncertainty must be surfaced.
Support the Technical Solutions Architect during customer discovery and proposals: scoping dataset programs, sizing annotation effort, and explaining methodology to client stakeholders.
Stay current on the financial AI landscape: regulatory developments, benchmark releases, and emerging evaluation methodology for finance-domain models.
Contribute to Innodata internal IP: reusable taxonomies, evaluation rubrics, golden datasets, and methodology templates. 🎯 Requirements
5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
Real working knowledge of financial data and workflows: financial statements, SEC filings, transaction data, and other common financial-services document types.
Hands-on experience with unstructured and multimodal financial data — some combination of PDFs, scanned documents, spreadsheets, charts, or call transcripts.
Familiarity with financial standards or protocols such as XBRL, ISO 20022, or GAAP/IFRS reporting concepts, etc. is strongly preferred.
Hands-on experience designing datasets for ML — not just consuming them. You have written annotation guidelines, sized cohorts, set quality thresholds, and shipped data that downstream teams could actually train, evaluate, or monitor on.
Familiarity with LLM-based and multimodal financial AI workflows: prompt design, rubric-based evaluation, RAG, LLM-as-judge methods, and the limitations of automated evaluation in high-stakes contexts.
Strong Python and SQL; comfort with pandas, scikit-learn, or equivalent; working familiarity with Hugging Face, PyTorch, or model APIs.
Statistical literacy: sampling design, inter-annotator agreement metrics (e.g., Cohen's kappa), confidence intervals, and the ability to push back when a number is being over-interpreted.
Solid grasp of financial services privacy, compliance, and governance: PII handling, GLBA or equivalent privacy regimes, MNPI sensitivity, and documentation fit for regulated AI programs.
Excellent collaboration skills — upstream with a Technical Solutions Architect, sideways with research scientists and engineers, and downstream with SME annotators and quality teams.
A bias toward financial workflow realism. You would rather build a smaller dataset that reflects what analysts, advisors, or customers actually see than a larger one that looks impressive on paper but fails in practice.
Degree in a relevant field — statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience. Formal finance credentials aren't required, but CFA, FRM, or MBA backgrounds, etc. are especially encouraged.
Experience designing evaluations for LLMs, VLMs, or multimodal models in financial reasoning, filings analysis, or fraud/AML contexts.
Experience with document AI, OCR/post-OCR quality, or table and chart extraction for complex financial documents.
Familiarity with agentic evaluation, AI observability, experiment tracking, or tools such as Weights & Biases or LangFuse.
Familiarity with model risk management frameworks, validation documentation, fairness/bias auditing, or consumer protection analysis.
Experience with multilingual or cross-border financial data, or published/open-source work in financial AI or model governance. Apply Now 📊 Check your resume score for this job Improve your chances of getting an interview by checking your resume score before you apply. Check Resume Score Similar Jobs Data Scientist – Materials R&D 🔥 50 minutes ago IPG 1001 - 5000 🏭 Manufacturing 🤝 B2B 🛍️ eCommerce Website LinkedIn All Job Openings Senior Data Scientist supporting R&D efforts in bio-polymers and sustainable materials. Collaborating with scientists and engineers, applying data science and machine learning techniques. 🇺🇸 United States – Remote ⏰ Full Time 🟠 Senior 🔴 Lead 📊 Data Scientist Product Manager/Product Owner, Data Science 🔥 1 hour ago E Source 201 - 500 ⚡ Energy 💼 Consulting 🤝 B2B Website LinkedIn All Job Openings Product Manager/Product Owner for Data Science Storm Insights at E Source. Leading strategy, roadmap, and delivery of critical analytics products for the Utility industry. 🇺🇸 United States – Remote 💵 $140k - $185k / year ⏰ Full Time 🟡 Mid-level 🟠 Senior 📊 Data Scientist People Analytics Lead 🔥 2 hours ago Ascend 1001 - 5000 🤝 B2B 💼 Consulting Website LinkedIn All Job Openings People Analytics Lead dedicated to building the infrastructure for people strategy at Ascend Partner Services. Managing AI-driven people analytics and influencing stakeholders across 20+ firms. 🇺🇸 United States – Remote 💰 $2M Seed Round - Ascend on 2025-02 ⏰ Full Time 🟠 Senior 📊 Data Scientist News Data Automation Lead, Content AI 🔥 2 hours ago Dayforce 5001 - 10000 👥 HR Tech ☁️ SaaS 🏢 Enterprise Website LinkedIn All Job Openings Lead the design and implementation of AI-assisted journalism workflows at USA TODAY NETWORK. Focus on data automation and innovative newsroom collaboration. 🇺🇸 United States – Remote 💵 $24 - $50 / hour 💰 $1G Post-IPO Debt - Dayforce on 2024-03 ⏰ Full Time 🟠 Senior 📊 Data Scientist Data Scientist 🔥 2 hours ago PLACE 201 - 500 💼 Consulting 🏥 Healthcare 🛡️ Insurance Website LinkedIn All Job Openings Data Scientist at PLACE, focusing on AI and machine learning solutions in a rapidly growing startup. Collaborating within a small team to enhance data science applications and production deployment. 🇺🇸 United States – Remote 💵 $135k - $170k / year ⏰ Full Time 🟡 Mid-level 🟠 Senior 📊 Data Scientist 🦅 H1B Visa Sponsor View More Data Science Jobs 🌐 Worldwide Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com Search Search Jobs by country Search jobs by city Search jobs by job title Search entry-level jobs Search junior-level jobs Search senior-level jobs Search jobs by tech stack Search jobs by contract type Search remote internships Search remote part-time jobs Remote jobs Anywhere in the World Companies Hiring Anywhere in the World Companies Hiring Sales People Anywhere in the World Companies Hiring Software Engineers Anywhere in the World Resources Advice Tips for finding remote jobs Interview questions and answers Resume examples Cover letter examples Post a job Affiliates Is Remote Rocketship legit? Privacy policy Terms of service Job board SEO course OpenClaw job finder Find jobs using your resume Jobs by Country Remote jobs anywhere in the world (Worldwide remote jobs) Remote jobs United States Remote jobs Australia Remote jobs Brazil Remote jobs Canada Remote jobs France Remote jobs Ireland Remote jobs Germany Remote jobs Netherlands Remote jobs Spain Remote jobs UK Popular Jobs Remote data analyst jobs Remote customer support jobs Remote executive assistant jobs Remote marketing jobs Remote product designer jobs Remote product manager jobs Remote project manager jobs Remote recruiter jobs Remote sales jobs Remote software engineer jobs Jobs by Type Remote full-time jobs Remote part-time jobs Remote contract jobs Remote internship jobs Remote entry-level jobs Remote jobs with no experience required Remote junior jobs (1-3 years of experience) Digital nomad jobs Remote jobs with no degree required Freelance remote jobs Temporary remote jobs Remote jobs hiring now Stay at home mom jobs