AI Workflow Analyst

Sophie Society Co.

AI Workflow Analyst at Sophie Society creating test cases and ensuring AI outputs are accurate for Amazon tasks. Responsible for testing and approving AI outputs before they go live.

Posted 4/21/2026full-timeRemote • 🏈 Anywhere in North AmericaMid-LevelSenior💰 $35,000 - $60,000 per yearWebsite

Tech Stack

Tools & technologies

PythonTypeScript

About the role

Key responsibilities & impact

Create test cases from real Amazon data (search term reports, campaigns, listings)
Run AI tools and review the outputs
Check for: - Wrong keyword classification - Bad campaign structure decisions - Missing or incorrect insights - Hallucinated data
Write simple scoring criteria to evaluate outputs
Identify what’s causing issues (prompt, input data, or model)
Work with the builder to fix and improve results
Approve or block tools before they go live

Requirements

What you’ll need

Strong hands-on use of ChatGPT and Claude
Ability to write clear prompts and improve them through testing
Strong attention to detail (you catch errors others miss)
Clear, structured writing
Comfortable with: - Git, Markdown, YAML - Reading basic Python or TypeScript

Benefits

Comp & perks

Fully remote, comfortable work environment — work from anywhere

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonTypeScriptprompt engineeringtest case creationdata analysisscoring criteria developmenterror identificationoutput evaluation

Soft Skills

attention to detailclear writingstructured writingproblem-solvingcollaboration