Magic

Software Engineer

• Build and maintain the internal evals platform used across Magic
• Design, implement, and validate eval tasks for pre-training, post-training, reinforcement learning, inference, and product systems
• Develop infrastructure for running large-scale evaluations
• Build systems to measure dataset quality and identify opportunities to improve training data
• Improve evaluation correctness, reproducibility, and reliability
• Audit and improve upon public benchmarks, evaluation methodologies, and open-source implementations
• Partner with research, data, inference, and product teams to define metrics that accurately reflect model quality
• Build tooling and frameworks that enable teams across Magic to make decisions based on trustworthy measurements

Member of Technical Staff – Evals