Full-stack Engineer

• Define quality metrics: Translate SOC use cases into measurable KPI's (e.g., precision/recall, MTTR, false-positive rate, step success, latency/cost budgets).
• Build continuous evaluations: Develop offline/online evaluation pipelines, regression suites, and A/B or canary test; integrate them into CI/CD for release gating.
• Curate and manage datasets: Maintain gold-standard datasets and red-team scenarios; establish data governance and drift monitoring practices.
• Ensure safety, reliability, and explainability: Partner with Platform and Security Research to encode guardrails, policy enforcement, and runtime safety checks.
• Expand adversarial test coverage (prompt injection, data exfiltration, abuse scenarios).
• Ensure explainability and auditability of agent decisions, maintaining traceability and compliance of AI-driven workflows.
• Production reliability & observability: Monitor and maintain reliability of agentic AI features post-release -- define and uphold SLIs/SLOs, establish alerting and rollback strategies, and conduct incident post-mortems.
• Design and implement infrastructure to scale evaluation and production pipelines for real-time SOC workflows across cloud environments.
• Drive agentic system engineering: Experiment with multi-agent systems, tool-using language models, retrieval-augmented workflows, and prompt orchestration.
• Manage model and prompt lifecycle -- track version, rollout strategies, and fallbacks; measure impact through statistically sound experiments.
• Collaborate cross-functionally: Work with Product, UX and Engineering to prioritize high-leverage improvements, resolve regressions quickly, and advance overall system reliability.

Senior Engineer, AI Evaluation & Reliability

Product Engineer

Forward Deployed Software Engineer

Lead Engineer, Digital Optimization and Analytics

Distinguished Software Engineer

Senior Software Engineer – Full Stack