Operations

• Design and implement the observability architecture for AI and Agentic systems, enabling end-to-end visibility across models, agents, and data pipelines.
• Develop instrumentation frameworks to collect and analyze technical, behavioral, and cognitive metrics for deployed AI systems.
• Integrate and configure monitoring, tracing, and logging tools (Prometheus, Grafana, OpenTelemetry, ELK Stack, Datadog, etc.) to ensure full operational insight.
• Build dashboards and alerting mechanisms to detect data drift, performance issues, hallucinations, or reasoning inconsistencies in LLMs and agents.
• Collaborate with MLOps, Data, and Architecture teams to establish model lineage, drift detection, and governance pipelines.
• Design and maintain custom metrics for model and agent reliability — precision, latency, cost, reasoning depth, autonomy, and consistency.
• Contribute to the Responsible AI framework, ensuring transparency, fairness, and auditability in AI decision-making.
• Continuously research and experiment with new observability tools and practices (AgentOps, LLMOps, RAG Observability).

AgentOps Engineer – Observability

Banking Strategy & Ops Associate

Trade Operations Specialist

Operations & Strategy

Operations Intern, French Speaking