Fundamental

Engineer

• Design, build, and maintain production model serving infrastructure using Triton Inference Server as the primary framework
• Implement and optimize inference pipelines including custom backends, dynamic batching strategies, and model ensemble configurations in Triton
• Optimize Python inference code for performance, with a strong focus on GIL contention, multi-threading, and concurrency patterns
• Tune throughput and latency across the full serving stack, batching policies, thread pool sizing, model instance groups, and memory layout
• Work closely with the research team to understand new model architectures at a computational level, batching behavior, dynamic shapes, memory access patterns etc
• Own the full resource observability and control loop for production inference - instrument GPU memory, CPU, batch queue depth, and latency metrics, and actively tune model instance groups, concurrency limits, memory budgets, and batching configuration in response to observed behavior
• Evaluate and integrate alternative inference frameworks and runtimes as the model ecosystem evolves
• Contribute to GPU utilization improvements and resource efficiency across the serving fleet

Model Serving Engineer