OpenAI

Perception Engineer

OpenAI

full-time

Posted on:

Origin:  • 🇺🇸 United States • California

Visit company website
✨ AI Apply
Apply

Salary

💰 $325,000 - $405,000 per year

Job Level

Mid-LevelSenior

Tech Stack

Python

About the role

  • Design and implement perception pipelines for detection, tracking, segmentation, depth/optical flow, and scene understanding
  • Build robust sensor ingest systems including camera capture, IMU/time sync, buffering, and zero-copy data paths
  • Calibrate and fuse multimodal sensors (intrinsics, extrinsics, temporal sync) and implement state estimation (EKF/UKF, VIO/SLAM)
  • Optimize on-device inference and vision kernels across CPU/GPU/NPU/ISP, focusing on quantization, compilation, and scheduling under power/thermal limits
  • Define clean interfaces to planning/control teams, specifying contracts, latency/throughput budgets, and fault-tolerant fallbacks
  • Instrument performance and reliability through profiling, telemetry, replayable logs, dataset curation, and automated regression testing (SIL/HIL)
  • Collaborate cross-functionally with camera, systems, and controls engineers to ship features and triage field issues with root-cause fixes
  • Own the full stack from sensor ingest and calibration to on-device optimization and clean system interfaces

Requirements

  • Deep experience building and shipping real-time perception for robots, mobile platforms, or edge devices
  • Fluency in C++ and Python
  • Strong foundations in 3D geometry, camera models, probabilistic estimation, and computer vision/ML fundamentals
  • Hands-on expertise with camera pipelines, sensor fusion, and calibration/time synchronization
  • Experience implementing state estimation (EKF/UKF, VIO/SLAM)
  • Skilled in deploying and optimizing ML models on resource-constrained devices (CPU/GPU/NPU/ISP)
  • Experience with quantization, compilation, and scheduling under power/thermal limits
  • Ability to profile and tune systems for latency, memory, and power efficiency
  • Experience building robust sensor ingest systems including camera capture, IMU/time sync, buffering, and zero-copy data paths
  • Ability to define interfaces to planning/control teams, specifying contracts, latency/throughput budgets, and fault-tolerant fallbacks
  • Experience instrumenting performance and reliability through profiling, telemetry, replayable logs, dataset curation, and automated regression testing (SIL/HIL)
  • Strong cross-functional collaboration with camera, systems, and controls engineers to ship features and triage field issues
  • Bonus: Expertise in VIO/SLAM, mapping, and relocalization
  • Bonus: Experience with depth/ranging modalities (stereo, ToF, LiDAR, structured light) and uncertainty-aware fusion
  • Bonus: Familiarity with real-time media/compute pipelines (e.g., GStreamer, zero-copy)
  • Bonus: Experience designing for robustness in challenging conditions (motion blur, HDR/low light, occlusion, sensor faults)
  • Bonus: Knowledge of ROS 2, simulation frameworks for closed-loop evaluation, or HIL lab design
  • Bonus: Contributions to open-source vision/perception stacks