Machine Learning Systems Engineer
Substack
AirflowAWSCloudDistributed SystemsDockerJavaScriptKubernetesPythonPyTorchRayTensorflowTypeScript
Job Level
Senior
Tech Stack
AirflowDistributed SystemsDockerGoGRPCJavaKubernetesPython
About the role
- Design, build, and scale the Notes Execution Platform — enable seamless execution of internal and external code in a secure, isolated, and observable way.
- Architect sandboxed environments to run untrusted or user-supplied code with appropriate resource quotas and security controls.
- Develop backend services and APIs to manage job lifecycles: scheduling, monitoring, cancellation, logging, and reporting.
- Ensure the system is resilient and scales horizontally to meet growing demand.
- Integrate the Notes platform with Lyric’s data platform and ML workflows.
- Collaborate with product managers, data scientists, and engineers to understand use cases and deliver intuitive developer experiences.
- Drive technical excellence: establish best practices in code execution safety, performance tuning, and observability.
Requirements
- 5+ years of backend engineering experience, ideally building distributed systems or platform infrastructure.
- Deep expertise in at least one backend language (e.g., Python, Go, Java, or similar) and experience designing RESTful and/or gRPC APIs.
- Experience with containerization and orchestration technologies (e.g., Docker, Kubernetes) and building isolated execution environments.
- Strong understanding of secure code execution, sandboxing, and resource quota enforcement.
- Familiarity with distributed job queues and task orchestration systems (e.g., Celery, Airflow, Kubernetes Jobs, or similar).
- Proven ability to build systems that are performant, resilient, and observable at scale.
- Excellent communication and collaboration skills, with a track record of working across teams.
- Nice to have: Experience building notebook-like platforms (e.g., JupyterHub, Databricks, or similar).
- Nice to have: Exposure to data and ML platforms, and an understanding of their unique execution needs.
- Nice to have: Experience with multi-tenant systems and securing shared infrastructure.
Similar jobs on JobTailor
Senior Engineer, Machine Learning Platform
CrowdStrike
AirflowApacheCloudCyber SecurityDistributed SystemsGoGRPCJavaJenkinsKubernetesPythonRay+3 more
Platform/Backend Engineer, GenAI Infrastructure
Lyric
CloudDistributed SystemsDockerGoGraphQLGRPCJavaKubernetesPythonScala
Senior Machine Learning Engineer, Infrastructure
Rad AI
AirflowAnsibleAWSAzureCloudDistributed SystemsDockerGoogle Cloud PlatformGrafanaJavaScriptKubernetesPostgres+5 more
Platform Engineer – Senior/Staff/Principal
Quilter
AirflowAWSCloudDockerGoGrafanaKubernetesPrometheusPythonTerraformTypeScript