Tech Stack
AirflowApacheCloudCyber SecurityDistributed SystemsGoGoogle Cloud PlatformGRPCJavaJenkinsKubernetesPythonRayScalaSparkTerraform
About the role
- Charter: ML Platform team builds tools for exploration, model development, ML engineering and insights activation on the Data LakeHouse
- Work with hyper-scale Data Lakehouse (>200 PB), batch and near real-time ingestion used for threat analytics
- Build ML Experimentation Platform from ground up and enable self-service for thousands of users
- Design, implement, and maintain scalable ML pipelines for data preparation, cataloging, feature engineering, training and model serving
- Collaborate with Data Platform Software Engineers, Data Scientists, and Threat Analysts
- Modularize ML code into standard components and establish repeatable patterns for development, deployment, and monitoring
- Leverage workflow orchestration, Kubernetes, blob storage, and queues in a cloud-first environment
- Review data scientist code and champion software development best practices
- Contribute to production-focused culture bridging model development and operational success
- Future plans include generative AI investments for modeling attack paths
- Candidates must be comfortable visiting the office once a week
Requirements
- B.S. in Computer Science, Data Science, Statistics, Applied Mathematics, or related field with 10+ years experience; or M.S. with 8+ years; or Ph.D. with 6+ years
- 3+ years experience developing and deploying machine learning solutions to production
- Familiarity with typical machine learning algorithms and supervised/unsupervised approaches
- 3+ years experience with ML platform tools (Jupyter, NVidia Workbench, MLFlow, Ray, Vertex AI, etc.)
- Experience building data platform products using Apache Spark, Flink or comparable tools in GCP
- Experience with Iceberg (highly desirable)
- Proficiency in distributed computing and orchestration technologies (Kubernetes, Airflow)
- Production experience with infrastructure-as-code tools such as Terraform, FluxCD
- Expert-level experience with Python; Java/Scala exposure recommended
- Ability to write Python interfaces for data scientists to use internal tools
- Expert-level experience with CI/CD frameworks such as GitHub Actions
- Expert-level experience with containerization frameworks
- Strong analytical and problem solving skills
- Exceptional interpersonal and communication skills
- Distributed systems knowledge
- Data platform experience
- Machine learning concepts
- Desirable: Go, Iceberg, Pinot or other time-series/OLAP database, Jenkins, Parquet, Protocol Buffers/gRPC