Process Automation Engineer
Truckstop
AirflowAWSAzureBigQueryCloudGoogle Cloud PlatformPythonSQL
Lead AI Platform Engineer
AssemblyAI
full-time
Posted on:
Location Type: Remote
Location: Remote • New York • 🇺🇸 United States
Visit company websiteSalary
💰 $230,000 - $275,000 per year
Job Level
Senior
Tech Stack
AirflowApacheBigQueryGoogle Cloud PlatformPySparkPythonSQL
About the role
- Drive technical direction of data and AI training infrastructure team and shape vision and architecture
- Enhance and scale infrastructure to improve training velocity and research outcomes
- Design scalable, future-proof data platforms optimized for AI research workloads
- Build efficient data pipelines leveraging GCP services
- Implement cost-effective storage and monitoring solutions for ML at scale
- Create flexible training resource management with intelligent queuing and optimize resource allocation
- Participate in on-call rotation to ensure system reliability
- Lead adoption of cutting-edge ML tools and frameworks and streamline workflows
- Enhance tooling and documentation, implement guardrails for cost, quality, and performance
- Collaborate closely with Research team and senior leadership and provide technical guidance across the organization
Requirements
- 8+ years of experience in AI/ML Infrastructure, Research Platform Engineering, or related software engineering roles
- 3+ years of professional experience working as an AI data and infrastructure setting or similar position most recently at a startup
- Strong proficiency in Python and SQL
- Deep expertise with GCP services like BigTable, BigQuery, Dataproc, Dataflow
- Experience with distributed processing frameworks (Apache Beam, PySpark, etc.)
- Familiarity with workflow orchestration tools (Airflow, Composer, Astronomer)
- Understanding of distributed training systems and data loading optimization
- Experience with experiment tracking and training tooling
- Ability to thrive in a startup environment with aggressive prioritization and rapidly changing business requirements
Benefits
- Fully remote team
- Competitive base salary range: $230,000 - $275,000
- Pay transparency policy
- Inclusive equal opportunity workplace
- Opportunity to work with experienced AI researchers and cutting-edge Speech AI
ATS Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
PythonSQLGCPBigTableBigQueryDataprocDataflowApache BeamPySparkAirflow
Soft skills
leadershipcollaborationtechnical guidanceadaptabilityprioritization
Similar jobs on JobTailor
Analytics Engineer
RADAR
AirflowApacheBigQueryPythonSparkSQLTableau
Senior Data Engineer
Velir
AirflowApacheAWSAzureBigQueryCloudETLKafkaPythonRustScalaSpark+1 more
Data Engineer I
Bombora
AirflowBigQueryCloudETLGoogle Cloud PlatformKubernetesPythonSQL
Head of Data and Analytics
Sanity
AirflowBigQueryHerokuPythonSQL