Senior Manager, Software Engineering
ATPCO
Senior Engineering Manager – Accelerated Compute Memory Systems
Pryon
full-time
Posted on:
Location Type: Remote
Location: Remote • New York • 🇺🇸 United States
Visit company websiteJob Level
Senior
Tech Stack
AWSAzureCloudDistributed SystemsGoGoogle Cloud PlatformKafkaKubernetesPulsarPythonRabbitMQRay
About the role
- Build and lead a team delivering the ingestion, retrieval, and inference layers that will power mission-critical deployments for commercial and federal entities with millions of public users.
- Architect and deliver horizontally scalable, fault-tolerant systems capable of handling billions of documents and burst loads of 30K+ concurrent users.
- Guide implementation of multimodal ingestion pipelines (eg PDF, HTML, DOCX, JSON, XML, PPTX, TIFF).
- Oversee design and optimization of LLM-driven data ingestion and retrieval workflows.
- Own optimization and tuning of high-throughput, low-latency production environments via async orchestration frameworks.
- Establish performance benchmarking, compliance frameworks, and automated testing for scale.
- You will balance technical leadership with people leadership, guiding architecture decisions, while also scaling and mentoring a high-performing team.
- Collaborate cross-functionally with Product, Executive Leadership, and Customer Success.
Requirements
- 10+ years in software engineering, 5+ years in management roles with large-scale AI/ML systems and infrastructure.
- Expert-level proficiency in Python and Golang, with 5+ years building production distributed systems.
- Experience with orchestration frameworks (Kubernetes, Ray, Dask).
- Proficiency with vector databases (Pinecone, Weaviate, Qdrant, or similar).
- Experience with message queuing systems (Kafka, Pulsar, RabbitMQ).
- In-depth knowledge and hands on experience building scalable distributed architectures and high-performance compute systems.
- Proven experience in multimodal ingestion pipelines within RAG platforms.
- Direct experience in designing, fine-tuning, and optimizing LLMs for ingestion and retrieval workloads.
- Previous success managing engineering teams delivering production-grade, HPC-scale RAG systems.
- Deep understanding of infra domains: compute, storage, networking, observability, security, disaster recovery, and cost management.
- Familiarity with HPC cluster management softwares such as Slurm.
- Familiarity with cloud platforms (AWS, Azure, GCP) and/or on-prem datacenter operations.
Benefits
- Remote first organization
- 100% Company paid Health/Dental/Vision benefits for you and your dependents
- Life Insurance, Short-term and Long-term Disability
- 401k
- Unlimited PTO
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
PythonGolangKubernetesRayDaskPineconeWeaviateQdrantKafkaPulsar
Soft skills
technical leadershippeople leadershipmentoringcollaboration
