Staff Software Engineer, Model Infrastructure

Навыки

Обязанности

• 01Lead the design and implementation of Harvey's Model Infrastructure platform
• 02Build systems to ensure high availability, low latency, and operational excellence for AI inference
• 03Design and improve Harvey's Unified Model Controller (UMC) and Model Selector platform to automatically detect model degradations and intelligently route traffic
• 04Develop systems for model provisioning, capacity management, failover, and traffic engineering across multiple AI providers
• 05Integrate new model providers and maintain provider APIs and SDKs
• 06Improve observability through health dashboards, alerting, token usage analytics, cost reporting, and end-to-end telemetry
• 07Partner with Product Engineering to support model launches, experimentation, and proactive monitoring of production AI workloads
• 08Drive infrastructure efficiency through capacity planning, utilization optimization, and cost visibility
• 09Collaborate with AI Research to build the infrastructure foundation for future model evaluation, training, and deployment
• 10Lead cross-functional technical initiatives and mentor engineers across the organization

Требования

• 017+ years of software engineering experience building large-scale distributed systems
• 02Experience designing and operating highly available production services
• 03Strong programming skills in Go, Java, Python, Rust, or C++
• 04Deep understanding of distributed systems, cloud infrastructure, networking, and observability
• 05Experience leading technical projects across multiple engineering teams
• 06Ability to balance long-term architecture with pragmatic execution
• 07Strong communication and collaboration skills
• 08Passion for building foundational platforms that enable other engineering teams

Условия

• 01$236,000 - $290,000 USD