Scale AI11 дней назад
Machine Learning Research Scientist, Evaluations
15 050–18 813 ₽в месяц · до вычета
РЫНОК
16 250 ₽медиана по профессии
Data Scientist · 212 вакансий с указанной зарплатой
5 000половина предложений: 13 084–21 768120 000
В рынке · выше 41% предложений
San Francisco
Обязанности
- 01Analyze model behavior to identify, characterize, and diagnose failure modes in frontier LLMs and Agents
- 02Identify capability gaps, reasoning errors, robustness and alignment issues, focusing on root cause analysis
- 03Design and build benchmarks and evaluation methods that measure LLM capabilities in both text and multimodal modalities
- 04Apply post-training expertise (SFT, RLHF, reward modeling) to connect observed failures to data and training interventions
- 05Publish research findings in top-tier AI conferences
- 06Collaborate with researchers and engineers to define best practices in evaluation-driven AI development
- 07Partner with top foundation model labs to translate failure analysis into technical and strategic input on next-generation generative AI models
Требования
- 01Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field
- 02Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning
- 03Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning, and with LLM evaluation or benchmark development
- 04Excellent written and verbal communication skills
- 05Published research in machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals
- 06Previous experience in a customer-facing role
Условия
- 01Base salary range: $180,600 – $225,750 USD per year
- 02Equity compensation
- 03Comprehensive health, dental, and vision coverage
- 04Retirement benefits
- 05Learning and development stipend
- 06Generous paid time off (PTO)
- 07Commuter stipend (where applicable)
- 08Full-time position
- 09Locations: San Francisco, New York, Seattle