Cohere4 days ago
Member of Technical Staff, North Modelling (Evals)
Salary not specified
MARKET
15,900 ₽median for this role
Data Scientist · 115 jobs with disclosed pay
5,000half of the offers: 12,796–20,50043,793
The employer didn't disclose pay — compare with the market yourself.
Полная занятостьУдалёнка
Responsibilities
- 01Own the eval strategy for North: define what we need to measure across agent workflows, tool use, enterprise knowledge work such as deep research, document creation or editing, and other human-AI interactions
- 02Build high-quality evals from the realities of the product: user feedback, production failures, privacy-preserving usage logs, internal dogfooding, customer needs, and forward-looking product goals
- 03Create systems that continuously turn what North is learning from users, customers, and feature teams into evals, so measurement keeps pace with the product rather than becoming a static benchmark
- 04Be the voice of North inside modelling: extract clear insight from eval results, customer-derived data and product context, then turn model failures, capability gaps, and North-specific needs into actionable recommendations for the central modelling teams
Requirements
- 01You have improved LLM-powered, agent-powered, or AI-product systems through evals, feedback loops, data curation, prompting, model adaptation, or model selection
- 02You care deeply about evaluation as a craft: representative tasks, precise rubrics, clean data, failure analysis, regression tracking, and knowing when a metric is giving false confidence
- 03You have strong applied MLE judgment and can reason clearly about model behavior, eval validity, product outcomes, and production tradeoffs
- 04You are comfortable working close to users and product teams, while translating messy qualitative signals into measurement that other modelling teams can act on
- 05You are self-directed, practical, and motivated by open-ended problems where the right answer requires both technical depth and product understanding
- 06You care about making AI systems genuinely useful in real enterprise settings, not just better on abstract benchmarks
What we offer
- 01Remote-friendly with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul
- 02Weekly lunch stipend of $75/£75 or equivalent in local currency
- 03Full health and dental benefits, including mental health budget
- 04RRSP matching, 401K, Pension Scheme
- 05100% Parental Leave top-up for up to 6 months
- 06Annual enrichment benefits for arts & culture, fitness/wellness, quality time, and workspace improvement
- 07Education & learning stipend for conferences, courses, and coaching
- 086 weeks of paid vacation (30 working days)
- 09Budget for traveling to other offices if remote
- 10Annual company offsite
- 11$500 home office stipend
- 12For office workers: daily lunch program, snacks, community and social events
- 13For remote workers: co-working benefit