Lovable17 days ago
Data Scientist, Agent
Salary not specified
MARKET
16,167 ₽median for this role
Data Scientist · 96 jobs with disclosed pay
5,800half of the offers: 13,208–20,77743,793
The employer didn't disclose pay — compare with the market yourself.
Полная занятостьОфис
Responsibilities
- 01Define and own the metrics for agent quality: success, completion, error rates, and the behaviors that drive them
- 02Build the eval systems and experiment framework that decide whether an agent change ships, like an A/B-tested rollout that catches a change increasing errors before it reaches everyone
- 03Turn agent traces and telemetry into concrete fixes, working directly with the agent engineering team
- 04Build the tooling and agents that produce these evaluations continuously as the agent evolves
- 05Set the bar for how we judge agent behavior where there is no answer key to check against
Requirements
- 01A data scientist who wants to make an AI agent measurably better, not just report on it
- 02Experience or strong interest in LLM evaluation and observability: building evals, scoring outputs, tracing agent behavior, and catching regressions
- 03Strong SQL and Python, applied statistics, and experimentation
- 04Comfortable designing A/B tests for agent changes where outcomes are noisy
- 05You build systems and agents that produce this insight continuously, rather than one-off analyses
- 06Instinct for what "good" looks like in agent behavior (success, error rates, task completion) and how to measure it when there is no clean answer key
- 07Entrepreneurial
- 08Thrives in ambiguity
- 09Works closely with the engineers building the agent