Spotify6 days ago
Senior Product Quality Analyst - AI Voice
RUB 9,541–13,629per month · before tax
PermanentNew York
Responsibilities
- 01Own the quality bar for Spotify’s AI voice models, defining what “good enough” means for different products, audiences, languages and use cases.
- 02Independently design evaluation rubrics and build an evaluation approach that measures qualities such as naturalness, tone, pacing, pronunciation, accent, audio quality and overall experience.
- 03Plan and carry out evaluation campaigns, turning qualitative and quantitative findings into clear recommendations for Voice Production, Speak and product teams.
- 04Establish an ongoing evaluation rhythm, including when and how we reassess model quality as our technology and products evolve.
- 05Benchmark Spotify’s AI voices against third-party models and competitors to understand where our experiences lead, where they differ and where there are opportunities to improve.
- 06Help determine appropriate quality trade-offs as Spotify expands into additional languages and regions, considering the product use case and needs of each market.
- 07Partner closely with voice experts, machine learning teams, product partners and vendors to maintain a consistent quality bar across demanding and sometimes competing priorities.
- 08Surface the most meaningful gaps in model performance and use evaluation insights to help influence future research and technical priorities.
Requirements
- 01Experience in product quality, evaluation, data curation or a closely related field where human judgment is used to assess model or product performance.
- 02Know how to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics.
- 03Comfortable designing and running both qualitative and MOS-style evaluations and synthesizing the results into actionable insights.
- 04Understanding of voice or audio quality, including qualities such as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly.
- 05Strong judgment and can maintain a consistent quality bar while balancing different product requirements, markets and stakeholder perspectives.
- 06Comfortable working autonomously, identifying what needs to be evaluated and establishing the processes and cadence to make that evaluation useful over time.
- 07Communicate clearly across creative, technical and product teams and can act as a reliable point of contact for internal partners and external vendors.
- 08Curious about generative AI, speech technology and the evolving AI voice landscape and can translate changes in the industry into meaningful benchmarks for Spotify.
What we offer
- 01Based in New York or Los Angeles.
- 02Flexibility to work where you work best, with some in-person meetings and option to work from home.
- 03Health insurance.
- 04Six-month paid parental leave.
- 05401(k) retirement plan.
- 06Monthly meal allowance.
- 0723 paid days off.
- 08Paid flexible holidays.
- 09Paid sick leave.