Spotify6 days ago
Senior Product Quality Analyst - AI Voice
Salary not specified
PermanentLondon
Responsibilities
- 01Own the quality bar for Spotify’s AI voice models, defining what “good enough” means for different products, audiences, languages and use cases
- 02Independently design evaluation rubrics and build an evaluation approach that measures qualities such as naturalness, tone, pacing, pronunciation, accent, audio quality and overall experience
- 03Plan and carry out evaluation campaigns, turning qualitative and quantitative findings into clear recommendations for Voice Production, Speak and product teams
- 04Establish an ongoing evaluation rhythm, including when and how we reassess model quality as our technology and products evolve
- 05Benchmark Spotify’s AI voices against third-party models and competitors to understand where our experiences lead, where they differ and where there are opportunities to improve
- 06Help determine appropriate quality trade-offs as Spotify expands into additional languages and regions, considering the product use case and needs of each market
- 07Partner closely with voice experts, machine learning teams, product partners and vendors to maintain a consistent quality bar across demanding and sometimes competing priorities
- 08Surface the most meaningful gaps in model performance and use evaluation insights to help influence future research and technical priorities
Requirements
- 01You have experience in product quality, evaluation, data curation or a closely related field where human judgment is used to assess model or product performance
- 02You know how to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics
- 03You are comfortable designing and running both qualitative and MOS-style evaluations and synthesizing the results into actionable insights
- 04You have an understanding of voice or audio quality, including qualities such as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly
- 05You have strong judgment and can maintain a consistent quality bar while balancing different product requirements, markets and stakeholder perspectives
- 06You are comfortable working autonomously, identifying what needs to be evaluated and establishing the processes and cadence to make that evaluation useful over time
- 07You communicate clearly across creative, technical and product teams and can act as a reliable point of contact for internal partners and external vendors
- 08You are curious about generative AI, speech technology and the evolving AI voice landscape and can translate changes in the industry into meaningful benchmarks for Spotify
What we offer
- 01This role is based in London
- 02We offer you the flexibility to work where you work best
- 03There will be some in person meetings, but still allows for flexibility to work from home
- 04Spotify is an equal opportunity employer
- 05We are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone