OpenAI6 дней назад
Data Scientist по исследованиям в области справедливости ИИ
Зарплата не указана
РЫНОК
16 250 ₽медиана по профессии
Data Scientist · 212 вакансий с указанной зарплатой
5 000половина предложений: 13 084–21 768120 000
Работодатель не указал зарплату — сравните с рынком сами.
Полная занятостьУдалёнка
Обязанности
- 01Define and lead fairness and bias-testing strategies for AI-assisted People processes, models, agents, and decision-support systems from development through deployment and ongoing monitoring
- 02Design rigorous algorithmic audits and validation studies, including adverse-impact analysis, subgroup and intersectional evaluation, error-rate analysis, calibration, measurement invariance, reliability, criterion-related validity, and sensitivity testing
- 03Identify the appropriate fairness criteria for each use case, evaluate tradeoffs among competing definitions of fairness, and clearly document the assumptions, limitations, and residual risks of each approach
- 04Evaluate end-to-end human-AI decision systems, including model outputs, user behavior, human overrides, escalation pathways, and whether AI assistance changes the quality, consistency, or equity of decisions
- 05Develop evaluation approaches for generative and agentic AI, including test-set design, counterfactual testing, behavioral evaluation, human-rating studies, robustness testing, and analysis of disparate performance across populations and contexts
- 06Investigate the sources of observed disparities, including data representation, label and measurement bias, proxy variables, model design, decision thresholds, workflow design, and differential adoption or usage
- 07Partner with engineering, People Operations, Legal, Privacy, Security, and People Systems teams to recommend and evaluate mitigations such as data improvements, model changes, threshold adjustments, workflow redesign, monitoring controls, and additional human oversight
- 08Build scalable fairness-evaluation infrastructure, including reusable datasets, automated validation pipelines, regression tests, monitoring systems, self-service tools, and standardized reporting
- 09Establish research and documentation standards for fairness test plans, dataset and model documentation, validation reports, limitations, monitoring plans, and decision records
- 10Translate complex findings into concise, decision-ready narratives, helping leaders understand the significance of identified risks, the strength of the evidence, available mitigation options, and remaining uncertainty
Требования
- 01Deep expertise in algorithmic fairness, bias measurement, responsible AI, psychometrics, applied statistics, or the evaluation of high-impact decision systems
- 02Exceptional strength in research design, measurement, experimentation, causal inference, and statistical modeling
- 03Hands-on experience applying methods such as subgroup and intersectional analysis, adverse-impact testing, equalized-odds and equal-opportunity analysis, demographic-parity assessment, calibration analysis, counterfactual testing, measurement invariance, reliability analysis, and validation studies
- 04Strong judgment about the limitations of fairness metrics, including the ability to determine which measures are appropriate for a particular decision context rather than applying a single universal definition of fairness
- 05Experience evaluating machine-learning models, generative AI systems, agents, or human-AI workflows using quantitative and qualitative evidence
- 06High proficiency in Python or R and SQL, with experience working across complex, sensitive, and imperfect datasets
- 07Experience building reproducible evaluation pipelines, automated testing frameworks, analytical tools, monitoring systems, or governed research workflows
- 08Ability to distinguish statistical disparities from their potential causes and to communicate findings without overstating certainty or making unsupported causal or legal conclusions
- 09Ability to work effectively with technical, operational, legal, privacy, and executive stakeholders and influence consequential decisions through evidence and sound judgment
- 10Deep curiosity, intellectual humility, strong attention to detail, and a commitment to developing AI systems and organizational processes that work well for people across different backgrounds and circumstances
Условия
- 01Role preferred to be based in San Francisco, CA