DeepAI AI Chat
Log In Sign Up

Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness

by   Caner Hazirbas, et al.
The Hong Kong University of Science and Technology

Developing robust and fair AI systems require datasets with comprehensive set of labels that can help ensure the validity and legitimacy of relevant measurements. Recent efforts, therefore, focus on collecting person-related datasets that have carefully selected labels, including sensitive characteristics, and consent forms in place to use those attributes for model testing and development. Responsible data collection involves several stages, including but not limited to determining use-case scenarios, selecting categories (annotations) such that the data are fit for the purpose of measuring algorithmic bias for subgroups and most importantly ensure that the selected categories/subcategories are robust to regional diversities and inclusive of as many subgroups as possible. Meta, in a continuation of our efforts to measure AI algorithmic bias and robustness (, is working on collecting a large consent-driven dataset with a comprehensive list of categories. This paper describes our proposed design of such categories and subcategories for Casual Conversations v2.


page 1

page 2

page 3

page 4


More Data Can Lead Us Astray: Active Data Acquisition in the Presence of Label Bias

An increased awareness concerning risks of algorithmic bias has driven a...

A Seven-Layer Model for Standardising AI Fairness Assessment

Problem statement: Standardisation of AI fairness rules and benchmarks i...

Measurement as governance in and for responsible AI

Measurement of social phenomena is everywhere, unavoidably, in sociotech...

Algorithmic Fairness

An increasing number of decisions regarding the daily lives of human bei...