Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity

05/01/2023
by   Josh Gardner, et al.
0

Modern machine learning increasingly supports paradigms that are multi-institutional (using data from multiple institutions during training) or cross-institutional (using models from multiple institutions for inference), but the empirical effects of these paradigms are not well understood. This study investigates cross-institutional learning via an empirical case study in higher education. We propose a framework and metrics for assessing the utility and fairness of student dropout prediction models that are transferred across institutions. We examine the feasibility of cross-institutional transfer under real-world data- and model-sharing constraints, quantifying model biases for intersectional student identities, characterizing potential disparate impact due to these biases, and investigating the impact of various cross-institutional ensembling approaches on fairness and overall model performance. We perform this analysis on data representing over 200,000 enrolled students annually from four universities without sharing training data between institutions. We find that a simple zero-shot cross-institutional transfer procedure can achieve similar performance to locally-trained models for all institutions in our study, without sacrificing model fairness. We also find that stacked ensembling provides no additional benefits to overall performance or fairness compared to either a local model or the zero-shot transfer procedure we tested. We find no evidence of a fairness-accuracy tradeoff across dozens of models and transfer schemes evaluated. Our auditing procedure also highlights the importance of intersectional fairness analysis, revealing performance disparities at the intersection of sensitive identity groups that are concealed under one-dimensional analysis.

READ FULL TEXT

page 8

page 13

page 17

page 18

page 20

research
07/03/2017

Discriminatory Transfer

We observe standard transfer learning can improve prediction accuracies ...
research
03/28/2021

Should College Dropout Prediction Models Include Protected Attributes?

Early identification of college dropouts can provide tremendous value fo...
research
05/14/2021

Towards Equity and Algorithmic Fairness in Student Grade Prediction

Equity of educational outcome and fairness of AI with respect to race ha...
research
08/22/2022

Evaluation of group fairness measures in student performance prediction problems

Predicting students' academic performance is one of the key tasks of edu...
research
08/21/2023

FairBench: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models

Detecting stereotypes and biases in Large Language Models (LLMs) can enh...
research
07/24/2020

Cross-study learning for generalist and specialist predictions

Jointly using data from multiple similar sources for the training of pre...
research
09/10/2017

Institutionally Distributed Deep Learning Networks

Deep learning has become a promising approach for automated medical diag...

Please sign up or login with your details

Forgot password? Click here to reset