A statistical machine learning approach for benchmarking in the presence of complex contextual factors and peer groups

11/17/2020
by   Daniel W. Kennedy, et al.
0

The ability to compare between individuals or organisations fairly is important for the development of robust and meaningful quantitative benchmarks. To make fair comparisons, contextual factors must be taken into account, and comparisons should only be made between similar organisations such as peer groups. Previous benchmarking methods have used linear regression to adjust for contextual factors, however linear regression is known to be sub-optimal when nonlinear relationships exist between the comparative measure and covariates. In this paper we propose a random forest model for benchmarking that can adjust for these potential nonlinear relationships, and validate the approach in a case-study of high noise data. We provide new visualisations and numerical summaries of the fitted models and comparative measures to facilitate interpretation by both analysts and non-technical audiences. Comparisons can be made across the cohort or within peer groups, and bootstrapping provides a means of estimating uncertainty in both adjusted measures and rankings. We conclude that random forest models can facilitate fair comparisons between organisations for quantitative measures including in cases on complex contextual factor relationships, and that the models and outputs are readily interpreted by stakeholders.

READ FULL TEXT

page 8

page 9

research
03/27/2023

Nonparametric approaches for analyzing carbon emission: from statistical and machine learning perspectives

Linear regression models, especially the extended STIRPAT model, are rou...
research
03/13/2018

A machine learning-based approach for estimating and testing associations with multivariate outcomes

We propose a method for summarizing the strength of association between ...
research
08/17/2017

Extensions of Morse-Smale Regression with Application to Actuarial Science

The problem of subgroups is ubiquitous in scientific research (ex. disea...
research
11/17/2020

Peer groups for organisational learning: clustering with practical constraints

Peer-grouping is used in many sectors for organisational learning, polic...
research
04/05/2023

Opening the random forest black box by the analysis of the mutual impact of features

Random forest is a popular machine learning approach for the analysis of...
research
12/30/2019

Wisdom of collaborators: a peer-review approach to performance appraisal

Individual performance and reputation within a company are major factors...
research
11/03/2016

Extracting Actionability from Machine Learning Models by Sub-optimal Deterministic Planning

A main focus of machine learning research has been improving the general...

Please sign up or login with your details

Forgot password? Click here to reset