A Framework for Evaluating Dashboards in Healthcare

09/10/2020
by   Mengdie Zhuang, et al.
0

In the era of "information overload", effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, data dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place a heavy requirement on dashboards as information systems, resulting in poor usability and ineffective design. Understanding these shortfalls is a challenge given the absence of a consistent and comprehensive approach to dashboard evaluation. In this paper we systematically review literature on dashboard implementation in the healthcare domain, a field where dashboards have been employed widely, and in which there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with a dashboard task-based classification, which underpin our subsequent analysis. From a total of 81 papers we derive seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the subtle and less well posed questions. We conclude by outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.

READ FULL TEXT

page 3

page 9

research
07/24/2023

A conceptual framework for SPI evaluation

Software Process Improvement (SPI) encompasses the analysis and modifica...
research
11/28/2022

Bayesian Network Models of Causal Interventions in Healthcare Decision Making: Literature Review and Software Evaluation

This report summarises the outcomes of a systematic literature search to...
research
08/31/2023

Strategies for engaging clinical participants in the co-design of software for healthcare domains

Co-design is an effective method for designing software, but implementin...
research
11/19/2021

Blockchain for Genomics: A Systematic Literature Review

Human genomic data carry unique information about an individual and offe...
research
08/03/2023

SoK: Assessing the State of Applied Federated Machine Learning

Machine Learning (ML) has shown significant potential in various applica...
research
01/30/2018

A State-of-the-Art of Semantic Change Computation

This paper reviews the state-of-the-art of semantic change computation, ...
research
04/13/2022

COCTEAU: an Empathy-Based Tool for Decision-Making

Traditional approaches to data-informed policymaking are often tailored ...

Please sign up or login with your details

Forgot password? Click here to reset