Machine Learning State-of-the-Art with Uncertainties

04/11/2022
by   Peter Steinbach, et al.
38

With the availability of data, hardware, software ecosystem and relevant skill sets, the machine learning community is undergoing a rapid development with new architectures and approaches appearing at high frequency every year. In this article, we conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results as well as impact the reviewing process. In addition, we explore the hallmarks and limitations of this approximation. We discuss the relevance of this approach reflecting on a spotlight publication of ICLR22. A reproducible workflow is made available as an open-source adjoint to this publication. Based on our discussion, we make suggestions for improving the authoring and reviewing process of machine learning articles.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/16/2021

In Defense of the Paper

The machine learning publication process is broken, of that there can be...
research
05/26/2015

MLlib: Machine Learning in Apache Spark

Apache Spark is a popular open-source platform for large-scale data proc...
research
09/14/2020

A matter of time: publication dates in Web of Science Core Collection

Web of Science Core Collection, one of the most authoritative bibliograp...
research
07/08/2021

SmartReviews: Towards Human- and Machine-actionable Reviews

Review articles summarize state-of-the-art work and provide a means to o...
research
03/10/2023

What is the state of the art? Accounting for multiplicity in machine learning benchmark performance

Machine learning methods are commonly evaluated and compared by their pe...
research
07/20/2017

ShortScience.org - Reproducing Intuition

We present ShortScience.org, a platform for post-publication discussion ...
research
01/15/2023

A data science and machine learning approach to continuous analysis of Shakespeare's plays

The availability of quantitative methods that can analyze text has provi...

Please sign up or login with your details

Forgot password? Click here to reset