The Scientific Method in the Science of Machine Learning

by   Jessica Zosa Forde, et al.
Columbia University

In the quest to align deep learning with the sciences to address calls for rigor, safety, and interpretability in machine learning systems, this contribution identifies key missing pieces: the stages of hypothesis formulation and testing, as well as statistical and systematic uncertainty estimation -- core tenets of the scientific method. This position paper discusses the ways in which contemporary science is conducted in other domains and identifies potentially useful practices. We present a case study from physics and describe how this field has promoted rigor through specific methodological practices, and provide recommendations on how machine learning researchers can adopt these practices into the research ecosystem. We argue that both domain-driven experiments and application-agnostic questions of the inner workings of fundamental building blocks of machine learning models ought to be examined with the tools of the scientific method, to ensure we not only understand effect, but also begin to understand cause, which is the raison d'être of science.


page 1

page 2

page 3

page 4


Machine Learning Interpretability: A Science rather than a tool

The term "interpretability" is oftenly used by machine learning research...

Data Centred Intelligent Geosciences: Research Agenda and Opportunities, Position Paper

This paper describes and discusses our vision to develop and reason abou...

Towards A Rigorous Science of Interpretable Machine Learning

As machine learning systems become ubiquitous, there has been a surge of...

Gamification in Science: A Study of Requirements in the Context of Reproducible Research

The need for data preservation and reproducible research is widely recog...

Developing and Publishing Code for Trusted Research Environments: Best Practices and Ways of Working

This report discusses 3 distinct, but overlapping topics. Firstly, it re...

Statistical modeling: the three cultures

Two decades ago, Leo Breiman identified two cultures for statistical mod...

Levels of Analysis for Machine Learning

Machine learning is currently involved in some of the most vigorous deba...

Please sign up or login with your details

Forgot password? Click here to reset