Generalizing in the Real World with Representation Learning

10/18/2022
by   Tegan Maharaj, et al.
0

Machine learning (ML) formalizes the problem of getting computers to learn from experience as optimization of performance according to some metric(s) on a set of data examples. This is in contrast to requiring behaviour specified in advance (e.g. by hard-coded rules). Formalization of this problem has enabled great progress in many applications with large real-world impact, including translation, speech recognition, self-driving cars, and drug discovery. But practical instantiations of this formalism make many assumptions - for example, that data are i.i.d.: independent and identically distributed - whose soundness is seldom investigated. And in making great progress in such a short time, the field has developed many norms and ad-hoc standards, focused on a relatively small range of problem settings. As applications of ML, particularly in artificial intelligence (AI) systems, become more pervasive in the real world, we need to critically examine these assumptions, norms, and problem settings, as well as the methods that have become de-facto standards. There is much we still do not understand about how and why deep networks trained with stochastic gradient descent are able to generalize as well as they do, why they fail when they do, and how they will perform on out-of-distribution data. In this thesis I cover some of my work towards better understanding deep net generalization, identify several ways assumptions and problem settings fail to generalize to the real world, and propose ways to address those failures in practice.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/22/2021

Applications of artificial intelligence in drug development using real-world data

The US Food and Drug Administration (FDA) has been actively promoting th...
research
09/28/2022

Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applications

The exceptional progress in the field of machine learning (ML) in recent...
research
06/08/2022

Automating Ambiguity: Challenges and Pitfalls of Artificial Intelligence

Machine learning (ML) and artificial intelligence (AI) tools increasingl...
research
06/20/2023

Guideline for Trustworthy Artificial Intelligence – AI Assessment Catalog

Artificial Intelligence (AI) has made impressive progress in recent year...
research
03/16/2021

MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases

The emerging age of connected, digital world means that there are tons o...
research
10/20/2020

Where Is the Normative Proof? Assumptions and Contradictions in ML Fairness Research

Across machine learning (ML) sub-disciplines researchers make mathematic...
research
03/03/2021

Out of Distribution Generalization in Machine Learning

Machine learning has achieved tremendous success in a variety of domains...

Please sign up or login with your details

Forgot password? Click here to reset