Characterizing the invariances of learning algorithms using category theory

05/06/2019
by   Kenneth D. Harris, et al.
0

Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are formalized by categories of predictor and target spaces, whose morphisms represent the algorithm's invariances, and an index category whose morphisms represent permutations of the training examples. An invariant learning algorithm is a natural transformation between two functors from the product of these categories to the category of sets, representing training datasets and learned functions respectively. We illustrate the framework by characterizing and contrasting the invariances of linear regression and ridge regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/19/2022

A Note on Categories about Rough Sets

Using the concepts of category and functor, we provide some insights and...
research
11/28/2017

Backprop as Functor: A compositional perspective on supervised learning

A supervised learning algorithm searches over a set of functions A → B p...
research
06/05/2023

Towards a theory of natural directed paths

We introduce the abstract setting of presheaf category on a thick catego...
research
04/25/2023

Dynamic Tracing: a graphical language for rewriting protocols

The category Set_* of sets and partial functions is well-known to be tra...
research
03/05/2019

Lenses and Learners

Lenses are a well-established structure for modelling bidirectional tran...
research
10/30/2016

Discovering containment: from infants to machines

Current artificial learning systems can recognize thousands of visual ca...
research
11/28/2019

Follow the Flow: sets, relations, and categories as special cases of functions with no domain

We introduce, develop, and apply a new approach for dealing with the int...

Please sign up or login with your details

Forgot password? Click here to reset