Compositional Deep Learning

07/16/2019
by   Bruno Gavranović, et al.
6

Neural networks have become an increasingly popular tool for solving many real-world problems. They are a general framework for differentiable optimization which includes many other machine learning approaches as special cases. In this thesis we build a category-theoretic formalism around a class of neural networks exemplified by CycleGAN. CycleGAN is a collection of neural networks, closed under composition, whose inductive bias is increased by enforcing composition invariants, i.e. cycle-consistencies. Inspired by Functorial Data Migration, we specify the interconnection of these networks using a categorical schema, and network instances as set-valued functors on this schema. We also frame neural network architectures, datasets, models, and a number of other concepts in a categorical setting and thus show a special class of functors, rather than functions, can be learned using gradient descent. We use the category-theoretic framework to conceive a novel neural network architecture whose goal is to learn the task of object insertion and object deletion in images with unpaired data. We test the architecture on three different datasets and obtain promising results.

READ FULL TEXT

page 11

page 38

page 39

page 41

page 42

research
09/15/2020

Learning Functors using Gradient Descent

Neural networks are a general framework for differentiable optimization ...
research
07/03/2019

Neural Network Architecture Search with Differentiable Cartesian Genetic Programming for Regression

The ability to design complex neural network architectures which enable ...
research
12/01/2022

Graph Convolutional Neural Networks as Parametric CoKleisli morphisms

We define the bicategory of Graph Convolutional Neural Networks 𝐆𝐂𝐍𝐍_n f...
research
03/23/2023

Learning and generalization of compositional representations of visual scenes

Complex visual scenes that are composed of multiple objects, each with a...
research
10/03/2018

Optimization Algorithm Inspired Deep Neural Network Structure Design

Deep neural networks have been one of the dominant machine learning appr...
research
12/22/2021

MC-DGCNN: A Novel DNN Architecture for Multi-Category Point Set Classification

Point set classification aims to build a representation learning model t...
research
06/28/2019

Category-Theoretic Foundations of "STCLang: State Thread Composition as a Foundation for Monadic Dataflow Parallelism"

This manuscript gives a category-theoretic foundation to the composition...

Please sign up or login with your details

Forgot password? Click here to reset