Training Neural Networks to Produce Compatible Features

04/08/2020
by   Michael Gygli, et al.
0

This paper makes a first step towards compatible and hence reusable network components. Rather than training networks for different tasks independently, we adapt the training process to produce network components that are compatible across tasks. We propose and compare several different approaches to accomplish compatibility. Our experiments on CIFAR-10 show that: (i) we can train networks to produce compatible features, without degrading task accuracy compared to training networks independently; (ii) the degree of compatibility is highly dependent on where we split the network into a feature extractor and a classification head; (iii) random initialization has a large effect on compatibility; (iv) we can train incrementally: given previously trained components, we can train new ones which are also compatible with them. This work is part of a larger goal to increase network reusability: we envision that compatibility will enable solving new tasks by mixing and matching suitable components.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

03/26/2020

Towards Backward-Compatible Representation Learning

We propose a way to learn visual features that are compatible with previ...
09/02/2017

An Automated Compatibility Prediction Engine using DISC Theory Based Classification and Neural Networks

Traditionally psychometric tests were used for profiling incoming worker...
04/15/2020

Roommate Compatibility Detection Through Machine Learning Techniques

Our objective is to develop an artificially intelligent system which aim...
04/29/2021

Compatibility of Partitions, Hierarchies, and Split Systems

The question whether a partition 𝒫 and a hierarchy ℋ or a tree-like spli...
11/15/2021

CoReS: Compatible Representations via Stationarity

In this paper, we propose a novel method to learn internal feature repre...
03/14/2020

Compatible Learning for Deep Photonic Neural Network

Realization of deep learning with coherent optical field has attracted r...
08/05/2020

Neural Loop Combiner: Neural Network Models for Assessing the Compatibility of Loops

Music producers who use loops may have access to thousands in loop libra...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.