Merging of neural networks

04/21/2022
by   Martin Pašen, et al.
0

We propose a simple scheme for merging two neural networks trained with different starting initialization into a single one with the same size as the original ones. We do this by carefully selecting channels from each input network. Our procedure might be used as a finalization step after one tries multiple starting seeds to avoid an unlucky one. We also show that training two networks and merging them leads to better performance than training a single network for an extended period of time. Availability: https://github.com/fmfi-compbio/neural-network-merging

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/28/2023

An Empirical Study of Multimodal Model Merging

Model merging (e.g., via interpolation or task arithmetic) fuses multipl...
research
11/03/2020

Towards a Universal Gating Network for Mixtures of Experts

The combination and aggregation of knowledge from multiple neural networ...
research
06/02/2023

Resolving Interference When Merging Models

Transfer learning - i.e., further fine-tuning a pre-trained model on a d...
research
01/24/2020

A Branching and Merging Convolutional Network with Homogeneous Filter Capsules

We present a convolutional neural network design with additional branche...
research
07/28/2020

Admissible ways of merging p-values under arbitrary dependence

Methods of merging several p-values into a single p-value are important ...
research
03/02/2021

DeepMerge II: Building Robust Deep Learning Algorithms for Merging Galaxy Identification Across Domains

In astronomy, neural networks are often trained on simulation data with ...
research
06/09/2023

Revisiting Permutation Symmetry for Merging Models between Different Datasets

Model merging is a new approach to creating a new model by combining the...

Please sign up or login with your details

Forgot password? Click here to reset