Modularity as a Means for Complexity Management in Neural Networks Learning

02/25/2019
by   David Castillo-Bolado, et al.
0

Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN is decomposed into a control module and several functional modules, implementing primitive operations. We illustrate the modular concept by comparing performances between a monolithic and a modular NN on a list sorting problem and show the benefits in terms of training speed, training stability and maintainability. We also discuss some questions that arise in modular NNs.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset