Learning Sparse Deep Feedforward Networks via Tree Skeleton Expansion

03/16/2018
by   Zhourong Chen, et al.
0

Despite the popularity of deep learning, structure learning for deep models remains a relatively under-explored area. In contrast, structure learning has been studied extensively for probabilistic graphical models (PGMs). In particular, an efficient algorithm has been developed for learning a class of tree-structured PGMs called hierarchical latent tree models (HLTMs), where there is a layer of observed variables at the bottom and multiple layers of latent variables on top. In this paper, we propose a simple method for learning the structures of feedforward neural networks (FNNs) based on HLTMs. The idea is to expand the connections in the tree skeletons from HLTMs and to use the resulting structures for FNNs. An important characteristic of FNN structures learned this way is that they are sparse. We present extensive empirical results to show that, compared with standard FNNs tuned-manually, sparse FNNs learned by our method achieve better or comparable classification performance with much fewer parameters. They are also more interpretable.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/14/2018

Building Sparse Deep Feedforward Networks using Tree Receptive Fields

Sparse connectivity is an important factor behind the success of convolu...
research
07/25/2020

From Boltzmann Machines to Neural Networks and Back Again

Graphical models are powerful tools for modeling high-dimensional data, ...
research
11/03/2021

On the Effectiveness of Interpretable Feedforward Neural Network

Deep learning models have achieved state-of-the-art performance in many ...
research
01/16/2014

Nonparametric Latent Tree Graphical Models: Inference, Estimation, and Structure Learning

Tree structured graphical models are powerful at expressing long range o...
research
01/31/2019

Probabilistic Discriminative Learning with Layered Graphical Models

Probabilistic graphical models are traditionally known for their success...
research
09/30/2021

Strengthening Probabilistic Graphical Models: The Purge-and-merge Algorithm

Probabilistic graphical models (PGMs) are powerful tools for solving sys...
research
05/02/2015

Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models

Deep learning has shown state-of-art classification performance on datas...

Please sign up or login with your details

Forgot password? Click here to reset