Unsupervised model compression for multilayer bootstrap networks

03/22/2015
by   Xiao-Lei Zhang, et al.
0

Recently, multilayer bootstrap network (MBN) has demonstrated promising performance in unsupervised dimensionality reduction. It can learn compact representations in standard data sets, i.e. MNIST and RCV1. However, as a bootstrap method, the prediction complexity of MBN is high. In this paper, we propose an unsupervised model compression framework for this general problem of unsupervised bootstrap methods. The framework compresses a large unsupervised bootstrap model into a small model by taking the bootstrap model and its application together as a black box and learning a mapping function from the input of the bootstrap model to the output of the application by a supervised learner. To specialize the framework, we propose a new technique, named compressive MBN. It takes MBN as the unsupervised bootstrap model and deep neural network (DNN) as the supervised learner. Our initial result on MNIST showed that compressive MBN not only maintains the high prediction accuracy of MBN but also is over thousands of times faster than MBN at the prediction stage. Our result suggests that the new technique integrates the effectiveness of MBN on unsupervised learning and the effectiveness and efficiency of DNN on supervised learning together for the effectiveness and efficiency of compressive MBN on unsupervised learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/24/2019

Deep topic modeling by multilayer bootstrap network and lasso

Topic modeling is widely studied for the dimension reduction and analysi...
research
07/04/2021

A Comparison of the Delta Method and the Bootstrap in Deep Learning Classification

We validate the recently introduced deep learning classification adapted...
research
10/24/2019

Multi-channel Speech Separation Using Deep Embedding Model with Multilayer Bootstrap Networks

Recently, deep clustering (DPCL) based speaker-independent speech separa...
research
01/04/2021

Multi-Model Least Squares-Based Recomputation Framework for Large Data Analysis

Most multilayer least squares (LS)-based neural networks are structured ...
research
03/27/2020

Learning representations in Bayesian Confidence Propagation neural networks

Unsupervised learning of hierarchical representations has been one of th...
research
08/05/2014

Multilayer bootstrap networks

Multilayer bootstrap network builds a gradually narrowed multilayer nonl...
research
10/18/2021

Unsupervised Learned Kalman Filtering

In this paper we adapt KalmanNet, which is a recently pro-posed deep neu...

Please sign up or login with your details

Forgot password? Click here to reset