Modeling Winner-Take-All Competition in Sparse Binary Projections

07/27/2019
by   Wenye Li, et al.
0

Inspired by the advances in biological science, the study of sparse binary projection models has attracted considerable recent research attention. The models project dense input samples into a higher-dimensional space and output sparse binary vectors after Winner-Take-All competition, subject to the constraint that the projection matrix is also sparse and binary. Following the work along this line, we developed a supervised-WTA model under the supervised setting where training samples with both input and output representations are available, from which the projection matrix can be obtained with a simple, efficient yet effective algorithm. We further extended the model and the algorithm to an unsupervised setting where only the input representation of the samples is available. In a series of empirical evaluation on similarity search tasks, both models reported significantly improved results over the state-of-the-art methods in both search accuracy and running time. The successful results give us strong confidence that the proposed work provides a highly practical tool to real world applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/29/2020

Binary Random Projections with Controllable Sparsity Patterns

Random projection is often used to project higher-dimensional vectors on...
research
11/05/2019

Sparse Lifting of Dense Vectors: Unifying Word and Sentence Representations

As the first step in automated natural language processing, representing...
research
12/18/2019

Comparison of Classification Methods for Very High-Dimensional Data in Sparse Random Projection Representation

The big data trend has inspired feature-driven learning tasks, which can...
research
09/09/2017

Simultaneously Learning Neighborship and Projection Matrix for Supervised Dimensionality Reduction

Explicitly or implicitly, most of dimensionality reduction methods need ...
research
01/14/2020

Bio-Inspired Hashing for Unsupervised Similarity Search

The fruit fly Drosophila's olfactory circuit has inspired a new locality...
research
08/14/2019

On the Robustness of Projection Neural Networks For Efficient Text Representation: An Empirical Study

Recently, there has been strong interest in developing natural language ...
research
06/13/2017

Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks

Recommendation algorithms that incorporate techniques from deep learning...

Please sign up or login with your details

Forgot password? Click here to reset