Representation Learning by Reconstructing Neighborhoods

by   Chin-Chia Michael Yeh, et al.

Since its introduction, unsupervised representation learning has attracted a lot of attention from the research community, as it is demonstrated to be highly effective and easy-to-apply in tasks such as dimension reduction, clustering, visualization, information retrieval, and semi-supervised learning. In this work, we propose a novel unsupervised representation learning framework called neighbor-encoder, in which domain knowledge can be easily incorporated into the learning process without modifying the general encoder-decoder architecture of the classic autoencoder.In contrast to autoencoder, which reconstructs the input data itself, neighbor-encoder reconstructs the input data's neighbors. As the proposed representation learning problem is essentially a neighbor reconstruction problem, domain knowledge can be easily incorporated in the form of an appropriate definition of similarity between objects. Based on that observation, our framework can leverage any off-the-shelf similarity search algorithms or side information to find the neighbor of an input object. Applications of other algorithms (e.g., association rule mining) in our framework are also possible, given that the appropriate definition of neighbor can vary in different contexts. We have demonstrated the effectiveness of our framework in many diverse domains, including images, text, and time series, and for various data mining tasks including classification, clustering, and visualization. Experimental results show that neighbor-encoder not only outperforms autoencoder in most of the scenarios we consider, but also achieves the state-of-the-art performance on text document clustering.


page 6

page 7


Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning

We propose a symmetric graph convolutional autoencoder which produces a ...

NECA: Network-Embedded Deep Representation Learning for Categorical Data

We propose NECA, a deep representation learning method for categorical d...

Unsupervised Image Classification for Deep Representation Learning

Deep clustering against self-supervised learning is a very important and...

Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery

Discovering out-of-domain (OOD) intent is important for developing new s...

DAC: Deep Autoencoder-based Clustering, a General Deep Learning Framework of Representation Learning

Clustering performs an essential role in many real world applications, s...

Provable benefits of representation learning

There is general consensus that learning representations is useful for a...

Please sign up or login with your details

Forgot password? Click here to reset