Unsupervised Visual Representation Learning with Increasing Object Shape Bias

by   Zhibo Wang, et al.
University of Central Florida
Michigan State University

(Very early draft)Traditional supervised learning keeps pushing convolution neural network(CNN) achieving state-of-art performance. However, lack of large-scale annotation data is always a big problem due to the high cost of it, even ImageNet dataset is over-fitted by complex models now. The success of unsupervised learning method represented by the Bert model in natural language processing(NLP) field shows its great potential. And it makes that unlimited training samples becomes possible and the great universal generalization ability changes NLP research direction directly. In this article, we purpose a novel unsupervised learning method based on contrastive predictive coding. Under that, we are able to train model with any non-annotation images and improve model's performance to reach state-of-art performance at the same level of model complexity. Beside that, since the number of training images could be unlimited amplification, an universal large-scale pre-trained computer vision model is possible in the future.


page 3

page 4


Representation Learning with Contrastive Predictive Coding

While supervised learning has enabled great progress in many application...

Universal Sentence Representation Learning with Conditional Masked Language Model

This paper presents a novel training method, Conditional Masked Language...

Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification

Unsupervised learning and supervised learning are key research topics in...

Learning Finer-class Networks for Universal Representations

Many real-world visual recognition use-cases can not directly benefit fr...

Big Learning: A Universal Machine Learning Paradigm?

Recent breakthroughs based on big/foundation models reveal a vague avenu...

Unsupervised Learning for Large-Scale Fiber Detection and Tracking in Microscopic Material Images

Constructing 3D structures from serial section data is a long standing p...

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

We propose split-brain autoencoders, a straightforward modification of t...

Please sign up or login with your details

Forgot password? Click here to reset