DeepAI AI Chat
Log In Sign Up

Improvements to context based self-supervised learning

by   T. Nathan Mundhenk, et al.
Lawrence Livermore National Laboratory
berkeley college

We develop a set of methods to improve on the results of self-supervised learning using context. We start with a baseline of patch based arrangement context learning and go from there. Our methods address some overt problems such as chromatic aberration as well as other potential problems such as spatial skew and mid-level feature neglect. We prevent problems with testing generalization on common self-supervised benchmark tests by using different datasets during our development. The results of our methods combined yield top scores on all standard self-supervised benchmarks, including classification and detection on PASCAL VOC 2007, segmentation on PASCAL VOC 2012, and "linear tests" on the ImageNet and CSAIL Places datasets. We obtain an improvement over our baseline method of between 4.0 to 7.1 percentage points on transfer learning classification tests. We also show results on different standard network architectures to demonstrate generalization as well as portability.


page 3

page 4

page 5

page 12

page 13

page 14


Self-supervised Learning with Fully Convolutional Networks

Although deep learning based methods have achieved great success in many...

Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics

We introduce a novel principle for self-supervised feature learning base...

Self-supervised similarity models based on well-logging data

Adopting data-based approaches leads to model improvement in numerous Oi...

Learning Visual Representations for Transfer Learning by Suppressing Texture

Recent literature has shown that features obtained from supervised train...

What Do We Maximize in Self-Supervised Learning?

In this paper, we examine self-supervised learning methods, particularly...

Transfer Learning Application of Self-supervised Learning in ARPES

Recent development in angle-resolved photoemission spectroscopy (ARPES) ...

Multi-task Self-Supervised Visual Learning

We investigate methods for combining multiple self-supervised tasks--i.e...