DeepAI AI Chat
Log In Sign Up

Representation Learning on Large and Small Data

by   Chun-Nan Chou, et al.

Deep learning owes its success to three key factors: scale of data, enhanced models to learn representations from data, and scale of computation. This book chapter presented the importance of the data-driven approach to learn good representations from both big data and small data. In terms of big data, it has been widely accepted in the research community that the more data the better for both representation and classification improvement. The question is then how to learn representations from big data, and how to perform representation learning when data is scarce. We addressed the first question by presenting CNN model enhancements in the aspects of representation, optimization, and generalization. To address the small data challenge, we showed transfer representation learning to be effective. Transfer representation learning transfers the learned representation from a source domain where abundant training data is available to a target domain where training data is scarce. Transfer representation learning gave the OM and melanoma diagnosis modules of our XPRIZE Tricorder device (which finished 2^nd out of 310 competing teams) a significant boost in diagnosis accuracy.


page 16

page 17


Representation Learning for Medical Data

We propose a representation learning framework for medical diagnosis dom...

Toward Improved Generalization: Meta Transfer of Self-supervised Knowledge on Graphs

Despite the remarkable success achieved by graph convolutional networks ...

Task Guided Compositional Representation Learning for ZDA

Zero-shot domain adaptation (ZDA) methods aim to transfer knowledge abou...

Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction

Deep learning algorithms excel at extracting patterns from raw data. Thr...

A Distributed Deep Representation Learning Model for Big Image Data Classification

This paper describes an effective and efficient image classification fra...

Improving Password Guessing via Representation Learning

Learning useful representations from unstructured data is one of the cor...

Combining Representation Learning with Logic for Language Processing

The current state-of-the-art in many natural language processing and aut...