DeepAI AI Chat
Log In Sign Up

MET: Masked Encoding for Tabular Data

06/17/2022
by   Kushal Majmundar, et al.
Google
Carnegie Mellon University
4

We consider the task of self-supervised representation learning (SSL) for tabular data: tabular-SSL. Typical contrastive learning based SSL methods require instance-wise data augmentations which are difficult to design for unstructured tabular data. Existing tabular-SSL methods design such augmentations in a relatively ad-hoc fashion and can fail to capture the underlying data manifold. Instead of augmentations based approaches for tabular-SSL, we propose a new reconstruction based method, called Masked Encoding for Tabular Data (MET), that does not require augmentations. MET is based on the popular MAE approach for vision-SSL [He et al., 2021] and uses two key ideas: (i) since each coordinate in a tabular dataset has a distinct meaning, we need to use separate representations for all coordinates, and (ii) using an adversarial reconstruction loss in addition to the standard one. Empirical results on five diverse tabular datasets show that MET achieves a new state of the art (SOTA) on all of these datasets and improves up to 9 current SOTA methods. We shed more light on the working of MET via experiments on carefully designed simple datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/16/2022

Improving self-supervised representation learning via sequential adversarial masking

Recent methods in self-supervised learning have demonstrated that maskin...
10/18/2022

Towards Efficient and Effective Self-Supervised Learning of Visual Representations

Self-supervision has emerged as a propitious method for visual represent...
01/31/2022

Adversarial Masking for Self-Supervised Learning

We propose ADIOS, a masked image model (MIM) framework for self-supervis...
02/12/2021

Bootstrapped Representation Learning on Graphs

Current state-of-the-art self-supervised learning methods for graph neur...
10/17/2022

Unifying Graph Contrastive Learning with Flexible Contextual Scopes

Graph contrastive learning (GCL) has recently emerged as an effective le...
07/14/2022

Benchmarking Omni-Vision Representation through the Lens of Visual Realms

Though impressive performance has been achieved in specific visual realm...
09/28/2021

PSI: Constructing ad-hoc Simplices to Interpolate High-Dimensional Unstructured Data

Interpolating unstructured data using barycentric coordinates becomes in...