DeepAI AI Chat
Log In Sign Up

Measuring Generalization with Optimal Transport

by   Ching-Yao Chuang, et al.

Understanding the generalization of deep neural networks is one of the most important tasks in deep learning. Although much progress has been made, theoretical error bounds still often behave disparately from empirical observations. In this work, we develop margin-based generalization bounds, where the margins are normalized with optimal transport costs between independent random subsets sampled from the training distribution. In particular, the optimal transport cost can be interpreted as a generalization of variance which captures the structural properties of the learned feature space. Our bounds robustly predict the generalization error, given training data and network parameters, on large scale datasets. Theoretically, we demonstrate that the concentration and separation of features play crucial roles in generalization, supporting empirical results in the literature. The code is available at <>.


Inductive Semi-supervised Learning Through Optimal Transport

In this paper, we tackle the inductive semi-supervised learning problem ...

Instance-Dependent Generalization Bounds via Optimal Transport

Existing generalization bounds fail to explain crucial factors that driv...

k-Mixup Regularization for Deep Learning via Optimal Transport

Mixup is a popular regularization technique for training deep neural net...

Predicting Out-of-Distribution Error with Confidence Optimal Transport

Out-of-distribution (OOD) data poses serious challenges in deployed mach...

An Optimal Transport Kernel for Feature Aggregation and its Relationship to Attention

We introduce a kernel for sets of features based on an optimal transport...

Representation Based Complexity Measures for Predicting Generalization in Deep Learning

Deep Neural Networks can generalize despite being significantly overpara...

Regularized Optimal Transport Layers for Generalized Global Pooling Operations

Global pooling is one of the most significant operations in many machine...

Code Repositories


Code for Measuring Generalization with Optimal Transport

view repo