Regularized Optimal Transport Layers for Generalized Global Pooling Operations

12/13/2022
by   Hongteng Xu, et al.
0

Global pooling is one of the most significant operations in many machine learning models and tasks, which works for information fusion and structured data (like sets and graphs) representation. However, without solid mathematical fundamentals, its practical implementations often depend on empirical mechanisms and thus lead to sub-optimal, even unsatisfactory performance. In this work, we develop a novel and generalized global pooling framework through the lens of optimal transport. The proposed framework is interpretable from the perspective of expectation-maximization. Essentially, it aims at learning an optimal transport across sample indices and feature dimensions, making the corresponding pooling operation maximize the conditional expectation of input data. We demonstrate that most existing pooling methods are equivalent to solving a regularized optimal transport (ROT) problem with different specializations, and more sophisticated pooling operations can be implemented by hierarchically solving multiple ROT problems. Making the parameters of the ROT problem learnable, we develop a family of regularized optimal transport pooling (ROTP) layers. We implement the ROTP layers as a new kind of deep implicit layer. Their model architectures correspond to different optimization algorithms. We test our ROTP layers in several representative set-level machine learning scenarios, including multi-instance learning (MIL), graph classification, graph set representation, and image classification. Experimental results show that applying our ROTP layers can reduce the difficulty of the design and selection of global pooling – our ROTP layers may either imitate some existing global pooling methods or lead to some new pooling layers fitting data better. The code is available at <https://github.com/SDS-Lab/ROT-Pooling>.

READ FULL TEXT

page 2

page 10

page 14

page 15

research
01/23/2022

Revisiting Pooling through the Lens of Optimal Transport

Pooling is one of the most significant operations in many machine learni...
research
06/22/2020

An Optimal Transport Kernel for Feature Aggregation and its Relationship to Attention

We introduce a kernel for sets of features based on an optimal transport...
research
03/04/2018

Greedy stochastic algorithms for entropy-regularized optimal transport problems

Optimal transport (OT) distances are finding evermore applications in ma...
research
05/08/2023

Earth Movers in The Big Data Era: A Review of Optimal Transport in Machine Learning

Optimal Transport (OT) is a mathematical framework that first emerged in...
research
08/18/2023

Generalized Sum Pooling for Metric Learning

A common architectural choice for deep metric learning is a convolutiona...
research
02/16/2020

Differentiable Top-k Operator with Optimal Transport

The top-k operation, i.e., finding the k largest or smallest elements fr...
research
02/24/2022

Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies

Deep declarative networks and other recent related works have shown how ...

Please sign up or login with your details

Forgot password? Click here to reset