A Transfer Principle: Universal Approximators Between Metric Spaces From Euclidean Universal Approximators

04/24/2023
βˆ™
by   Anastasis Kratsios, et al.
βˆ™
0
βˆ™

We build universal approximators of continuous maps between arbitrary Polish metric spaces 𝒳 and 𝒴 using universal approximators between Euclidean spaces as building blocks. Earlier results assume that the output space 𝒴 is a topological vector space. We overcome this limitation by "randomization": our approximators output discrete probability measures over 𝒴. When 𝒳 and 𝒴 are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for HΓΆlder-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of 𝒳 and 𝒴. For barycentric 𝒴, including Banach spaces, ℝ-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to 𝒴-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.

READ FULL TEXT

page 3

page 4

page 15

research
βˆ™ 06/03/2020

Non-Euclidean Universal Approximation

Modifications to a neural network's input and output layers are often re...
research
βˆ™ 01/31/2022

Metric Hypertransformers are Universal Adapted Maps

We introduce a universal class of geometric deep learning models, called...
research
βˆ™ 01/13/2021

Quantitative Rates and Fundamental Obstructions to Non-Euclidean Universal Approximation with Deep Narrow Feed-Forward Networks

By incorporating structured pairs of non-trainable input and output laye...
research
βˆ™ 04/27/2023

Universal Algebra for Generalised Metric Spaces

We study in this work a generalisation of the framework of quantitative ...
research
βˆ™ 04/26/2018

Universal approximations of invariant maps by neural networks

We describe generalizations of the universal approximation theorem for n...
research
βˆ™ 06/03/2019

Approximation capability of neural networks on spaces of probability measures and tree-structured domains

This paper extends the proof of density of neural networks in the space ...
research
βˆ™ 07/18/2020

Combinatorial and computational investigations of Neighbor-Joining bias

The Neighbor-Joining algorithm is a popular distance-based phylogenetic ...

Please sign up or login with your details

Forgot password? Click here to reset