Log In Sign Up

The Generalized Eigenvalue Problem as a Nash Equilibrium

by   Ian Gemp, et al.

The generalized eigenvalue problem (GEP) is a fundamental concept in numerical linear algebra. It captures the solution of many classical machine learning problems such as canonical correlation analysis, independent components analysis, partial least squares, linear discriminant analysis, principal components, successor features and others. Despite this, most general solvers are prohibitively expensive when dealing with massive data sets and research has instead concentrated on finding efficient solutions to specific problem instances. In this work, we develop a game-theoretic formulation of the top-k GEP whose Nash equilibrium is the set of generalized eigenvectors. We also present a parallelizable algorithm with guaranteed asymptotic convergence to the Nash. Current state-of-the-art methods require 𝒪(d^2k) complexity per iteration which is prohibitively expensive when the number of dimensions (d) is large. We show how to achieve 𝒪(dk) complexity, scaling to datasets 100× larger than those evaluated by prior methods. Empirically we demonstrate that our algorithm is able to solve a variety of GEP problem instances including a large-scale analysis of neural network activations.


Fast Complete Algorithm for Multiplayer Nash Equilibrium

We describe a new complete algorithm for computing Nash equilibrium in m...

Stochastic generalized Nash equilibrium seeking under partial-decision information

We consider for the first time a stochastic generalized Nash equilibrium...

First-Order Algorithms for Nonlinear Generalized Nash Equilibrium Problems

We consider the problem of computing an equilibrium in a class of nonlin...

A Generalized EigenGame with Extensions to Multiview Representation Learning

Generalized Eigenvalue Problems (GEPs) encompass a range of interesting ...

The Computation of Approximate Generalized Feedback Nash Equilibria

We present the concept of a Generalized Feedback Nash Equilibrium (GFNE)...

Robust SVM Optimization in Banach spaces

We address the issue of binary classification in Banach spaces in presen...