Orthogonal Trace-Sum Maximization: Applications, Local Algorithms, and Global Optimality
share
This paper studies a problem of maximizing the sum of traces of matrix quadratic forms on a product of Stiefel manifolds. This orthogonal trace-sum maximization (OTSM) problem generalizes many interesting problems such as generalized canonical correlation analysis (CCA), Procrustes analysis, and cryo-electron microscopy of the Nobel prize fame. For these applications finding global solutions is highly desirable but has been out of reach for a long time. For example, generalizations of CCA do not possess obvious global solutions unlike their classical counterpart to which a global solution is readily obtained through singular value decomposition; it is also not clear how to test global optimality. We provide a simple method to certify global optimality of a given local solution. This method only requires testing the sign of the smallest eigenvalue of a symmetric matrix, and does not rely on a particular algorithm as long as it converges to a stationary point. Our certificate result relies on a semidefinite programming (SDP) relaxation of OTSM, but avoids solving an SDP of lifted dimensions. Surprisingly, a popular algorithm for generalized CCA and Procrustes analysis may generate oscillating iterates. We propose a simple modification of this standard algorithm and prove that it reliably converges. Our notion of convergence is stronger than conventional objective value convergence or subsequence convergence.The convergence result utilizes the Kurdyka-Lojasiewicz property of the problem.
READ FULL TEXT