Projection-free Graph-based Classifier Learning using Gershgorin Disc Perfect Alignment
share
In semi-supervised graph-based binary classifier learning, a subset of known labels x̂_i are used to infer unknown labels, assuming that the label signal x is smooth with respect to a similarity graph specified by a Laplacian matrix. When restricting labels x_i to binary values, the problem is NP-hard. While a conventional semi-definite programming (SDP) relaxation can be solved in polynomial time using, for example, the alternating direction method of multipliers (ADMM), the complexity of iteratively projecting a candidate matrix M onto the positive semi-definite (PSD) cone (M ≽ 0) remains high. In this paper, leveraging a recent linear algebraic theory called Gershgorin disc perfect alignment (GDPA), we propose a fast projection-free method by solving a sequence of linear programs (LP) instead. Specifically, we first recast the SDP relaxation to its SDP dual, where a feasible solution H ≽ 0 can be interpreted as a Laplacian matrix corresponding to a balanced signed graph sans the last node. To achieve graph balance, we split the last node into two that respectively contain the original positive and negative edges, resulting in a new Laplacian H̅. We repose the SDP dual for solution H̅, then replace the PSD cone constraint H̅≽ 0 with linear constraints derived from GDPA – sufficient conditions to ensure H̅ is PSD – so that the optimization becomes an LP per iteration. Finally, we extract predicted labels from our converged LP solution H̅. Experiments show that our algorithm enjoyed a 40× speedup on average over the next fastest scheme while retaining comparable label prediction performance.
READ FULL TEXT