A Continuation Method for Discrete Optimization and its Application to Nearest Neighbor Classification

by   Ali Shameli, et al.

The continuation method is a popular approach in non-convex optimization and computer vision. The main idea is to start from a simple function that can be minimized efficiently, and gradually transform it to the more complicated original objective function. The solution of the simpler problem is used as the starting point to solve the original problem. We show a continuation method for discrete optimization problems. Ideally, we would like the evolved function to be hill-climbing friendly and to have the same global minima as the original function. We show that the proposed continuation method is the best affine approximation of a transformation that is guaranteed to transform the function to a hill-climbing friendly function and to have the same global minima. We show the effectiveness of the proposed technique in the problem of nearest neighbor classification. Although nearest neighbor methods are often competitive in terms of sample efficiency, the computational complexity in the test phase has been a major obstacle in their applicability in big data problems. Using the proposed continuation method, we show an improved graph-based nearest neighbor algorithm. The method is readily understood and easy to implement. We show how the computational complexity of the method in the test phase scales gracefully with the size of the training set, a property that is particularly important in big data applications.


page 1

page 2

page 3

page 4


k-NN Graph Construction: a Generic Online Approach

Nearest neighbor search and k-nearest neighbor graph construction are tw...

Sample Efficient Graph-Based Optimization with Noisy Observations

We study sample complexity of optimizing "hill-climbing friendly" functi...

Finding Relevant Points for Nearest-Neighbor Classification

In nearest-neighbor classification problems, a set of d-dimensional trai...

A Hierarchical Multi-Output Nearest Neighbor Model for Multi-Output Dependence Learning

Multi-Output Dependence (MOD) learning is a generalization of standard c...

Mass Classification Method in Mammogram Using Fuzzy K-Nearest Neighbour Equality

Mass classification of objects is an important area of research and appl...

Recognizing Cuneiform Signs Using Graph Based Methods

The cuneiform script constitutes one of the earliest systems of writing ...

Accelerating Large-Scale Graph-based Nearest Neighbor Search on a Computational Storage Platform

K-nearest neighbor search is one of the fundamental tasks in various app...

Please sign up or login with your details

Forgot password? Click here to reset