Learning Sign-Constrained Support Vector Machines

01/05/2021
by   Kenya Tajima, et al.
0

Domain knowledge is useful to improve the generalization performance of learning machines. Sign constraints are a handy representation to combine domain knowledge with learning machine. In this paper, we consider constraining the signs of the weight coefficients in learning the linear support vector machine, and develop two optimization algorithms for minimizing the empirical risk under the sign constraints. One of the two algorithms is based on the projected gradient method, in which each iteration of the projected gradient method takes O(nd) computational cost and the sublinear convergence of the objective error is guaranteed. The second algorithm is based on the Frank-Wolfe method that also converges sublinearly and possesses a clear termination criterion. We show that each iteration of the Frank-Wolfe also requires O(nd) cost. Furthermore, we derive the explicit expression for the minimal iteration number to ensure an ϵ-accurate solution by analyzing the curvature of the objective function. Finally, we empirically demonstrate that the sign constraints are a promising technique when similarities to the training examples compose the feature vector.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2023

Algorithms for the Training of Neural Support Vector Machines

Neural support vector machines (NSVMs) allow for the incorporation of do...
research
10/12/2017

Sign-Constrained Regularized Loss Minimization

In practical analysis, domain knowledge about analysis target has often ...
research
10/12/2016

Exploring the Entire Regularization Path for the Asymmetric Cost Linear Support Vector Machine

We propose an algorithm for exploring the entire regularization path of ...
research
05/12/2023

On the Fair Comparison of Optimization Algorithms in Different Machines

An experimental comparison of two or more optimization algorithms requir...
research
05/30/2010

Empirical learning aided by weak domain knowledge in the form of feature importance

Standard hybrid learners that use domain knowledge require stronger know...
research
04/13/2020

Scenario optimization with relaxation: a new tool for design and application to machine learning problems

Scenario optimization is by now a well established technique to perform ...
research
08/31/2023

Least Squares Maximum and Weighted Generalization-Memorization Machines

In this paper, we propose a new way of remembering by introducing a memo...

Please sign up or login with your details

Forgot password? Click here to reset