b-Bit Minwise Hashing for Large-Scale Linear SVM

by   Ping Li, et al.

In this paper, we propose to (seamlessly) integrate b-bit minwise hashing with linear SVM to substantially improve the training (and testing) efficiency using much smaller memory, with essentially no loss of accuracy. Theoretically, we prove that the resemblance matrix, the minwise hashing matrix, and the b-bit minwise hashing matrix are all positive definite matrices (kernels). Interestingly, our proof for the positive definiteness of the b-bit minwise hashing kernel naturally suggests a simple strategy to integrate b-bit hashing with linear SVM. Our technique is particularly useful when the data can not fit in memory, which is an increasingly critical issue in large-scale machine learning. Our preliminary experimental results on a publicly available webspam dataset (350K samples and 16 million dimensions) verified the effectiveness of our algorithm. For example, the training time was reduced to merely a few seconds. In addition, our technique can be easily extended to many other linear and nonlinear machine learning applications such as logistic regression.



There are no comments yet.


page 1

page 2

page 3

page 4


Hashing Algorithms for Large-Scale Learning

In this paper, we first demonstrate that b-bit minwise hashing, whose es...

Training Logistic Regression and SVM on 200GB Data Using b-Bit Minwise Hashing and Comparisons with Vowpal Wabbit (VW)

We generated a dataset of 200 GB with 10^9 features, to test our recent ...

RiffleScrambler - a memory-hard password storing function

We introduce RiffleScrambler: a new family of directed acyclic graphs an...

Min-Max Kernels

The min-max kernel is a generalization of the popular resemblance kernel...

On b-bit min-wise hashing for large-scale regression and classification with sparse data

Large-scale regression problems where both the number of variables, p, a...

One Permutation Hashing for Efficient Search and Learning

Recently, the method of b-bit minwise hashing has been applied to large-...

The Bitwise Hashing Trick for Personalized Search

Many real world problems require fast and efficient lexical comparison o...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.