-
Deep Learning for Visual Tracking: A Comprehensive Survey
Visual target tracking is one of the most sought-after yet challenging r...
read it
-
Data-driven Random Fourier Features using Stein Effect
Large-scale kernel approximation is an important problem in machine lear...
read it
-
On Small-World Networks: Survey and Properties Analysis
Complex networks has been a hot topic of research over the past several ...
read it
-
Privacy in Deep Learning: A Survey
The ever-growing advances of deep learning in many areas including visio...
read it
-
A Survey on Approximation in Parameterized Complexity: Hardness and Algorithms
Parameterization and approximation are two popular ways of coping with N...
read it
-
A Unified Analysis of Random Fourier Features
We provide the first unified theoretical analysis of supervised learning...
read it
-
TSN Algorithms for Large Scale Networks: A Survey and Conceptual Comparison
This paper provides a comprehensive survey of queueing and scheduling me...
read it
Random Features for Kernel Approximation: A Survey in Algorithms, Theory, and Beyond
Random features is one of the most sought-after research topics in statistical machine learning to speed up kernel methods in large-scale situations. Related works have won the NeurIPS test-of-time award in 2017 and the ICML best paper finalist in 2019. However, comprehensive studies on this topic seem to be missing, which results in different, sometimes conflicting, statements. In this survey, we attempt to throughout and systematically review the past ten years work on random features regarding to algorithmic and theoretical aspects. First, the fundamental characteristics, primary motivations, and contributions of representative random features based algorithms are summarized according to their sampling scheme, learning procedure, variance reduction, and exploitation of training data. Second, we review theoretical results of random features to answer the key question: how many random features are needed to ensure a high approximation quality or no loss of empirical risk and expected risk in a learning estimator. Third, popular random features based algorithms are comprehensively evaluated on several large scale benchmark datasets on the approximation quality and the prediction performance for classification and regression. Last, we link random features to current over-parameterized deep neural networks (DNNs) by investigating their relationships, the usage of random features to analysis over-parameterized networks, and the gap in the current theoretical results. As a result, this survey could be a gentle use guide for practitioners to follow this topic, apply representative algorithms, and grasp theoretical results under various technical assumptions. We think that this survey helps to facilitate a discussion on ongoing issues for this topic, and specifically, it sheds light on promising research directions.
READ FULL TEXT
Comments
There are no comments yet.