DeepAI AI Chat
Log In Sign Up

A Shallow High-Order Parametric Approach to Data Visualization and Compression

by   Martin Renqiang Min, et al.
National Research Council Canada
NEC Laboratories America
University of California, San Diego

Explicit high-order feature interactions efficiently capture essential structural knowledge about the data of interest and have been used for constructing generative models. We present a supervised discriminative High-Order Parametric Embedding (HOPE) approach to data visualization and compression. Compared to deep embedding models with complicated deep architectures, HOPE generates more effective high-order feature mapping through an embarrassingly simple shallow model. Furthermore, two approaches to generating a small number of exemplars conveying high-order interactions to represent large-scale data sets are proposed. These exemplars in combination with the feature mapping learned by HOPE effectively capture essential data variations. Moreover, through HOPE, these exemplars are employed to increase the computational efficiency of kNN classification for fast information retrieval by thousands of times. For classification in two-dimensional embedding space on MNIST and USPS datasets, our shallow method HOPE with simple Sigmoid transformations significantly outperforms state-of-the-art supervised deep embedding models based on deep neural networks, and even achieved historically low test error rate of 0.65 which demonstrates the representational efficiency and power of supervised shallow models with high-order feature interactions.


Exemplar-Centered Supervised Shallow Parametric Data Embedding

Metric learning methods for dimensionality reduction in combination with...

Generalized Embedding Machines for Recommender Systems

Factorization machine (FM) is an effective model for feature-based recom...

Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery

Deep graph embedding is an important approach for community discovery. D...

Fast and More Powerful Selective Inference for Sparse High-order Interaction Model

Automated high-stake decision-making such as medical diagnosis requires ...

High Order Neural Networks for Video Classification

Capturing spatiotemporal correlations is an essential topic in video cla...

ContextNet: A Click-Through Rate Prediction Framework Using Contextual information to Refine Feature Embedding

Click-through rate (CTR) estimation is a fundamental task in personalize...

EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction

Learning effective high-order feature interactions is very crucial in th...