DeepAI
Log In Sign Up

SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing

07/03/2021
by   Hui Li, et al.
6

Deep learning provides a promising way to extract effective representations from raw data in an end-to-end fashion and has proven its effectiveness in various domains such as computer vision, natural language processing, etc. However, in domains such as content/product recommendation and risk management, where sequence of event data is the most used raw data form and experts derived features are more commonly used, deep learning models struggle to dominate the game. In this paper, we propose a symbolic testing framework that helps to answer the question of what kinds of expert-derived features could be learned by a neural network. Inspired by this testing framework, we introduce an efficient architecture named SHORING, which contains two components: event network and sequence network. The event network learns arbitrarily yet efficiently high-order event-level embeddings via a provable reparameterization trick, the sequence network aggregates from sequence of event-level embeddings. We argue that SHORING is capable of learning certain standard symbolic expressions which the standard multi-head self-attention network fails to learn, and conduct comprehensive experiments and ablation studies on four synthetic datasets and three real-world datasets. The results show that SHORING empirically outperforms the state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/17/2020

End-to-End Neural Event Coreference Resolution

Traditional event coreference systems usually rely on pipeline framework...
11/28/2019

Self-attention with Functional Time Representation Learning

Sequential modelling with self-attention has achieved cutting edge perfo...
10/29/2018

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

Click-through rate (CTR) prediction, which aims to predict the probabili...
09/07/2020

A Hybrid Neuro-Symbolic Approach for Complex Event Processing

Training a model to detect patterns of interrelated events that form sit...
04/12/2018

DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction

Learning sophisticated feature interactions behind user behaviors is cri...
12/09/2022

Towards High-Order Complementary Recommendation via Logical Reasoning Network

Complementary recommendation gains increasing attention in e-commerce si...
06/29/2018

Topology classification with deep learning to improve real-time event selection at the LHC

We show how event topology classification based on deep learning could b...