Searching to Sparsify Tensor Decomposition for N-ary Relational Data

by   Shimin Di, et al.

Tensor, an extension of the vector and matrix to the multi-dimensional case, is a natural way to describe the N-ary relational data. Recently, tensor decomposition methods have been introduced into N-ary relational data and become state-of-the-art on embedding learning. However, the performance of existing tensor decomposition methods is not as good as desired. First, they suffer from the data-sparsity issue since they can only learn from the N-ary relational data with a specific arity, i.e., parts of common N-ary relational data. Besides, they are neither effective nor efficient enough to be trained due to the over-parameterization problem. In this paper, we propose a novel method, i.e., S2S, for effectively and efficiently learning from the N-ary relational data. Specifically, we propose a new tensor decomposition framework, which allows embedding sharing to learn from facts with mixed arity. Since the core tensors may still suffer from the over-parameterization, we propose to reduce parameters by sparsifying the core tensors while retaining their expressive power using neural architecture search (NAS) techniques, which can search for data-dependent architectures. As a result, the proposed S2S not only guarantees to be expressive but also efficiently learns from mixed arity. Finally, empirical results have demonstrated that S2S is efficient to train and achieves state-of-the-art performance.



There are no comments yet.


page 1

page 2

page 3

page 4


Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases

With the rapid development of knowledge bases (KBs), link prediction tas...

Logistic Tensor Factorization for Multi-Relational Data

Tensor factorizations have become increasingly popular approaches for va...

Tensor Decompositions for Modeling Inverse Dynamics

Modeling inverse dynamics is crucial for accurate feedforward robot cont...

Time-Aware Tensor Decomposition for Missing Entry Prediction

Given a time-evolving tensor with missing entries, how can we effectivel...

Efficient Learning of Mixed Membership Models

We present an efficient algorithm for learning mixed membership models w...

Regularized Orthogonal Tensor Decompositions for Multi-Relational Learning

Multi-relational learning has received lots of attention from researcher...

Robust Tensor Decomposition for Image Representation Based on Generalized Correntropy

Traditional tensor decomposition methods, e.g., two dimensional principa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.