Low-Rank Tensor Decomposition via Multiple Reshaping and Reordering Operations

05/22/2018 ∙ by Chao Li, et al. ∙ 0

Tensor decomposition has been widely applied to find low-rank representations for real-world data and more recently for neural-network parameters too. For the latter, the unfolded matrices may not always be low-rank because the modes of the parameter tensor do not usually have any physical meaning that can be exploited for efficiency. This raises the following question: how can we find low-rank structures when the tensor modes do not have any physical meaning associated with them? For this purpose, we propose a new decomposition method in this paper. Our method uses reshaping and reordering operations that are strictly more general than the unfolding operation. These operations enable us to discover new low-rank structures that are beyond the reach of existing tensor methods. We prove an important theoretical result establishing conditions under which our method results in a unique solution. The experimental results confirm the correctness of our theoretical works and the effectiveness of our methods for weight compression in deep neural networks.



There are no comments yet.


page 9

page 10

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.