Solving Tensor Low Cycle Rank Approximation

by   Yichuan Deng, et al.

Large language models have become ubiquitous in modern life, finding applications in various domains such as natural language processing, language translation, and speech recognition. Recently, a breakthrough work [Zhao, Panigrahi, Ge, and Arora Arxiv 2023] explains the attention model from probabilistic context-free grammar (PCFG). One of the central computation task for computing probability in PCFG is formulating a particular tensor low rank approximation problem, we can call it tensor cycle rank. Given an n × n × n third order tensor A, we say that A has cycle rank-k if there exists three n × k^2 size matrices U , V, and W such that for each entry in each A_a,b,c = ∑_i=1^k ∑_j=1^k ∑_l=1^k U_a,i+k(j-1)⊗ V_b, j + k(l-1)⊗ W_c, l + k(i-1) for all a ∈ [n], b ∈ [n], c ∈ [n]. For the tensor classical rank, tucker rank and train rank, it has been well studied in [Song, Woodruff, Zhong SODA 2019]. In this paper, we generalize the previous “rotation and sketch” technique in page 186 of [Song, Woodruff, Zhong SODA 2019] and show an input sparsity time algorithm for cycle rank.


page 1

page 2

page 3

page 4


A Tensor Rank Theory and Maximum Full Rank Subtensors

A matrix always has a full rank submatrix such that the rank of this mat...

Experimental observation on a low-rank tensor model for eigenvalue problems

Here we utilize a low-rank tensor model (LTM) as a function approximator...

Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization

There has been an increased interest in multimodal language processing i...

Kriging in Tensor Train data format

Combination of low-tensor rank techniques and the Fast Fourier transform...

Finding the smallest or largest element of a tensor from its low-rank factors

We consider the problem of finding the smallest or largest entry of a te...

Low-Rank Tucker Approximation of a Tensor From Streaming Data

This paper describes a new algorithm for computing a low-Tucker-rank app...

Block-Term Tensor Decomposition: Model Selection and Computation

The so-called block-term decomposition (BTD) tensor model has been recen...

Please sign up or login with your details

Forgot password? Click here to reset