notation
None
view repo
We propose a notation for tensors with named axes, which relieves the author, reader, and future implementers from the burden of keeping track of the order of axes and the purpose of each. It also makes it easy to extend operations on low-order tensors to higher order ones (e.g., to extend an operation on images to minibatches of images, or extend the attention mechanism to multiple attention heads). After a brief overview of our notation, we illustrate it through several examples from modern machine learning, from building blocks like attention and convolution to full models like Transformers and LeNet. Finally, we give formal definitions and describe some extensions. Our proposals build on ideas from many previous papers and software libraries. We hope that this document will encourage more authors to use named tensors, resulting in clearer papers and less bug-prone implementations. The source code for this document can be found at https://github.com/namedtensor/notation/. We invite anyone to make comments on this proposal by submitting issues or pull requests on this repository.
READ FULL TEXT
Tensors are higher-order extensions of matrices. In recent work [Kilmer ...
read it
Numerical applications and, more recently, machine learning applications...
read it
Tensor network methods are a conceptually elegant framework for encoding...
read it
Tensors are multidimensional arrays of numerical values and therefore
ge...
read it
The success of deep learning often derives from well-chosen operational
...
read it
We present a new framework for computing Z-eigenvectors of general tenso...
read it
A common challenge in scientific and technical domains is the quantitati...
read it
None
Comments
There are no comments yet.