research
∙
02/03/2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
We propose a new class of linear Transformers called FourierLearner-Tran...
share
research
∙
06/04/2022
Is L^2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
The Physics-Informed Neural Network (PINN) approach is a new and promisi...
research
∙
05/26/2022
Your Transformer May Not be as Powerful as You Expect
Relative Positional Encoding (RPE), which encodes the relative distance ...
research
∙
11/02/2021
Can Vision Transformers Perform Convolution?
Several recent studies have demonstrated that attention-based networks, ...
research
∙
06/23/2021