research
∙
02/03/2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
We propose a new class of linear Transformers called FourierLearner-Tran...
share
research
∙
01/23/2023
Rethinking the Expressive Power of GNNs via Graph Biconnectivity
Designing expressive Graph Neural Networks (GNNs) is a central topic in ...
research
∙
10/04/2022
One Transformer Can Understand Both 2D 3D Molecular Data
Unlike vision and language data which usually has a unique format, molec...
research
∙
05/26/2022
Your Transformer May Not be as Powerful as You Expect
Relative Positional Encoding (RPE), which encodes the relative distance ...
research
∙
03/09/2022
Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets
This technical note describes the recent updates of Graphormer, includin...
research
∙
02/28/2022
An Empirical Study of Graphormer on Large-Scale Molecular Modeling Datasets
This technical note describes the recent updates of Graphormer, includin...
research
∙
06/23/2021
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
The attention module, which is a crucial component in Transformer, canno...
research
∙
06/15/2021
First Place Solution of KDD Cup 2021 OGB Large-Scale Challenge Graph-Level Track
In this technical report, we present our solution of KDD Cup 2021 OGB La...
research
∙
06/09/2021
Do Transformers Really Perform Bad for Graph Representation?
The Transformer architecture has become a dominant choice in many domain...
research
∙
02/16/2021
Revisiting Language Encoding in Learning Multilingual Representations
Transformer has demonstrated its great power to learn contextual word re...
research
∙
09/07/2020