Geometry-aware Transformer for molecular property prediction

by   Bumju Kwak, et al.

Recently, graph neural networks (GNNs) have achieved remarkable performances for quantum mechanical problems. However, a graph convolution can only cover a localized region, and cannot capture long-range interactions of atoms. This behavior is contrary to theoretical interatomic potentials, which is a fundamental limitation of the spatial based GNNs. In this work, we propose a novel attention-based framework for molecular property prediction tasks. We represent a molecular conformation as a discrete atomic sequence combined by atom-atom distance attributes, named Geometry-aware Transformer (GeoT). In particular, we adopt a Transformer architecture, which has been widely used for sequential data. Our proposed model trains sequential representations of molecular graphs based on globally constructed attentions, maintaining all spatial arrangements of atom pairs. Our method does not suffer from cost intensive computations, such as angle calculations. The experimental results on several public benchmarks and visualization maps verified that keeping the long-range interatomic attributes can significantly improve the model predictability.


ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction

Effective molecular representation learning is of great importance to fa...

Hierarchical graph neural nets can capture long-range interactions

Graph neural networks (GNNs) based on message passing between neighborin...

Path-Augmented Graph Transformer Network

Much of the recent work on learning molecular representations has been b...

Physical Pooling Functions in Graph Neural Networks for Molecular Property Prediction

Graph neural networks (GNNs) are emerging in chemical engineering for th...

Geometric Transformer for End-to-End Molecule Properties Prediction

Transformers have become methods of choice in many applications thanks t...

Equivariant Graph Attention Networks for Molecular Property Prediction

Learning and reasoning about 3D molecular structures with varying size i...

3D-Transformer: Molecular Representation with Transformer in 3D Space

Spatial structures in the 3D space are important to determine molecular ...