Understanding attention in graph neural networks

05/08/2019
by   Boris Knyazev, et al.
0

We aim to better understand attention over nodes in graph neural networks and identify factors influencing its effectiveness. Motivated by insights from the work on Graph Isomorphism Networks (Xu et al., 2019), we design simple graph reasoning tasks that allow us to study attention in a controlled environment. We find that under typical conditions the effect of attention is negligible or even harmful, but under certain conditions it provides an exceptional gain in performance of more than 40 have yet to satisfy these conditions in practice.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset