Finding Strong Gravitational Lenses Through Self-Attention

10/18/2021
by   Hareesh Thuruthipilly, et al.
14

The upcoming large scale surveys are expected to find approximately 10^5 strong gravitational systems by analyzing data of many orders of magnitude than the current era. In this scenario, non-automated techniques will be highly challenging and time-consuming. We propose a new automated architecture based on the principle of self-attention to find strong gravitational lensing. The advantages of self-attention based encoder models over convolution neural networks are investigated and encoder models are analyzed to optimize performance. We constructed 21 self-attention based encoder models and four convolution neural networks trained to identify gravitational lenses from the Bologna Lens Challenge. Each model is trained separately using 18,000 simulated images, cross-validated using 2 000 images, and then applied to a test set with 100 000 images. We used four different metrics for evaluation: classification accuracy, the area under the receiver operating characteristic curve (AUROC), the TPR_0 score and the TPR_10 score. The performance of the self-attention based encoder models and CNN's participated in the challenge are compared. The encoder models performed better than the CNNs and surpassed the CNN models that participated in the bologna lens challenge by a high margin for the TPR_0 and TPR_10. In terms of the AUROC, the encoder models scored equivalent to the top CNN model by only using one-sixth parameters to that of the CNN. Self-Attention based models have a clear advantage compared to simpler CNNs. A low computational cost and complexity make it a highly competing architecture to currently used residual neural networks. Moreover, introducing the encoder layers can also tackle the over-fitting problem present in the CNN's by acting as effective filters.

READ FULL TEXT

page 2

page 6

page 9

page 10

research
04/23/2018

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

Current end-to-end machine reading and question answering (Q&A) models a...
research
10/09/2022

Strong Gravitational Lensing Parameter Estimation with Vision Transformer

Quantifying the parameters and corresponding uncertainties of hundreds o...
research
10/07/2019

Improving Relation Extraction with Knowledge-attention

While attention mechanisms have been proven to be effective in many NLP ...
research
11/28/2022

FsaNet: Frequency Self-attention for Semantic Segmentation

Considering the spectral properties of images, we propose a new self-att...
research
07/24/2019

Self-attention based BiLSTM-CNN classifier for the prediction of ischemic and non-ischemic cardiomyopathy

Approximately 26 million individuals are suffering from heart failure, a...
research
10/24/2020

Blind Deinterleaving of Signals in Time Series with Self-attention Based Soft Min-cost Flow Learning

We propose an end-to-end learning approach to address deinterleaving of ...

Please sign up or login with your details

Forgot password? Click here to reset