Leveraging Medical Visual Question Answering with Supporting Facts

05/28/2019
by   Tomasz Kornuta, et al.
0

In this working notes paper, we describe IBM Research AI (Almaden) team's participation in the ImageCLEF 2019 VQA-Med competition. The challenge consists of four question-answering tasks based on radiology images. The diversity of imaging modalities, organs and disease types combined with a small imbalanced training set made this a highly complex problem. To overcome these difficulties, we implemented a modular pipeline architecture that utilized transfer learning and multi-task learning. Our findings led to the development of a novel model called Supporting Facts Network (SFN). The main idea behind SFN is to cross-utilize information from upstream tasks to improve the accuracy on harder downstream ones. This approach significantly improved the scores achieved in the validation set (18 point improvement in F-1 score). Finally, we submitted four runs to the competition and were ranked seventh.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2018

R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering

Recently, Visual Question Answering (VQA) has emerged as one of the most...
research
03/23/2018

Studio Ousia's Quiz Bowl Question Answering System

In this chapter, we describe our question answering system, which was th...
research
07/03/2020

Visual Question Answering as a Multi-Task Problem

Visual Question Answering(VQA) is a highly complex problem set, relying ...
research
08/07/2017

ISS-MULT: Intelligent Sample Selection for Multi-Task Learning in Question Answering

Transferring knowledge from a source domain to another domain is useful,...
research
05/19/2021

Multiple Meta-model Quantifying for Medical Visual Question Answering

Transfer learning is an important step to extract meaningful features an...
research
12/17/2020

MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering

In this paper, we introduce MIX : a multi-task deep learning approach to...

Please sign up or login with your details

Forgot password? Click here to reset