Privacy Preserving Visual Question Answering

02/15/2022
by   Cristian-Paul Bara, et al.
4

We introduce a novel privacy-preserving methodology for performing Visual Question Answering on the edge. Our method constructs a symbolic representation of the visual scene, using a low-complexity computer vision model that jointly predicts classes, attributes and predicates. This symbolic representation is non-differentiable, which means it cannot be used to recover the original image, thereby keeping the original image private. Our proposed hybrid solution uses a vision model which is more than 25 times smaller than the current state-of-the-art (SOTA) vision models, and 100 times smaller than end-to-end SOTA VQA models. We report detailed error analysis and discuss the trade-offs of using a distilled vision model and a symbolic representation of the visual scene.

READ FULL TEXT

page 2

page 5

page 10

page 11

page 12

page 13

research
10/04/2018

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

We marry two powerful ideas: deep representation learning for visual rec...
research
10/26/2022

Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems

For vision-and-language reasoning tasks, both fully connectionist, end-t...
research
06/11/2020

Privacy-Preserving Visual Feature Descriptors through Adversarial Affine Subspace Embedding

Many computer vision systems require users to upload image features to t...
research
06/19/2018

Learning Conditioned Graph Structures for Interpretable Visual Question Answering

Visual Question answering is a challenging problem requiring a combinati...
research
12/12/2016

VIBIKNet: Visual Bidirectional Kernelized Network for Visual Question Answering

In this paper, we address the problem of visual question answering by pr...
research
03/22/2021

How to Design Sample and Computationally Efficient VQA Models

In multi-modal reasoning tasks, such as visual question answering (VQA),...

Please sign up or login with your details

Forgot password? Click here to reset