DeepVA: Bridging Cognition and Computation through Semantic Interaction and Deep Learning

07/31/2020
by   Yali Bian, et al.
0

This paper examines how deep learning (DL) representations, in contrast to traditional engineered features, can support semantic interaction (SI) in visual analytics. SI attempts to model user's cognitive reasoning via their interaction with data items, based on the data features. We hypothesize that DL representations contain meaningful high-level abstractions that can better capture users' high-level cognitive intent. To bridge the gap between cognition and computation in visual analytics, we propose DeepVA (Deep Visual Analytics), which uses high-level deep learning representations for semantic interaction instead of low-level hand-crafted data features. To evaluate DeepVA and compare to SI models with lower-level features, we design and implement a system that extends a traditional SI pipeline with features at three different levels of abstraction. To test the relationship between task abstraction and feature abstraction in SI, we perform visual concept learning tasks at three different task abstraction levels, using semantic interaction with three different feature abstraction levels. DeepVA effectively hastened interactive convergence between cognitive understanding and computational modeling of the data, especially in high abstraction tasks.

READ FULL TEXT

page 1

page 5

page 7

research
05/26/2023

DeepSI: Interactive Deep Learning for Semantic Interaction

In this paper, we design novel interactive deep learning methods to impr...
research
11/17/2017

Deep Local Binary Patterns

Local Binary Pattern (LBP) is a traditional descriptor for texture analy...
research
12/02/2021

Neurosymbolic Systems of Perception Cognition: The Role of Attention

A cognitive architecture aimed at cumulative learning must provide the n...
research
02/13/2023

Mixed Multi-Model Semantic Interaction for Graph-based Narrative Visualizations

Narrative sensemaking is an essential part of understanding sequential d...
research
06/16/2021

Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective

Visual analytics have played an increasingly critical role in the Intern...
research
10/08/2020

Generating Instructions at Different Levels of Abstraction

When generating technical instructions, it is often convenient to descri...
research
08/12/2020

We Have So Much In Common: Modeling Semantic Relational Set Abstractions in Videos

Identifying common patterns among events is a key ability in human and m...

Please sign up or login with your details

Forgot password? Click here to reset