Interactive Visual Data Exploration with Subjective Feedback: An Information-Theoretic Approach

10/23/2017
by   Kai Puolamäki, et al.
0

Visual exploration of high-dimensional real-valued datasets is a fundamental task in exploratory data analysis (EDA). Existing methods use predefined criteria to choose the representation of data. There is a lack of methods that (i) elicit from the user what she has learned from the data and (ii) show patterns that she does not know yet. We construct a theoretical model where identified patterns can be input as knowledge to the system. The knowledge syntax here is intuitive, such as "this set of points forms a cluster", and requires no knowledge of maths. This background knowledge is used to find a Maximum Entropy distribution of the data, after which the system provides the user data projections in which the data and the Maximum Entropy distribution differ the most, hence showing the user aspects of the data that are maximally informative given the user's current knowledge. We provide an open source EDA system with tailored interactive visualizations to demonstrate these concepts. We study the performance of the system and present use cases on both synthetic and real data. We find that the model and the prototype system allow the user to learn information efficiently from various data sources and the system works sufficiently fast in practice. We conclude that the information theoretic approach to exploratory data analysis where patterns observed by a user are formalized as constraints provides a principled, intuitive, and efficient basis for constructing an EDA system.

READ FULL TEXT

page 7

page 8

research
05/07/2019

Guided Visual Exploration of Relations in Data Sets

Efficient explorative data analysis systems must take into account both ...
research
05/20/2018

Human-guided data exploration using randomisation

An explorative data analysis system should be aware of what the user alr...
research
10/12/2017

Subjectively Interesting Subgroup Discovery on Real-valued Targets

Deriving insights from high-dimensional data is one of the core problems...
research
04/23/2010

STORM - A Novel Information Fusion and Cluster Interpretation Technique

Analysis of data without labels is commonly subject to scrutiny by unsup...
research
04/09/2018

Human-Guided Data Exploration

The outcome of the explorative data analysis (EDA) phase is vital for su...
research
04/25/2019

Summarizing Data Succinctly with the Most Informative Itemsets

Knowledge discovery from data is an inherently iterative process. That i...
research
04/25/2019

Tell Me What I Need to Know: Succinctly Summarizing Data with Itemsets

Data analysis is an inherently iterative process. That is, what we know ...

Please sign up or login with your details

Forgot password? Click here to reset