A survey on Information Visualization in light of Vision and Cognitive sciences

05/26/2015 ∙ by Jose Rodrigues-Jr, et al. ∙ 0

Information Visualization techniques are built on a context with many factors related to both vision and cognition, making it difficult to draw a clear picture of how data visually turns into comprehension. In the intent of promoting a better picture, here, we survey concepts on vision, cognition, and Information Visualization organized in a theorization named Visual Expression Process. Our theorization organizes the basis of visualization techniques with a reduced level of complexity; still, it is complete enough to foster discussions related to design and analytical tasks. Our work introduces the following contributions: (1) a Theoretical compilation of vision, cognition, and Information Visualization; (2) Discussions supported by vast literature; and (3) Reflections on visual-cognitive aspects concerning use and design. We expect our contributions will provide further clarification about how users and designers think about InfoVis, leveraging the potential of systems and techniques.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

page 18

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Understanding why and how visual representations work is an important issue addressed in many works on Information Visualization (Dill et al., 2012)(Thomas & Cook, 2005). On the track of this issue, in this work, we assume that well-designed visualizations must stimulate visual and cognitive processes in a way that reasoning is amplified. The effective promotion of such reasoning depends on principles mastered in the sciences of vision and cognition: vision is the gate through which information derived from computer graphics reaches the brain; cognition refers to the processing that is induced by such graphics. Vision and cognition are closely intertwined, a fact to be considered in the design of visualizations. Accordingly, understanding Information Visualization (InfoVis for short) in light of these sciences may improve design principles that, usually, are performed intuitively. To this end, we review the steps that take place during vision-cognition phenomena when the goal is data analysis. In survey fashion, we compile the literature introducing the following contributions:

  • Theoretical compendium: we draw a descriptive relationship between vision, cognition, and Information Visualization;

  • Discussions: we debate our rationalizations over extensive literature;

  • Reflections: we provide study cases to revisit design practices.

We organize visualization concepts aiming at the needs pointed out by Johnson et al. (2006), who recommend the characterization of how and why visualizations work, and by Scaife & Rogers (1996), who stress the importance of the cognitive aspects underlying visualizations. Furthermore, we review principles for visual representations that, according to Card et al. (1999), are an initial step towards more effective visualization techniques, a demand defended by Wong et al. (2012).

We note that, while we build an association between Information Visualization, vision, and cognitive sciences, we do not reach a definitive settlement. This is because neither vision nor cognition are yet fully understood. Rather, constrained to the current state of the art, we introduce an organizational process that discusses use and design.

2 Related work

The literature presents several models that lay the basis to Information Visualization. Bertin (1977/1981) introduced the concept of deriving visual structures from a set of fundamental components. Cleveland & McGill (1984), and Mackinlay (1986), conducted studies on the usefulness of visual patterns in the form of frameworks for design. Card et al. (1999) follow Bertin by discussing the importance of the spatial substrate. Keim (2002) suggests a taxonomical space for quick referencing; Shneiderman (1996) reasons about the possibilities of visualization and interaction; and Chi (2000) focuses on data transformations. In the realm of design, Bugajska (2003) deals with spatial and abstract visualizations considering the design guidelines of Tweedie (1997). In the line of works that reflect about the visualization field, van Wijk (2006) systematically discusses visualizations based on a cost-oriented analysis; and Green et al. (2009) research many cognitive and perceptual aspects (Rensink, 2000) to build a model and a set of guidelines for design. Patterson et al. (2014) introduces a framework based on vision and cognition sciences, similar to ours, focusing on top-down processes. Comparatively, we conduct a complementary bottom-up approach; we translate vision-cognition phenomena into an original vocabulary that may bring such science closer to the design practice.

To organize our compilation on vision and cognitive sciences, we depart from the Visualization Pipeline of Card et al. (1999)

to draw the Visual Expression Process, a sequence of events delineated by the possibilities of the visual-cognitive interplay. According to our organization, (1) vision phenomena (pre-attentive stimuli) determines a map of potential interesting objects. Then, attentive selection concentrates on one single element, part of a set of (2) analytical perceptions. Such perceptions go through a pattern-matching process that turns them into (3) abstract patterns that, in working memory, support cognition in combination with domain knowledge originated from long-term memory. Finally, leading to (4) cognitive decision support. Nevertheless, our process does not explain the intercourse between vision, cognition, and Information Visualization; this is not feasible considering the current knowledge, and neither it would fit in a single article.

The rest of the paper is organized as follows. Section 3 draws a detailed panorama of vision and cognition in the realm of Information Visualization. Section 4 introduces the Visual Expression Process, our organizational theorization; and the last section presents conclusive remarks.

3 Concepts on Vision, Cognition, and Visualization

According to Vision Science, the visual process has two stages, namely, the parallel extraction of low-level properties, called pre-attentive processing, followed by a slower detailed scan. The first stage promotes the major benefit of visualizations, that is, improved data comprehension (Triesman, 1985). Meanwhile, the second stage addresses conventional reading practices that do not contribute towards faster cognition, but that are necessary for further analysis. In fact, Ware (2004)

states that understanding what is processed pre-attentively is probably the most important contribution that Vision Science can bring to visualization.

This two-stage process might be the underlying motivation for the broadly referenced “visualization mantra”: overview first, zoom and filter, then details on demand (Shneiderman, 1996) – Schneiderman came to this conclusion by means of intuition and empirical experiments, never drawing the vision/cognitive reasons of why this is the case. We discuss this process in the following sections according to our process, presented in section 4.

3.1 Maps of Saliences

In the early stages of vision, the brain deals with the problem of casting potential elements of interest; regions of the scene that should be considered for cognition. To cope with that, a complex process takes place so that certain characteristics pop out to the eyes. These characteristics appear as saliences over the scene. They define the so-called maps of saliences, exemplified in Figure 1; the first element of our theoretical organization.

Figure 1: (a) Example of a map of saliences over the Gapminder tool. In the scene, the bubbles with outstanding features stand for saliences corresponding to countries of interest. (b) The result is a map of potential targets that pop out due to their color, or their shape.

The principle of salience is to reinforce the perception of the areas in the scene whose visual properties contrast with those of their surroundings (Itti & Koch, 2001). It is a process that considers different visual aspects, such as position and color, and that ends as it combines them onto a single scene (Nothdurft, 2000).

The occurrence of salient features stems from their interplay with other stimuli, depending on a context that favors conspicuousness. The brain is highly trained to detect such configurations, being able to track salient features in parallel, in real-time, and covering the entire visual field. There is also evidence that this ability is strongly influenced by the task-at-hand, in a top-down (cognition-to-vision) process (Navalpakkam & Itti, 2007) (Patterson et al., 2014).

3.2 Attention and Attentive Selection

Although the brain perceives visual targets simultaneously in a map of saliences, it cannot process all of them in parallel. This is considered a prohibitively computational task even to the most sophisticated brains (Tsotsos, 1991). Primates and other animals handle this by restricting the consideration of the objects presented to their eyes: their vision concentrates on small regions considering objects one after the other. This is a serialization process ruled by what is called attention. In other words, once a map of candidates is ready, it is necessary to “filter out” one of these candidates for attention. Attention, here, occurs in accordance with the task that the user is performing; the task determines the information demands and, consequently, what visual stimuli should be extracted from the scene. Trial and error is part of the process, which is iterative.

This process of filtering has been modeled as a pyramidal neuronal structure – named

selective tuning model (Cutzu & Tsotsos, 2003) (Essen et al., 1992). This theoretical model predicts a broad layer of neurons in its first level, narrowing down as it advances to upper layers. The layers intercommunicate through feed-forward and feedback connections and, according to Cutzu & Tsotsos (2003) (Tsotsos et al., 1995), a pyramid of neurons successively performs three stages of processing, illustrated in Figure 2: (a) bottom-up feed-forward, (b) top-down winner-take-all feedback (Lippmann, 1987), and (c) bottom-up straight path. This process explains what is broadly referenced as filtering or attentive selection.

Following this widely-accepted model, it is worthy to note that, as described, filtering occurs according to both bottom-up and top-down processes. The bottom-up process depends on the visual stimuli, while the top-down process depends on the task at hand and is prioritized (Wolfe, 1994). Each process influences the other iteratively. Actually, according to Patterson (2012), the entire visual-cognition interplay in influenced by top-down mechanisms that emanate from working memory, but that may be highly influenced by long-term memory (Woodman et al., 2013). In this work, we focus on the bottom-up process, but there are works that explore higher-level factors related to the top-down process (Patterson et al., 2014).

Figure 2: Three-stage pyramidal visual selection: (a) bottom-up feed-forward, (b) top-down winner-take-all feedback (Lippmann, 1987), and (c) bottom-up straight path.

3.3 Cognition, Memory and Vision

After a target is selected, it is potentially useful for “details on demand”, or cognition. In general terms, cognition refers to the acquisition or use of knowledge Cohen (1985), and may occur analytically (conscious and slower) or intuitively (automatic and fast), as defended by the dual system theory (Evans, 2008)(Evans & Stanovich, 2013). Cognition is of special importance in data visualization as it supports analogical reasoning (Patterson et al., 2009). That is, the transfer of inferences from a relationship of elements in one domain (the analogue) to a relationship of elements in another domain (the target). In any case, cognition is intermediated by the memory system. The relationship between memory and cognition is studied by works on Cognitive Architectures, such as ACT-R (Buttner, 2010) and Soar (Young & Lewis, 1999).

In Soar and other theories, the structural configuration of memory roughly reflects the model of Baddeley & Hitch (1974). Working memory includes three components: the central executive module, the phonological loop, and the sketchpad. The central executive module determines the attention focus, guiding the visual system, for example, by top-down biasing the pyramidal selection mechanism. The phonological loop stores information related to sound. The sketchpad (also known as Visual Short-Term Memory – VSTM 111although, this is not a consensus) is associated with the maps of saliences discussed in Section 3.1, storing information related to space and to visual features.

Following these lines, memory comes to be the main element in supporting cognition as it allows complex mental operations. Consequently, it also supports the practice of data visualization. Miller’s Law (Miller, 1956) states that the memory is limited to 7+/-2 elements; more recent, and accepted, works state that this limit is at 4 chunks of elements – a chunk refers to the grouping of elements into larger units based on their meaning (Cowan, 2010). Alvarez & Cavanagh (2004) discuss memory limits considering the nature of the elements to be remembered, demonstrating that, despite the consensus of a (very) limited resource, there is not an ultimate conclusion about the topic. In any case, these limitations are severe because the greater the capacity of an individual’s memory, the more information she/he has available for solving problems (Just & Carpenter, 1992).

The memory system supports cognition in two ways: by retaining a list of elements for quick referencing, and by assisting in the construction of mental models (Johnson-Laird, 1983). According to Johnson-Laird (2010), mental models preserve the relationship between entities by defining analogies that save on logical reasoning, one of the principles behind complex visualization techniques and interaction (Liu & Stasko, 2010a). Following the study of Logie (1995), mental models are created in the visuo-spatial sketchpad, a specialization of VSTM. Stimuli encoded into VSTM, ruled by cognition, may activate enduring information known as long-term memory, including facts, meanings, relationships, skills, and procedures (Patterson et al., 2010) – for a deeper discussion on working memory and long-term memory, refer to the work of Rose et al. (2010).

The concepts presented in the former sections rely on ideas posed by widely accepted theories, among several others, for the visual-cognitive process. The choice for this specific line of thought has been motivated by its intuitive coherence and scope of influence in the literature. Notwithstanding, other theories are widely referenced, such as the Spotlight (Eriksen & Hoffman, 1973) and the Gradient (Cheal et al., 1994) models. For an ample discussion, refer to the work of Squire (2009).

4 The Visual Expression Process

In this section, we review the practice of visualizing data by considering the concepts presented so far. We organize the relationship among visualization, vision, and cognition according to a framework named Visual Expression Process – Figure 3. Our theoretical organization has four constituents: (1) pre-attentive stimuli, (2) analytical perceptions, (3) abstract patterns, and (4) decision support. For completeness of our survey, Table 1 summarizes previous models found in the literature. Our theorization is inspired by these previous works putting together concepts in a complementary point of view.

Model Summary
Lohse (1993)

An algebraic model to estimate the effort to answer questions based on a visualization.

Card et al. (1999) A pipeline of how to conduct the visualization practice.
van Wijk (2005) An economic model stated to evaluate efficacy and efficiency.
(Ware, 2005) A top-down (problem-solving) model that states that what we see in a visualization depends on what we are seeking for.
(Hegarty, 2011) A descriptive discussion of how visualizations work that culminates into a compilation of principles and perspectives.
(Patterson et al., 2014) A vision-cognition model that explains visualization and how to improve it from a top-down perspective.
Table 1: Summary of previous models on InfoVis.

According to our theorization, (1) pre-attentive stimuli come from the neuronal reaction to light, determining a map of potential interesting objects, or saliences. Then, attention concentrates on one single element, part of a limited taxonomic vocabulary of (2) analytical perceptions. Such perceptions go through a pattern-matching process that turns them into (3) abstract patterns. Finally, abstract patterns in working memory support cognition in combination with domain knowledge originated from long-term memory, leading to (4) decision support. Notice, in the figure, that the arrow in between each pair of the four constituents is bidirectional to indicate that the interplay occurs both bottom-up and top-down, notably in response to the task at hand. Also, notice that after cognition, there might be new information demands, what leads to the redefinition of the visualization by means of new parameters for interaction – as depicted in blue in Figure 3. Following, we present further details.

Figure 3: The Visual Expression Process for Information Visualization specified with four constituents. The black bidirectional arrows refer to bottom-up/top-down visual-cognitive processes. In blue, new parameters of interaction alter the scene iteratively.

4.1 Pre-attentive Stimuli - channels for data encoding

Pre-attentive stimuli impel maps of saliences, as highlighted on the leftmost side of Figure 3. In the work of Rodrigues et al. (2008), the authors verified that such stimuli manifest through position, shape, color, and time. Here, we refer to these factors as channels for data encoding, in the sense that they encode data into visual stimuli.

Although the consideration of four channels is a reductionist classification, it is supported by the literature. About color and texture, Watt (1995) affirms that, just like texture, color is the psychological response to the spectral characteristics of a surface; and that, different surfaces are perceived as having different colors. Furthermore, Motter (1994) observes that, early in visual processing, the incoming information is sorted and grouped according to the similarity of simple shape features, such as orientation or size, and of surface features, such as color, luminance, or texture.

The features of each channel span to a large set, but Card et al. (1999) observe that just a limited number of the many existing graphical properties are used for Information Visualization. Healey & Enns (2012) present an extensive survey on pre-attentive features and visualization. A non-exhaustive list of such features is presented in Table 2.

Pre-attentive class (data encoding channel) Features
Position 1D/2D/3D position, stereoscopic depth;
Shape line, area, volume, form, orientation, length, width, collinearity, size, curvature, marks, numerosity, convex/concave;
Color hue, saturation, brightness and texture;
Time movement, morphing, blinking, color/light intermittence.
Table 2: Classes/channels of pre-attentive features from the perspective of data representation.

Discussion

A well-designed visual representation must present a high overlap between its map of saliences and its (implicit) map of semantic relevance. That is, the design of visual representations is supposed to maximize pre-attentive effects. However, such maximization may not be possible without flexible human intervention over the channels of data encoding. In other words, interaction, the active redefinition of the channels, is a mandatory need not fully explored in many designs.

Liu et al. (2008) point out that many visualization systems are not sufficiently flexible to support user customization and appropriation. In fact, it is not difficult to find visualization tools that are limited in allowing the user to determine how to encode data. In these circumstances, a user may ask “may I change the positioning order of the elements?”, “can I have each year represented with a different shape?”, “can I color the left group in red?”, or “can I see that animated?”. Each of these examples refers to a particular pre-attentive feature or, as we propose, to a data encoding channel.

For instance, consider the seminal system GGobi (Cook & Swayne, 2007), which introduced a large set of features if compared to its former version, system XGobi. Still, a brief analysis reveals that there is much to be improved: positioning of views is limited, except for Parallel Coordinates; shape is not an option for coding in the same way that color is; and animation is restricted to scatter plots through touring techniques. Those design issues contrast to what is observed in commercial systems like TIBCO’s Spotfire (http://spotfire.tibco.com/) and Google’s Gapminder (http://www.gapminder.org/), which present higher levels of freedom for each encoding channel, but that, still, do not achieve full appropriation of the scene the way we discuss.

Accordingly, the design of visualizations must not only allow users to alter visual attributes, they must incorporate mechanisms to emphasize each channel. This is necessary because a given stimulus (position, shape, color, or time) may fail to capture attention when it is surrounded by other stimuli – this is called inattentional blindness. In such circumstance, this given stimulus fails in competing with other more interesting stimuli or, worse, it is ignored (Mack A, 1998)(DJ, 2000). “Blindness” occurs especially when the viewer has some previous expectation on the scene; in such situations, top-down processes can strongly influence on what is noticed. This issue has been noticed by Patterson et al. (2014), who claim that the design of a visualization technique must provide means to capture attention alerting users of changes.

Furthermore, according to the limitations of the memory system, as presented in Section 3.3, memory overloading is a problem that might constrain problem-solving. In fact, according to Wickens (2008), multiple tasks in one same cognitive dimension might decrease performance. One possibility to lessen this drawback is to selectively turn the visual channels on and off. For example, it is possible to conceive a dispersion plot in which size is simply turned off, so to temporarily avoid overlapping – see Figure 4 (b). Similarly, it is possible to turn off color, presenting all the information with black marks, emphasizing the role shape contours – Figure 4(c). It is also possible to include intermittence, or movement, much the way that cyclists do with lights in urban traffic. This way, specific graphical elements can start to blink – Figure 4(e). We conjecture that this kind of selective constraining of channels not only alters attention mechanisms, as there will be fewer candidates for attention, it also reduces the memory load, as there would be fewer targets. Furthermore, it has the potential of affecting top-down processes by de-emphasizing features that are expected by the user – and that prevent her/him from noticing unexpected traits due to “blindness”. In these cases, algorithms to assist visualizations may be helpful, as advocated by Chen (2006). Such algorithms might monitor statistical features or mine characteristics of interest to suggest new encodings “on the fly”, enabling more intense appropriation of the scene. However, not much research has been conducted on this issue, which has potential for further developments.

Figure 4:

(a) Dispersion plot using channels of position, shape contour, shape size, and color. (b) Shape size turned off. (c) Color turned off. (d) Shape contour turned off. (e) Time movement turned on for automatically detected outliers. (f) Shape size turned on.

4.2 Analytical Perceptions

Data encoding channels provide maps of potential targets for attention. Now, following vision theory, the next mechanism is attentive selection – see Figure 3. Biased by user intention, a subset of the prominent entities in a visualization will reach working memory. Once selected, the chosen visual stimuli will be the basis of the analogies that lead to mental models, see Section 3.3. Here, one question comes up – what perceptions are produced by the targets of attention in an information visualization design?

To answer this question, we have extensively inspected the literature tracking the ways in which visual manifestation occurs when the intent is data analysis. We have found a limited set of possibilities, defining a visual taxonomic vocabulary whose elements appear recurrently. We refer to these elements analytical perceptions, depicted in the second part of Figure 3. In the realm of interaction, Yi et al. (2007) followed a similar procedure based on extensive literature inspection; they achieved results concerning the user intent, producing a limited set of recurrent possibilities.

Analytical perceptions are the traits that any user attentively seeks for in a visual representation. Following the dual system theory presented in Section 3.3, such perceptions occur intuitively. Our investigation indicates that such elements include correspondence, differentiation, recognition, connectivity, arrangement, and variation in time. The most verified of these phenomena, correspondence and differentiation, are noted by Bertin (1977/1981) and by Card et al. (1999). The third analytical perception is presented by Mackinlay (1986) who states that the notion of relationship among graphical entities comes from the perception of connectivity. Meanwhile, arrangement arises from group positional configurations, largely studied by the Gestalt psychology (Koffka, 1935) as observed, for example, in graph layouts (Dwyer et al., 2009)(Tollis et al., 1998). Recognition, in turn, takes place as a resemblance to previous knowledge and/or expertise, a concept studied in psychological models and Information Visualization models. As Liu & Stasko (2010b) point out, the concept of internalization involves the encoding of perceived information into long-term memory (enduring information or pattern). Lastly, variation, manifests only along time – not necessarily for temporal data – and in combination to the other five perceptions.

The notion of analytical perceptions becomes evident when they are not found and the pipeline outlined in Figure 3 is broken, preventing Visual Expression – if none of the aforementioned perceptions occur, the user is unable to make sense. As depicted in our theorization, analytical perceptions occur after pre-attention (Section 4.1) and before abstract patterns (Section 4.3), independently of the data domain. Hence, they bridge vision and data interpretation. Specifically, we discuss the analytical perceptions and how they relate to the data encoding channels in the following.

  • correspondence: each position/shape/color has a direct correspondence to a referential map – discrete or continuous – that is part of the scene (explicit) or that is mental (implicit), defining analogical reasoning as explained in Section 3.3. Explicit maps include axes, geographical maps, shape/color dictionaries, and position/shape/color ranges. Implicit maps include known orderings and shape metaphors;

  • differentiation: each position/shape/color discriminates graphical items. Differentiation is a correspondence achieved by the user, who creates a referential map in memory. Such map is limited in the number of elements (or differentiations) according to the limitations of memory;

  • recognition: positions/shapes/colors whose decoding comes from the expertise of the user or from previous knowledge – recognition is a correspondence established from visual entities to concepts retained/learned in long-term memory;

  • connectivity: shapes, mainly edges, that convey information about relationships among entities in memory;

  • arrangement: Gestalt principles of organization – positional placements (closure, proximity, and symmetry) that convey perception about group properties, for example, clusters and structural cues;

  • variation in time: obtained when the parameters of position/shape/color are altered along time, inducing new perceptions for each of these channels.

Discussion

Our set of analytical perceptions is not an exhaustive listing, but a first reference that suggests that visualization designs tend to resort to the same basic set. For instance, the design of Google’s Gapminder tool, although contemporaneous, reproduces the same dispersion plots of statistical books a hundred years old. Nevertheless, as observed by Liu et al. (2014), works on new designs and on the evaluation of existing ones have not considered that there is a limited set of elements to instantiate in data representations. This fact could fruitfully support the definition of design languages and frameworks, which would benefit from recurrent constructs that lead to a limited set of analytical perceptions. Differently, current languages and frameworks rely on graphical patterns and it is up to the user to build the desired analytical perception; see Table 3 for a representative set.

Work Approach Elements
Protovis (Bostock & Heer, 2009)(Heer & Bostock, 2010) Graphical Marks, shapes and layout
(Bostock et al., 2011) Visualization pipeline Selection, operation, join, layout and transformation
Improvise (Weaver, 2004) Link and coordinate Variable, function and view
Prefuse (Heer et al., 2005) High-level API Filter, layout, interaction, color and size
ggplot2 (Wickham, 2009) Domain specific Layer, scale, coordinate system and facet
Flexible Linked Axes (Claessen & van Wijk, 2011) Linked axes Axes mapping, interaction, line and point
This work Visual-cognitive Pre-attentive encoding channels and analytical perceptions
Table 3: Previous works on languages and frameworks for visualization design.

By considering the concepts reviewed so far, it is possible to conceive a design language whose approach is based on cognition, and whose elements are interactive encoding channels, and analytical perceptions – refer to Table 3 for comparison. For example, in this design language, one would be able to state a visualization by choosing channel color and perception of differentiation. This same visualization would demand a few more elements, as channel position and perception of correspondence. As in any language, these elements would receive parameters according to an extensible library of data-to-marks mapping. In the case of relationships, one would be able to choose channel position with perception of arrangement, together with channel shape and perception of connectivity

. Underlying algorithms would abstract undesired complexity, as an algorithm for outlier detection to support color differentiation; and a force-directed algorithm to support arrangement. As a design language, this approach would bring the benefit of discriminating the recurring elements of visualization techniques and having them in libraries for composition. This alternative approach contrasts with the usual practice of combining recurrent elements in ensembles assumed as new techniques.

4.3 Abstract Patterns

According to Hutchins (1996), tools – or externalizations (Hegarty, 2004) – transform difficult tasks into in-mind manipulations of physical systems, or into pattern-matching problems (Giere, 2002)

. Pattern matching, or pattern-recognition

(Patterson et al., 2014), the association of a given stimulus to information retrieved from memory (Eysenck & Keane, 2003), is one of the principles of data visualization and the second step of the Visual Expression Process – see Figure 3.

According to our organization, once a user focuses on an analytical perception, she/he proceeds to match this perception to an abstract pattern. Based on the theory of vision – seen in Section 3.3, the generation of patterns is supported by memory, which is filled with data from the visual-sensorial system or from long-term memory. The efficiency of this intercourse is explained by the fact that the visual-sensorial system provides spatial information to memory at rates higher than that of long-term memory (Ware, 2004), in a time ranging from to (Kieras & Meyer, 1997). Hence, analytical perceptions in the visual-sensorial system work similarly to the images stored in long-term memory, providing efficient pattern-matching. According to the dual system theory, Section 3.3, this process is conscious and slow; but, with practice and experience, a given analyst can become a specialist. For specialists, pattern matching becomes intuitive (Evans, 2008) – refer to the work of Patterson et al. (2010) for a thorough discussion. It is a straight conclusion, then, that a period of practice and experience is necessary for most visualization techniques, and that this is an obstacle for use.

A suggestive set of the patterns – third part of Figure 3 – that arise from the perception-to-pattern matching, includes: correlation, tendency, classification, relationship, order, summarization, outlier, cluster, structure, and reading. Tufte (1997) and Amar et al. (2005) provide more exhaustive listings that follow different rationalizations.

Discussion

In our literature review, we noticed that many systems disregard the fact that visualization techniques converge through abstract patterns. According to current practices, instead of a set of familiar patterns, the user has to choose among a set of visualization designs that, quite often, they have little experience with (Chen, 2005). Thomas & Cook (2005) reinforce this notion; they state that abstract patterns correspond to the second factor of their four-steps analytical-reasoning process. Following their process – a pattern-to-construct sequence, users have constraints in relation to what they can search for in face of a given pattern. For example, suspiciousness tends to appear by means of tracking for outliers; while evidence of illegal lobbying practices emerge from clusters; and community detection in graphs is a task for relationship. Still, users are often offered a menu whose options are, for instance, dimensional stacking (LeBlanc et al., 1990), star coordinates (Kandogan, 2000), and table lens (Rao & Card, 1994); alternatives far from the pattern-to-construct task they have in their minds. A similar notion dates back to the 1990’s, in the work of Wehrend & Lewis (1990), who advocates in favor of more specific problem-oriented choices. We suspect that, because design-oriented interfaces neglect the more natural notion of abstract patterns, this is possibly one of the reasons why advanced visualization techniques have struggled to achieve a wider commercial dissemination, e.g, in office suites and in everyday spreadsheets.

Take, for instance, the visualization technique Treemap (Shneiderman, 1992), introduced for visualizing hierarchical data in general. In two decades, Treemap gained popularity at the academy, but, as a general hierarchical tool, it has failed in reaching a wider use. This is, possibly, because it has been criticized since its introduction (Barlow & Neville, 2001; Cawthon & Moere, 2007; Fabrikant & Skupin, 2005), being accused of lacking cognitive plausibility, having poorly perceived aesthetic qualities, and presenting poor task-driven performance (Wood & Dykes, 2008). Despite all, however, an especial design of the Treemap has remarkably succeeded. The SequoiaView system (www.win.tue.nl/sequoiaview) has achieved wide dissemination (check (van Wijk, 2006) for some impressive numbers), far beyond the academic walls. But, how can we have two flavors of the same technique evolve in different ways? Certainly, not one single aspect explains everything, but an outstanding factor comes up: the SequoiaView is domain and pattern-oriented, it is distributed to visualize the structure and the sizes of the files in your hard-drive, specifically. In accordance, users do not have to discover that the tool is good at doing this; instead, when they have this specific problem at hand, they are guided to SequoiaView, a more natural process.

In designing systems, an alternative course of action would be to initially present the user with a set of patterns to choose from. After that, she/he would be offered a set of visualization techniques that better suit the pattern they are seeking for. Users may know what to look for by means of previous knowledge of the data domain, by means of known problems to be solved, by means of suspicious clues perceived along the data usage, and also by means of previous visual exploration of the data.

4.4 Cognitive Decision Support

After producing patterns, vision is no longer an active agent, neither pre-attentively nor attentively. Now, the analysis follows the widely accepted pattern-then-cognition process (Margolis, 1990) (MacEachren, 2004) to achieve decision support. Indeed, according to the analytical reasoning of Thomas & Cook (2005), Information Visualization cannot ultimately provide decision support, which can only be achieved after interpreting the patterns in light of the data domain – rightmost side of Figure 3. That is, even though users can come up with patterns without considering the underlying data, these patterns are not of great use if the domain is not deeply understood.

Insufficient domain knowledge leads to unsatisfied expectations in relation to InfoVis, creating situations in which a user is presented to a supposedly insightful visualization but, then, everything one hears is “so what”? A disappointment that happens due to the enthusiasm according to which one can solve a wide range of problems just by looking at the data. However, visualization tools can do little if the analyst is not well-prepared to assess what the data ultimately describes and potentially carries within.

Discussion

To prevent the aforementioned problems, InfoVis systems should define systematic means to aid the user in recording and accessing the domain knowledge related to the problems at hand. An interesting approach to overcome the gap of domain knowledge is to use annotations (or analytic provenance (Xu et al., 2015)), either automatic or manual. As pointed out by Hullman et al. (2011), annotations help to direct the user’s attention and foreground particular insights, supporting the most efficient inferences. The work of Hullman et al. (2013) exemplifies this issue. Their work focuses on stock-price time series, which are hard to understand if one is not aware of the facts that influenced the behavior of the market. Their system solves this problem by identifying news that happened contemporaneously to outstanding patterns found in the plots, presenting them on demand – illustrated in Figure 5. Similar approaches (Dennis et al., 2003; Conesa et al., 2005) have been proposed for genomic data, an extreme case in which domain knowledge is necessary, otherwise no reading of the data (either visual or textual) will make sense.

Figure 5: Example of an annotated visualization produced with Contextifier – in the figure, labels provide domain knowledge. Reproduced from the work of Hullman et al. (2011).

An alternative is to use technique storytelling, as surveyed by Segel & Heer (2010). Plaisant (2005) defends that advanced interfaces need to address the long-term process of analysis that may require annotation, history keeping, collaboration with peers, and the dissemination of results and procedures used. Storytelling not only attacks the problem of lacking domain knowledge, it also provides knowledge about new findings in the form of further “story chapters” interactively created. The visual analysis, potentially, becomes an incremental set of insights from multiple experts in the form of bookmarks, keyword tags, text comments, and audio annotations.

By bringing existing (factual) or produced (annotated) domain knowledge to the stage, abstract patterns can support reasoning by supporting conditional inferences (Johnson-Laird & Byrne, 2002), possibly influenced by statistical regularities (Patterson et al., 2012). The result is the provision of inferences for decision making, notably, predictions, alternatives, and evaluations as depicted in Figure 3.

4.5 Remarks

In Sections 4.1 through 4.4, we have drawn conclusions by surveying accepted concepts of vision and cognition. Notwithstanding, we state these conclusions as conjectures with theoretical evidence and with examples. This is because the validation of these hypotheses would encompass vast experimentation; producing material enough to spam a few papers or a book, to be conservative. Therefore, we leave our discussions both as contributions – to guide new systematizations; and, as future work – to drive further refinements and discoveries.

Our conclusions also point to a challenging systematization. Rendering all the recommended aspects, together with multiple techniques and data domains, might involve a development effort similar to that of huge software pieces, as office suites for instance. This is academically non-attractive, and economically risky. Possibly, the solution is to set a well-defined development framework for collaborative work, with ample acceptation, and standard interfacing. In the realm of machine learning, software Weka

(Hall et al., 2009) has achieved great success in a similar endeavor. However, the graphical nature of InfoVis, and its data preprocessing techniques, imposes big challenges.

5 Conclusions

We reviewed concepts on vision, cognition, and Information Visualization by introducing the organizational theorization named Visual Expression Process, which proposes a course of action to explain visual data analysis. Over an extensive literature survey, the theorization provides comprehension and science for data graphical presentation. It proposes a new perspective to discuss and characterize techniques. From this perspective, for instance, it would be possible to “dissect” a given technique in terms of the channels, the analytical perceptions and the patterns that it supports, potentially revealing strengths and weaknesses. Our contributions are as follows:

  • Theoretical compendium: we plotted the Visual Expression Process to interrelate vision, cognition, and Information Visualization;

  • Discussions: we provided an extensive survey from different fields of science to serve as basis for relevant debate;

  • Reflections: we revisited design practices supported by examples and study cases.

Overall, we have put together key concepts to introduce an insightful consideration of visualizations. The reductionist perspective of our organization translates non-familiar concepts of vision and cognition into their corresponding effects with respect to design. In the form of a vocabulary taxonomically organized, we proposed a simplified comprehension of the factors that define techniques and systems. With our contributions, we expect to foster a more comprehensive, accessible, and applied science of visualization.

References

  • (1)
  • Alvarez & Cavanagh (2004) Alvarez, G. A. & Cavanagh, P. (2004), ‘The capacity of visual short-term memory is set both by visual information load and by number of objects’, Psychological science 15(2), 106–111.
  • Amar et al. (2005) Amar, R., Eagan, J. & Stasko, J. (2005), Low-level components of analytic activity in information visualization, in ‘Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization’, INFOVIS ’05, IEEE Computer Society, Washington, DC, USA, pp. 15–.
    http://dx.doi.org/10.1109/INFOVIS.2005.24
  • Baddeley & Hitch (1974) Baddeley, A. & Hitch, G. (1974), ‘Working memory’, The psychology of learning and motivation: advances in research and theory 8, 47–89.
  • Barlow & Neville (2001) Barlow, T. & Neville, P. (2001), A comparison of 2-d visualizations of hierarchies, in ‘Information Visualization, 2001. INFOVIS 2001. IEEE Symposium on’, pp. 131–138.
  • Bertin (1977/1981) Bertin, J. (1977/1981), Graphics and Graphic Information-Processing, Walter de Gruyter, Berlin. 273 pages.
  • Bostock & Heer (2009) Bostock, M. & Heer, J. (2009), ‘Protovis: A graphical toolkit for visualization’, Visualization and Computer Graphics, IEEE Transactions on 15(6), 1121 –1128.
  • Bostock et al. (2011) Bostock, M., Ogievetsky, V. & Heer, J. (2011), ‘D3 data-driven documents’, IEEE Transactions on Visualization and Computer Graphics 17(12), 2301–2309.
    http://dx.doi.org/10.1109/TVCG.2011.185
  • Bugajska (2003) Bugajska, M. (2003), Spatial Visualization of Abstract Information, PhD thesis, ETH Zurich. 197 pp.
  • Buttner (2010) Buttner, P. (2010), Hello java: Linking act-r 6 with a java simulation, in ‘Proceedings of the 10th International Conference on Cognitive Modeling’, pp. 289–290.
  • Card et al. (1999) Card, S., Mackinlay, J. & Shneiderman, B. (1999), Readings in Information Visualization: Using Vision to Think, San Francisco, CA, EUA: Morgan Kaufmann.
  • Cawthon & Moere (2007) Cawthon, N. & Moere, A. (2007), The effect of aesthetic on the usability of data visualization, in ‘Information Visualization, 2007. IV ’07. 11th International Conference’, pp. 637–648.
  • Cheal et al. (1994) Cheal, M., Lyon, D. & Gottlob, L. (1994), ‘A framework for understanding the allocation of attention in location-precued discrimination’, Quarterly Journal of Experimental Psychology: Human Experimental Psychology 47, 699––739.
  • Chen (2005) Chen, C. (2005), ‘Top 10 unsolved information visualization problems’, IEEE Computer Graphics and Applications 25(4), 12–16.
  • Chen (2006) Chen, C. (2006), Information Visualization: Beyond the Horizon, Springer Science.
  • Chi (2000) Chi, E. H. (2000), A taxonomy of visualization techniques using the data state reference model, in ‘IEEE Symposium on Information Visualization’, IEEE Computer Society Press, Salt Lake City, UT, USA, pp. 69–75.
  • Claessen & van Wijk (2011) Claessen, J. & van Wijk, J. (2011), ‘Flexible linked axes for multivariate data visualization’, Visualization and Computer Graphics, IEEE Transactions on 17(12), 2310–2316.
  • Cleveland & McGill (1984) Cleveland, W. S. & McGill, R. (1984), ‘Graphical perception: Theory, experimentation and application to the development of graphical methods’, J. Am. Stat. Assoc. 79, 387 5, 531–554.
  • Cohen (1985) Cohen, R. (1985), The Development of Spatial Cognition, Psychology Press.
  • Conesa et al. (2005) Conesa, A., Gotz, S., Garcia-Gomez, J. M., Terol, J., Talon, M. & Robles, M. (2005), ‘Blast2go: a universal tool for annotation, visualization and analysis in functional genomics research’, Bioinformatics 21(18), 3674–6.
  • Cook & Swayne (2007) Cook, D. & Swayne, D. F. (2007), Interactive and Dynamic Graphics for Data Analysis, Springer-Verlag.
  • Cowan (2010) Cowan, N. (2010), ‘The magical mystery four: How is working memory capacity limited, and why?’, Current directions in psychological science 19(1), 51–57.
  • Cutzu & Tsotsos (2003) Cutzu, F. & Tsotsos, J. (2003), ‘The selective tuning model of attention: psychophysical evidence for a suppressive annulus around an attended item’, Vision Research 43, 205–219.
  • Dennis et al. (2003) Dennis, G., Sherman, B., Hosack, D., Yang, J., Gao, W., Lane, H. C. & Lempicki, R. (2003), ‘DAVID: Database for Annotation, Visualization, and Integrated Discovery’, Genome Biology 4(5), P3+.
  • Dill et al. (2012) Dill, J., Earnshaw, R., Kasik, D., Vince, J. & Wong, P. (2012), Expanding the Frontiers of Visual Analytics and Visualization, Springer-Verlag.
  • DJ (2000) DJ, S. (2000), ‘Attentional capture and inattentional blindness’, Trends Cognitive Science 4(4), 147–155.
  • Dwyer et al. (2009) Dwyer, T., Lee, B., Fisher, D., Quinn, K., Isenberg, P., Robertson, G. & North, C. (2009), ‘A comparison of user-generated and automatic graph layouts’, Visualization and Computer Graphics, IEEE Transactions on 15(6), 961 –968.
  • Eriksen & Hoffman (1973) Eriksen, C. W. & Hoffman, J. E. (1973), ‘The extent of processing of noise elements during selective encoding from visual displays’, Perception and Psychophysics 14, 155––160.
  • Essen et al. (1992) Essen, D. V., Anderson, C. & Fellowman, D. (1992), ‘Information processing in the primate visual system: An integrated systems perspective’, Science 255(5043), 419–422.
  • Evans (2008) Evans, J. S. B. T. (2008), ‘Dual-processing accounts of reasoning, judgment, and social cognition’, Annual Review of Psychology 59(1), 255–278.
  • Evans & Stanovich (2013) Evans, J. S. B. T. & Stanovich, K. E. (2013), ‘Dual-process theories of higher cognition: Advancing the debate’, Perspectives on Psychological Science 8(3), 223–241.
    http://pps.sagepub.com/content/8/3/223.abstract
  • Eysenck & Keane (2003) Eysenck, M. W. & Keane, M. T. (2003), Cognitive Psychology: A Student’s Handbook, Taylor & Francis.
  • Fabrikant & Skupin (2005) Fabrikant, S. & Skupin, A. (2005), Cognitively plausible information visualization, in ‘Amsterdam’, Elsevier Science.
  • Giere (2002) Giere, R. N. (2002), Models as parts of distributed cognitive systems, in ‘Model-Based Reasoning’, Springer US, pp. 227–241.
  • Green et al. (2009) Green, T. M., Ribarsky, W. & Fisher, B. (2009), ‘Building and applying a human cognition model for visual analytics’, Information Visualization 8(1), 1–13.
  • Hall et al. (2009) Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. & Witten, I. H. (2009), ‘The weka data mining software: An update’, SIGKDD Explor. Newsl. 11(1), 10–18.
    http://doi.acm.org/10.1145/1656274.1656278
  • Healey & Enns (2012) Healey, C. G. & Enns, J. T. (2012), ‘Attention and visual memory in visualization and computer graphics’, Visualization and Computer Graphics, IEEE Transactions on 18(7), 1170–1188.
  • Heer & Bostock (2010) Heer, J. & Bostock, M. (2010), ‘Declarative language design for interactive visualization’, IEEE Transactions on Visualization and Computer Graphics 16, 1149–1156.
  • Heer et al. (2005) Heer, J., Card, S. K. & Landay, J. A. (2005), Prefuse: A toolkit for interactive information visualization, in ‘Proceedings of the SIGCHI Conference on Human Factors in Computing Systems’, CHI ’05, ACM, New York, NY, USA, pp. 421–430.
    http://doi.acm.org/10.1145/1054972.1055031
  • Hegarty (2004) Hegarty, M. (2004), Diagrams in the mind and in the world: Relations between internal and external visualizations, in ‘Diagrams’, pp. 1–13.
  • Hegarty (2011) Hegarty, M. (2011), ‘The cognitive science of visual-spatial displays: Implications for design’, Topics in Cognitive Science 3(3), 446–474.
  • Hullman et al. (2011) Hullman, J., Adar, E. & Shah, P. (2011), ‘Benefitting infovis with visual difficulties’, IEEE Transactions on Visualization and Computer Graphics 17(12), 2213–2222.
    http://dx.doi.org/10.1109/TVCG.2011.175
  • Hullman et al. (2013) Hullman, J., Diakopoulos, N. & Adar, E. (2013), Contextifier: Automatic generation of annotated stock visualizations, in ‘Proceedings of the SIGCHI Conference on Human Factors in Computing Systems’, CHI ’13, ACM, New York, NY, USA, pp. 2707–2716.
    http://doi.acm.org/10.1145/2470654.2481374
  • Hutchins (1996) Hutchins, E. (1996), Cognition in the Wild, The MIT Press.
  • Itti & Koch (2001) Itti, L. & Koch, C. (2001), ‘Computational modeling of visual attention’, Nature Reviews Neuroscience 2(3), 194–203.
  • Johnson et al. (2006) Johnson, C., Moorhead, R., Munzner, T., Pfister, H., Rheingans, P. & Yoo, T. (2006), NIH/NSF Visualization Research Challenges, Los Alamitos, CA, EUA: IEEE Press. 36 pages.
  • Johnson-Laird (1983) Johnson-Laird, P. N. (1983), Mental models: towards a cognitive science of language, inference, and consciousness, Harvard University Press.
  • Johnson-Laird (2010) Johnson-Laird, P. N. (2010), ‘Mental models and human reasoning’, Proceedings of the National Academy of Sciences 107(43), 18243–18250.
  • Johnson-Laird & Byrne (2002) Johnson-Laird, P. N. & Byrne, R. M. (2002), ‘Conditionals: a theory of meaning, pragmatics, and inference.’, Psychological review 109(4), 646.
  • Just & Carpenter (1992) Just, M. & Carpenter, P. (1992), ‘A capacity theory of comprehension: individual differences in working memory’, Psychological Review 99, 122–149.
  • Kandogan (2000) Kandogan, E. (2000), Star coordinates: A multi-dimensional visualization technique with uniform treatment of dimensions, in ‘IEEE Symposium on Information Visualization’, IEEE Press, Salt Lake City, pp. 4–8.
  • Keim (2002) Keim, D. (2002), ‘Information visualization and visual data mining’, IEEE Trans. Visualization and Computer Graphics 8(1), 1–8.
  • Kieras & Meyer (1997) Kieras, D. & Meyer, D. (1997), ‘An overview of the epic architecture for cognition and performance with application to human-computer interaction’, HCI 12, 391–438.
  • Koffka (1935) Koffka, K. (1935), Principles of Gestalt psychology, International library of psychology, philosophy and scientific method, New York : Harcourt, Brace and Company.
  • LeBlanc et al. (1990) LeBlanc, J., Ward, M. & Wittels, N. (1990), Exploring n-dimensional databases, in ‘Visualization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’, pp. 230 –237.
  • Lippmann (1987) Lippmann, R. P. (1987), ‘An introduction to computing with neural nets’, IEEE ASSP Magazine 4(2), 4–22.
  • Liu et al. (2014) Liu, S., Cui, W., Wu, Y. & Liu, M. (2014), ‘A survey on information visualization: recent advances and challenges’, The Visual Computer 30(12), 1373–1393.
  • Liu et al. (2008) Liu, Z., Nersessian, N. & Stasko, J. (2008), ‘Distributed cognition as a theoretical framework for information visualization’, Visualization and Computer Graphics, IEEE Transactions on 14(6), 1173 –1180.
  • Liu & Stasko (2010a) Liu, Z. & Stasko, J. (2010a), ‘Mental models, visual reasoning and interaction in information visualization: A top-down perspective’, IEEE Transactions on Visualization and Computer Graphics 16(6), 999–1008.
    http://dx.doi.org/10.1109/TVCG.2010.177
  • Liu & Stasko (2010b) Liu, Z. & Stasko, J. (2010b), ‘Mental models, visual reasoning and interaction in information visualization: A top-down perspective’, Visualization and Computer Graphics, IEEE Transactions on 16(6), 999 –1008.
  • Logie (1995) Logie, R. H. (1995), Visual-spatial working memory, Lawrence Erlbaum Associates.
  • Lohse (1993) Lohse, G. L. (1993), ‘A cognitive model for understanding graphical perception’, Hum.-Comput. Interact. 8(4), 353–388.
  • MacEachren (2004) MacEachren, A. M. (2004), How Maps Work: Representation, Visualization, and Design, The Guilford Press.
  • Mack A (1998) Mack A, R. I. (1998), ‘Inattentional blindness’, Psyche 5(3).
  • Mackinlay (1986) Mackinlay, J. (1986), ‘Automating the design of graphical presentations of relational information’, ACM Transactions on Graphics 5(2), 110–141.
  • Margolis (1990) Margolis, H. (1990), Patterns, Thinking, and Cognition: A Theory of Judgment, University Of Chicago Press.
  • Miller (1956) Miller, G. A. (1956), ‘The magical number seven, plus or minus two: Some limits on our capacity for processing information’, Psychological Review 2(63), 343–35.
  • Motter (1994) Motter, B. C. (1994), ‘Neural correlates of attentive selection for color or luminance in extrastriate area v4’, The Journal of Neuroscience 14(4), 2178–2189.
  • Navalpakkam & Itti (2007) Navalpakkam, V. & Itti, L. (2007), ‘Search goal tunes visual features optimally’, Neuron 53(4), 605–617.
  • Nothdurft (2000) Nothdurft, H. (2000), ‘Salience from feature contrast: additivity across dimensions’, Vision Research 40(10-12), 1183–1201.
  • Patterson (2012) Patterson, R. E. (2012), ‘Cognitive engineering, cognitive augmentation, and information display’, Journal of the Society for Information Display 20(4), 208–213.
    http://dx.doi.org/10.1889/JSID20.4.208
  • Patterson et al. (2014) Patterson, R. E., Blaha, L. M., Grinstein, G. G., Liggett, K. K., Kaveney, D. E., Sheldon, K. C., Havig, P. R. & Moore, J. A. (2014), ‘A human cognition framework for information visualization’, Computers & Graphics 42(0), 42 – 58.
    http://www.sciencedirect.com/science/article/pii/S0097849314000387
  • Patterson et al. (2009) Patterson, R. E., Pierce, B., Bell, H. H., Andrews, D. & Winterbottom, M. (2009), ‘Training robust decision making in immersive environments’, Cognitive Engineering and Decision Making 3(4), 331–361.
  • Patterson et al. (2010) Patterson, R. E., Pierce, B. J., Bell, H. H. & Klein, G. (2010), ‘Implicit learning, tacit knowledge, expertise development, and naturalistic decision making’, Journal of Cognitive Engineering and Decision Making 4(4), 289–303.
  • Patterson et al. (2012) Patterson, R. E., Pierce, B. J., Boydstun, A. S., Ramsey, L. M., Shannan, J., Tripp, L. & Bell, H. (2012), ‘Training intuitive decision making in a simulated real-world environment’, Human Factors: The Journal of the Human Factors and Ergonomics Society p. 0018720812454432.
  • Plaisant (2005) Plaisant, C. (2005), Chapter 3 - information visualization and the challenge of universal usability, in ‘Exploring Geovisualization’, International Cartographic Association, Elsevier, Oxford, pp. 53 – 82.
  • Rao & Card (1994) Rao, R. & Card, S. (1994), The table lens: Merging graphical and symbolic representation in an interactive focus+context visualization for tabular information, in ‘Human Factors in Computing Systems’, pp. 318–322.
  • Rensink (2000) Rensink, R. A. (2000), ‘The dynamic representation of scenes’, Visual Cognition 7(1-3), 17–42.
  • Rodrigues et al. (2008) Rodrigues, J., Traina, A., Oliveira, M. & Traina, C. (2008), ‘The spatial/perceptual design space: a new comprehension for data visualization’, Palgrave Information Visualization 6, 261–279.
  • Rose et al. (2010) Rose, N. S., Myerson, J., Roediger III, H. L. & Hale, S. (2010), ‘Similarities and differences between working memory and long-term memory: evidence from the levels-of-processing span task.’, Journal of Experimental Psychology: Learning, Memory, and Cognition 36(2), 471.
  • Scaife & Rogers (1996) Scaife, M. & Rogers, Y. (1996), ‘External cognition: how do graphical representations work?’, Int. J. Hum.-Comput. Stud. 45(2), 185–213.
  • Segel & Heer (2010) Segel, E. & Heer, J. (2010), ‘Narrative visualization: Telling stories with data’, IEEE Transactions on Visualization and Computer Graphics 16(6), 1139–1148.
    http://dx.doi.org/10.1109/TVCG.2010.179
  • Shneiderman (1992) Shneiderman, B. (1992), ‘Tree visualization with treemaps: A 2d space-filling approach’, ACM Trans. on Graphics 11(1), 92–99.
  • Shneiderman (1996) Shneiderman, B. (1996), The eyes have it: A task by data type taxonomy of information visualizations, in ‘IEEE Symp. Visual Languages’, IEEE CS Press, pp. 336–343.
  • Squire (2009) Squire, L. R. (2009), ‘Memory and brain systems: 1969–2009’, The Journal of Neuroscience 29(41), 12711–12716.
  • Thomas & Cook (2005) Thomas, J. & Cook, K., eds (2005), Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Press.
  • Tollis et al. (1998) Tollis, I. G., Di Battista, G., Eades, P. & Tamassia, R. (1998), Graph Drawing: Algorithms for the Visualization of Graphs, Prentice Hall.
  • Triesman (1985) Triesman, A. (1985), ‘Preattentive processing in vision’, Computer Vision, Graphics and Image Processing 31, 156–177.
  • Tsotsos (1991) Tsotsos, J. (1991), ‘Is complexity theory appropriate for analysing biological systems?’, Behavioral and Brain Sciences 14(4), 770–773.
  • Tsotsos et al. (1995) Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N. & Nuflo, F. (1995), ‘Modeling visual attention via selective tuning’, Artificial Intelligence 78(1-2), 507–547.
  • Tufte (1997) Tufte, E. R. (1997), Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press.
  • Tweedie (1997) Tweedie, L. (1997), Characterizing interactive externalizations, in ‘ACM SIGCHI conference on Human factors in computing systems’, ACM Press, Atlanta, Georgia, USA, pp. 375–382.
  • van Wijk (2005) van Wijk, J. (2005), The value of visualization, in ‘Visualization, 2005. VIS 05. IEEE’, pp. 79–86.
  • van Wijk (2006) van Wijk, J. (2006), ‘Views on visualization’, Visualization and Computer Graphics, IEEE Transactions on 12(4), 1000 –433.
  • Ware (2004) Ware, C. (2004), Information Visualization: Perception for design, San Francisco, CA: Morgan Kaufman. 486 pages.
  • Ware (2005) Ware, C. (2005), Knowledge and Information Visualization: Searching for Synergies, Springer Berlin Heidelberg, Berlin, Heidelberg, chapter Visual Queries: The Foundation of Visual Thinking, pp. 27–35.
  • Watt (1995) Watt, R. J. (1995), Some speculations on the role of texture processing in visual perception, in ‘Early vision and beyond’, Massachusetts Institute of Technology, Cambridge, MA, USA, pp. 59–67.
  • Weaver (2004) Weaver, C. (2004), Building highly-coordinated visualizations in improvise, in ‘Information Visualization, 2004. INFOVIS 2004. IEEE Symposium on’, pp. 159–166.
  • Wehrend & Lewis (1990) Wehrend, S. & Lewis, C. (1990), A problem-oriented classification of visualization techniques, in ‘Visualization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’, pp. 139–143, 469.
  • Wickens (2008) Wickens, C. D. (2008), ‘Multiple resources and mental workload’, Human Factors: The Journal of the Human Factors and Ergonomics Society 50(3), 449–455.
    http://hfs.sagepub.com/content/50/3/449.abstract
  • Wickham (2009) Wickham, H. (2009), ggplot2: Elegant Graphics for Data Analysis, Springer, New York.
  • Wolfe (1994) Wolfe, J. M. (1994), ‘Guided search 2.0 a revised model of visual search’, Psychonomic Bulletin & Review 1(2), 202–238.
  • Wong et al. (2012) Wong, P. C., Shen, H.-W., Johnson, C., Chen, C. & Ross, R. B. (2012), ‘The top 10 challenges in extreme-scale visual analytics’, Computer Graphics and Applications, IEEE 32(4), 63–67.
  • Wood & Dykes (2008) Wood, J. & Dykes, J. (2008), ‘Spatially ordered treemaps’, IEEE Transactions on Visualization and Computer Graphics 14(6), 1348–1355.
  • Woodman et al. (2013) Woodman, G. F., Carlisle, N. B. & Reinhart, R. M. (2013), ‘Where do we store the memory representations that guide attention?’, Journal of vision 13(3), 1.
  • Xu et al. (2015) Xu, K., Attfield, S., Jankun-Kelly, T., Wheat, A., Nguyen, P. & Selvaraj, N. (2015), ‘Analytic provenance for sensemaking: A research agenda’, Computer Graphics and Applications, IEEE 35(3), 56–64.
  • Yi et al. (2007) Yi, J. S., ah Kang, Y., Stasko, J. T. & Jacko, J. A. (2007), ‘Toward a deeper understanding of the role of interaction in information visualization’, Visualization and Computer Graphics, IEEE Transactions on 13(6), 1224–1231.
  • Young & Lewis (1999) Young, R. M. & Lewis, R. L. (1999), The soar cognitive architecture and human working memory, in A. Miyake & P. Shah, eds, ‘Models of Working Memory: Mechanisms of Active Maintenance and Executive Control’, Cambridge University Press, pp. 224–256.