Sea of Genes: Combining Animation and Narrative Strategies to Visualize Metagenomic Data for Museums

06/03/2019 ∙ by Keshav Dasu, et al. ∙ University of California-Davis Exploratorium 0

We examine the application of narrative strategies to present a complex and unfamiliar metagenomics dataset to the public in a science museum. Our dataset contains information about microbial gene expressions that scientists use to infer the behavior of microbes. This exhibit had three goals: to inform (the) public about microbes' behavior, cycles, and patterns; to link their behavior to the concept of gene expression; and to highlight scientists' use of gene expression data to understand the role of microbes. To address these three goals, we created a visualization with three narrative layers, each layer corresponding to a goal. This study presented us with an opportunity to assess existing frameworks for narrative visualization in a naturalistic setting. We present three successive rounds of design and evaluation of our attempts to engage visitors with complex data through narrative visualization. We highlight our design choices and their underlying rationale based on extant theories. We conclude that a central animation based on a curated dataset could successfully achieve our first goal, i.e., to communicate the aggregate behavior and interactions of microbes. We failed to achieve our second goal and had limited success with the third goal. Overall, this study highlights the challenges of telling multi-layered stories and the need for new frameworks for communicating layered stories in public settings.



There are no comments yet.


page 1

page 2

page 5

page 6

page 7

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Related Work

Figure 2: Initial brainstorm and design process. (a) Photo capturing a discussion from the initial brainstorm session with a scientist from SCOPE. (b) Sketches of microbe behavior inferred from genomic data at the brainstorm. (c) At the end of the brainstorm we produced a sketch of Prototype 1.

Extensive work has been done on the application of narrative devices to the design of museum exhibits and information visualizations, use of narratives in visualization of complex data, and frameworks for applying narrative techniques to information visualization design. There has also been research on the use of animation for teaching unfamiliar concepts and effective design principles.

1.1 Value of Narratives

Kosara and Mackinlay [38] define a narrative as ”a clearly defined path that is composed of a sequence of ordered steps, containing either text, images, visualizations, video, or any combination of the latter”. Each step can be thought of as a story that relates the ‘who, where, when and how’ of an event that occurred. The ordering of these steps creates the narrative. How the order is set impacts how it is received. Storytelling is the activity of sharing a narrative. Theorists posit that narratives are fundamental to human sensemaking [15] and intelligence [53]. Research has found that stories help us connect and make it easy to remember facts. The roles and forms that narratives may serve and can take in information visualization continue to be active areas of research. For example, Gershon and Page [23] discuss the value of storytelling when developing applications in an information visualization environment . These environments process streams of information and data sources possibly in real time. Segel and Heer [55] provide a comprehensive review of narrative visualization as used by online journalists. These works motivate the value of using narrative to communicate unfamiliar and complex concepts to the lay audience.

1.2 Narrative Frameworks

Many researchers have analyzed and designed a number of methodologies and frameworks to apply narrative techniques [55, 38, 23, 34, 42]. Narratives convey a message, enhance comprehension, make transparent causality, increase engagement, and summarize and simplify a complex message. In the narrative visualization literature there are a number of approaches on how to embed a narrative. Segel and Heer’s case study of Gapminder Human Development Trends explores how narratives make it possible for complex information to be comprehended quickly by the user [55]. This is achieved through the use of graphical elements such as animations and the use of color schemes. Hullman and Diakopoulos discuss a set of omission techniques [34] used for simplifying complex data. Lee et al. describe a process of working with data analysts to extract only what is relevant to the story [41]. All three studies present different approaches for presenting complex data. With Sea of Genes the content is both complex and unfamiliar so we applied each of these approaches in an effort to communicate our narrative.

Gershon and Page [23] contend that conveying a story in general is more effective when images are combined with text or data. While images in themselves convey a lot of information, we need data and text to reduce the ambiguity in the message. They also identify continuity as an important element. Continuity may result from a causal flow that also enables retention and recall. If a user perceives continuity, this may also imply that they have understood the causality. Since our interest is in our users comprehending our message we explored methods to aid users in perceiving continuity.

Segel and Heer [55] provide an overview of how visual elements have been employed in traditional media such as comics, books, and films to tell stories. Their focus is on the role of graphical elements and interactivity in maintaining continuity in the flow of the narrative. They identify author-driven and reader-driven as two polar extremes of visualization. In an author-driven approach, the story is linear and the user has no control on the flow. In a reader-driven approach, there is no predefined flow. They are concerned with approaches for balancing these two extreme approaches. By blending these two extremes they hope that some continuity can be maintained while allowing the user to explore or meander, an approach we took with our design.

The previous work suggests that by using these techniques we can explain content from an unfamiliar and complex domain to novices. The application of narratives is a widely used technique in history and art museums to help visitors make personal connections to an object or a collection [12, 52]. A research study on the roles narratives play in interactive science exhibits found that enhancing exhibits with personal stories improved the exhibit experience for visitors and helped visitors make personal connections to the scientific content. However, adding stories seemed to reduce the visitors’ physical interactions and explorations with the exhibit [10]. Similarly, a study looking at using a narrative introduction to describe the dataset visualized in an exhibit, found that it did not improve data exploration [43]. When and how to incorporate narrative into science exhibits remain unanswered design questions in the museum field. Sea of Genes provided us with an opportunity to assess the effectiveness of these methods and frameworks.

1.3 Animation for Learners

Research evaluating the effect animation has on learning has gone through two eras of consideration. In the first era (1990’s), scientists tried to ascertain the effectiveness animation has on learning by evaluating it next to static graphics [62, 30, 45]. These studies report inconsistent or inconclusive findings on the effect of animation on learning. In particular, although Schnotz and Grzondziel [54] found animation performed better, it had an interactive component [22] confounding the benefits of animation. Tversky and Morrison. [62] were highly skeptical animation could be effective for conveying complex systems. They suggest two principles to note that provide conditions for an animation to be effective, Apprehension and Congruence. The Apprehension principle states ”the structure and content of the external representation should be readily and accurately perceived.” A drawback of animation is the perceptual and cognitive limitation of processing a changing visualization, e.g. complex interactions may occur too quick to be understood. The Congruence principle states ”the structure and content of the external representation should correspond to the desired structure and content of the internal representation.” In principle, animation should be effective for expressing changes. Most animations violate these principles. They can have too many moving parts making it difficult for a viewer to know what to observe. People conceive a dynamic process as a sequence of steps, thus violating the Congruence principle. In order for an animation to be effective Tversky [61] believes that animations must explain rather than simply show.

Recent studies investigate the effect animation has on learning rather than comparing its effectiveness to static graphics. These studies focus on understanding the cognitive processes involved in processing dynamic visualizations and identifying the steps leading to comprehension [47, 8]. Berney and Betrancourt conduct a meta-analysis on animation for learning [13] and section the factors into three main groups: (a) specific to the learners, (b) specific to the instructional material, and (c) specific to the learning context. Studies [17, 37] that address group (a) found varying prior knowledge requires varying presentation forms in order to achieve a learning task. Since our visitors’ prior knowledge of the subject vary widely, due to the domain being unfamiliar, we needed to consider other ways to reinforce the animation. Chanlin [17] found animation enhanced both novice and experienced learners’ learning. Specifically, for novices it helped facilitate learning of descriptive facts. Berney and Betrancourt form a hypothesis off Ploetzner and Lowe’s [50] work that ”well-designed” expository animations contain all the elements needed to draw learners’ attention to the right place at the right time.

With our exhibit we have a complex system depicting that the interactions of microbes are driven by their genes. The work of Tverseky and Morrison [62], though, alludes that it is ineffective to use animation to convey a complex system there are considerations and areas where we can evaluate if it can be done. By eliminating unnecessary information and directing attention to key points it is possible to address the violations. Our strategy to adhere to these principles with Sea of Genes, is to provide a narrative to link each step in the animation to explain what was occurring to visitors. We apply narrative techniques to direct visitors’ attentions to each subplot of our narrative, so they are able to follow along and better infer what is to occur next. By having a narrative to stick with visitors should be able to stay focused and their mental model of what is occurring should fall in line how we present the narrative.

2 Sea of Genes

Sea of Genes is an interactive visualization exhibit at a science museum. Through a collaboration between visualization researchers, graphics designers, museum exhibit specialists, and biological scientists, we designed a visualization that tells museum visitors a narrative based on a metagenomic dataset.

2.1 The Dataset

The data used for Sea of Genes was collected and analyzed by ocean researchers at the Simons Collaboration on Ocean Processes and Ecology (SCOPE) at the University of Hawaii. They provided access to parts of the dataset and assisted in interpretation. These data were the basis for the 2014 study published from University of Hawaii in the journal Science [48]. They were collected using an Environmental Sampling Processor (ESP) [11], a glider that collects environmental and genomic data at specified depths in the ocean. Planktonic microbial assemblages were collected in the 0.225 microgram size range at 4-hour intervals and preserved aboard the ESP. Filter samples collected by the ESPs were stored at C within 24 hours of their retrieval. RNA was extracted, cDNA was generated, and Illumina sequencing was performed. Metatranscriptome reads were mapped to ortholog clusters of proteins constructed from the phylogenetic groups of interest, and a weighted transcriptomic network approach using the R packages WGCNA and igraph were used to analyze the resulting count tables. The data contain information about microbes such as: the depth of collection, gene expressed, and the peak time of expression. From this, SCOPE scientists infer which microbes are present and what functions they are performing.

2.2 Hardware and Implementation

For Sea of Genes we decided to use an interactive touch table, which have been show to encourage collaboration and attract attention  [33, 36]. A feature of the museum context is the ability to support social experiences [21]. Hinrichs et al.s [31] findings suggest that using a large interactive display gives the visualization a presence within an exhibition. These displays allow people from afar to enjoy and participate and decide whether to engage further. In our previous project we used the Multitaction object-tracking table [6]. This table worked well in attracting and engaging visitors with the visualization. To take advantage of the social context of the museum we used a larger 3M 65 touch-table as our exhibit display. With this exhibit we wanted to use a larger display as well as leverage 4k resolution. It is large enough to accommodate either 6 users interacting all around it or 3 users from one side.

To support our iterative development cycle Sea of Genes is web-based and written in ECMAScript 6, JavaScript 6. We used JavaScript to develop our exhibit as it is lightweight and is great for quick and rapid prototyping. The exhibit has a configuration file that can be modified by museum staff, allowing them to control a number of parameters such as the number of microbes and length of time for the 24hour period to cycle. Since the exhibit is written in a web-based language it can be deployed on any platform. For our own development we deployed the exhibit on OSX architecture. To package for distribution we used the open source software Electron by [1].

3 Museum Context and Evaluation Methods

To understand metagenomic data, museum visitors may need significant prior knowledge, including understanding of microbes, genes, and gene expression To create an experience around this complex dataset, we worked with SCOPE scientists to (1) synthesize their research into a narrative comprised of three stories, and (2) apply techniques from established narrative frameworks to tell this narrative:

  • Microbes interact with each other in a predictable daily rhythm.

  • Genes turn on and off according to a daily rhythm.

  • Scientists collect data about the genes of a microbes.

With the first story of our narrative, we sought to convey to visitors that microbes form communities. Microbes, interestingly enough, interact with one another in order to survive. The interactions and behaviors that these microbes exhibit during the day are different from what they perform at night, similar to us. The second story focused on how these microbial interactions and actions are a result of gene expressions. The final story is that scientists documented and found which expressed genes are responsible for a microbial function. Our design needed to convey the narrative to visitors through an exhibit. In addition, we had to consider the museum exhibit experience itself, which is often characterized as unmediated, episodic, and short. Given these constraints we needed to develop an exhibit that would effectively covey this narrative.

3.1 Museum Considerations

Museums are informal learning environments referred to as ”designed environments”, in which exhibits are developed and interpreted by the museum to help structure visitor experiences, in line with institutional goals and values [9]. As with other types of informal learning environments, the experience in museums, as compared to the formal setting of the classroom, is motivated and guided by personal interests rather than compulsory requirements [10, 12]. This is often referred to as a ”free-choice” learning environment.

As free-choice learning environments [9], museums employ a variety of techniques to attract and sustain visitors’ interests and engagement at exhibits. Horn et al. discuss possible advantages of tabletop games for learning in museums. These games motivate engagement and facilitate productive collaboration [32]. One exhibit of note is DeepTree [19], a project that produced interactive visualizations of the tree of life on a tabletop display. Davis et al. present an account of the role of design for DeepTree in allowing visitors to make sense of large information visualization exhibits. They were interested in understanding designs that encourage collaboration among children. They explain how key elements of their design, like strategically placing features that invite attention [19], encouraged collaboration as well as provided a medium for it to take place. Claes and Moere [18] find narrative strategies on public displays support deeper reflection on the data, connects authorship to the surrounding environment, and overcomes comprehension issues. With Sea of Genes we are interested in determining how the use of narrative design choices influenced user comprehension with an animation on a tabletop display. DeepTree is a visualization that is not driven by a narrative but is primarily an interactive tool. It is an example of teaching complex relationships through a game-like exhibit where dyads, a pair of people, can discover relationships.

3.2 Evaluation Process

Formative evaluation is an integral part of the iterative exhibit development process at the Exploratorium (Figure 3). Depending on the complexity of the exhibit, development may entail several rounds of prototyping and evaluation, with each successive round testing more sophisticated prototypes with modifications informed by visitor feedback and behavior data collected through evaluation.

For Sea of Genes we conducted three successive rounds of prototyping and evaluation. For the first two iterations we invited visitors to use the exhibit which we placed in the Life Sciences area of the Exploratorium. The evaluator stood near the exhibit and approached every third visitor, who appeared eight years or older, who crossed a predetermined imaginary line. If the visitor was with a group the larger group was invited to participate as well. Consenting visitors were asked to use the exhibit while an evaluator took notes on their interactions. The evaluator verbally described basic aspects of the animation to the participants. They were encouraged to think aloud while they interacted with the exhibit; this was also recorded. When participants indicated that they were done, the evaluator asked the one who interacted the most a set of questions designed to gauge usability and comprehension.

For the final round of evaluation an independent evaluator, Inverness Research, conducted a naturalistic study to evaluate whether the exhibit met both the design requirements.

Figure 3: Iterative design process for making an exhibit. It is very common to take 3-4 iterations before an acceptable design is derived.

3.3 Creating the Stories

Using the storytelling process described by Lee et al. [41], we spent time exploring the data and extracting data excerpts to use and support each story. We knew both the gene activity analyzed and the microbes were unfamiliar to visitors [42, 40]. We reduced the number of microbes we would focus on with the assumption it would improve visitor comprehension [34]. Our partners at UH suggested selecting microbes based on their roles in a microbial ecosystem: phototrophs (energy from the sun), photo-heterotrophs (energy from the sun and from eating other forms of energy); heterotrophs (cannot make their own energy from the sun and get by consuming other forms of energy); and viruses. Four microbes were picked for the first prototype (Figure 2). Next, we defined what microbial functions we would depict. We selected functions that had a strong daily pattern and functions that were familiar to museum visitors. We needed functions that had a strong daily pattern as a way to show observers microbes follow a daily pattern. To identify familiar functions we relied on the Next Generation Science Standards [40]. It specifies which science concepts are taught for each grade level and is often used to guide the design of educational experiences. The Exploratorium often designs for middle school level, however, we found that most of the functions were not covered until high school. We selected functions behind photosynthesis and cell division, which are taught by the end of high school. Viruses are not in Next Generation Science Standards, so we relied on a prior front-end study [57] that found 71 percent of teens knew viruses caused infections, and 79 percent recognized images of the type of virus used in the Hawaii dataset.

4 Visualization Design

Sea of Genes was created using an iterative design process, shown in Figure 3. Three iterations were designed and tested, each adding on one story of the narrative described in Section 4. The following discussion is organized according to key design decisions made during the iterative development and evaluation process.

Figure 4: (a) Screenshot from Sea of Genes Prototype 1, which visualized four main microbial characters depicted as 4 unique icons. This visualization was tested on a large tabletop display. (b) Microbial community interacting. (c) Cards providing information for each microbe.

4.1 Prototype 1

After assembling our stories, the team focused on the form the visualization should take to address the design goals of the exhibit. A previous study was conducted on Exploratorium visitors’ understanding of microbes [42]. The study found that 95 percent of visitors knew microbes lived in the ocean, and that the majority (71 percent) were aware that microbes have genetic material. This study also found that museum visitors had interest in the functions of microbes. From this study we could assume visitors knew microbes have genes, and focusing on functions would be attractive to visitors.

Once we had our functions selected we worked closely with our scientific partners at the UH to create a model of microbial interaction. The model was used to simulate a twenty-four hour period describing the functions each microbe performed. We chose to present this information through animation. We felt this was a valuable opportunity to evaluate and test the capabilities of animation when used for explaining metagenomics to the public. There is some promise that animation could be effective for conveying these concepts to novices [17]. We felt this could be a good opportunity to determine if a ”well-designed” expository animation really does contain all the elements needed to draw the learners attention to the right place at the right time [13].

With Prototype 1 our intent was to have a very minimalist animation. The animation had three main elements: (1) Four main microbial characters, (2) Watery background that changed shades between night and day to connote time, and (3) Control panels that described each of the four microbes (Figure 4c). Our animation (Figure 4a) was driven by our curated model of microbial interaction. We created non-scientific names for these microbial characters to reinforce their functional role in the ocean ecosystem and lower the cognitive load on our visitors [44, 46, 24]. “Sun Harvesters” were the name given to Prochlorococcus, a microbe that makes energy from the sun. “Sugar Eaters” were the name given to SAR116, a microbe that lives on sugars produced by other microbes. “Ammonia makers” were the name given to Crocosphera, a microbe that makes ammonia. “Virus” was the name given to cyanophage, a virus that infects Prochlorococcus. The background would transition from light to dark blue and back over a set period to illustrate a 24 hour period. The control panels were static and only provided textual information about each microbe. The animation itself (Figure 4b) portrayed microbes as icons interacting with one another. We included a simple interaction of tapping on a control panel to inject a microbe into the pool. The intent of this prototype was to determine if this exhibit design direction had any potential. We wanted the first story to come across, i.e., for visitors to observe that microbes are interacting.

4.1.1 Takeaways and Next Steps

For prototype 1, we used the evaluation process described in Section 4. We were interested in points of confusion while interpreting the animation. We also wanted to determine if the first story of our narrative could be conveyed. This evaluation was performed over four days. We recruited a total of 38 visitors 111The demographic breakdown of the evaluation participants was: 18 adults, 12 teenagers, and 8 children, with 20 females and 18 males..

We found that 55 percent of the 38 visitors reported interest in seeing interactions between microbes. This led us to believe they understood some aspect of microbial behaviors. We received comments of visitors interpreting or noticing interactions:

Visitor3: [It was interesting] infection of Cyanophage to
Prochlorococcus, and SAR116 eating. How it
[infected Prochlorococcus] burst and a bunch came
Visitor37: One thing was sugar eaters and sun harvesters
working together [ was interesting]. [I] noticed
the sugar eaters were going by sugar harvesters
and right after the sugar harvesters were making

A small number (29 percent) of visitors were able to notice and appreciate differences between day and night.

Visitor38: The light and dark. Seeing difference between
what’s there and thrives in the light versus dark.

However, 58 percent of the visitors found parts of the animation confusing. In particular, there were difficulties interpreting the animated behavior and making sense of that behavior; for example:

Visitor1: Some would disappear, especially this one [SAR116].
It was hard to see why they disappeared.
Visitor17: These [sugar eaters] in particular appear to move
randomly. A better graphic representation would be
Visitor24: No idea what those guys [ammonia makers] do.
Visitor29: Didn’t understand what sugar eaters were doing.

Figure 5: Genetic Rhythms Version 1 control panel and interpretive label. (a) Control panel describing the microbe, pictured is Sun Harvesters, and a Gantt chart showing the current active behavior. A caption below the chart describes what the microbe is currently doing. (b)Annotation providing information about the microbe’s current action. (c) Panel that runs on the vertical axis of the exhibit provides additional information about the data.

The majority of visitors noticed the microbial interactions and behaviors with a smaller minority noticing the day-night cycle. However, for some visitors, the noise in the form of random movement and poor assets occluded their ability to see the daily patterns encoded in microbial behavior. Our evaluation of this prototype showed visitors had interest and noticed interactions between microbes. A few were able to notice behaviors such as infection, eating, and working together. The majority, however could only glean that there is a microbial community here but failed to discern the specifics of the interaction. This may have been due to the presence of multiple animated elements and we hypothesize this prototype fell to Gestalt principle of Common Fate [49]. This principle states that we perceive visual elements that move in either the same speed or direction as parts of a single stimulus Multiple animated elements also have distracted visitors from paying attention to the background color change since there were so many microbes moving. We needed to improve how we portrayed our microbes to make clear the interactions of interest. Nevertheless, one takeaway was that this animation was successful in conveying the behaviors within a microbial community. Based on these findings it was clear that we had to improve our animation to better support visitors’ interpretation of the microbial interactions and their daily rhythm.

4.2 Prototype 2

For the next version we wanted to direct visitors to the other stories of the narrative embedded in the animation. Our previous evaluation indicated that visitors were drawn to the pool of microbes interacting. However, they were only able to observe that there is a pool of microbes intermingling. As a result they could not make the inferences necessary to continue to the rest of the narrative. We needed to better guide them such that they could see the details and gain a deeper understanding. This would require improving aspects of the animation so it would become easier to perceive. Furthermore, for this version we used more of an author-driven approach for presenting the narrative to have more control on guiding visitors through story. These changes introduce more information, therefore we must consider balancing how can be presented to our visitors before they are overwhelmed [23, 13]. To accomplish these tasks we focused on the following elements of the exhibit.

  • Defining the appearance of microbes and their behavior. Visual designers worked closely with the UH and PI Frazier to define how microbes and their behavior would appear in the exhibit (Figure 6).

  • Designing a control panel for interaction and interpretation. The key site of activity was a control panel with a well of microbes that could be dragged into the exhibit. It also contained a description of the creature and a visualization of the timing of activity in Gantt chart shown Figure 5a.

  • Adding annotations to direct visitors to the layer that microbial behavior and relationships follow a daily pattern (Figure 5b).

To improve perception of each story in our narrative, we sought to reduce noise and confusion by improving the quality of assets and animations of the previous version as well as reduce the number of microbes [34, 62]. Visitors who saw prototype 1 had trouble identifying the microbes. Therefore the microbe community was slimmed down and the assets of each microbe were changed to be pseudo realistic compared to the previous icon like style. The simulated ocean animation now only contained three microbial types: SAR116, Cyanophage, and Prochlorococcus as shown in Figure 6. We also want to reduce the amount of information visitors needed to process and allow them to now look around and see more. There were fewer more distinct animations for each microbe so a visitor could see what was occurring. The most significant addition to the exhibit was the use of staged annotations on the microbes to guide visitors to one of our stories.

With the previous iteration it was more reader-driven. For this version we wanted to assert our narrative so we added annotations to direct visitors to the stories. Annotations have been used effectively in several studies of information visualizations [34, 29, 35, 25] to add information, convey meaning, show data provenance, represent uncertainty, and highlight points of interest for users. Annotations also strengthen the narrative by drawing attention to aspects of the story we want to tell. Our annotations would pop-up and highlight an event (Figure 5b) and reinforce when it occurred and deliver a characterized message of what was occurring. The purpose of the message was to anthropomorphize the microbe and highlight the key interactions. In marketing, anthropomorphism has been shown to have positive and significant influence on personal value  [27, 51]. By providing human-like characteristics to the messages we theorized that visitors would engage more and be able to follow the narrative. These annotations would be triggered when an observable function occurred. Only one would be triggered at a time to not overwhelm the visitor but to guide them so the can follow along with what was being presented.

Lastly, we updated the control panels (Figure 5a) by simplifying the text and providing a Gantt chart. The introduction of an animated Gantt chart synced to the period of the day was to reinforce that these behaviors happened at a specific time. It showed the duration of an ongoing microbial behavior. These devices were introduced to evidence the time dimension of microbial behavior. Although animation implicitly illustrates time [8], we needed to convey to the visitors the repetition of similar behaviors during the 24 hour period. To support the third story of our narrative, data authenticity we added a side panel describing the data and why scientists are interested in it (Figure 5c).

Figure 6: The design process of transforming the microbes into the characters of our stories. For scientists to give us input we used icons to represent each, in the further iterations we used sprites. The final image are the representations of SAR116, Cyanophages, and Prochlorococcus in Version 1.
Figure 7: (a) Final design of the Sea of Genes exhibit. (b) Legend containing information about Sun Harvesters and in the center is the activity gene showing the genes responsible for making sugar are being expressed.(c) Annotation showing what the Sun Harvester is doing.

4.2.1 Takeaways and Next Steps

This evaluation was conducted with 21 museum visitors recruited from nearby the exhibit prototype 222There were 15 adults, 5 teenagers, and 2 child participants. Nine were male, and 12 were female..

Our evaluations showed that these improvements were noticed by our visitors. They continue to recognize community interaction and now could identify specific functions that occurred. We found that, similar to the first prototype, visitors understood that the exhibit was about marine microbes. The first layer was conveyed. Majority of visitors (71 percent) thought it was about the interaction of microbes, with 86 percent of visitors mentioning at least one microbial interaction when asked what they saw in the exhibit. These findings suggest that a majority of the visitors understood aspects of the first layer of the story: Microbes interact with each other. But, they continued to struggle with noticing the daily cycle, with only one-third of the visitors finding the day versus night differences interesting. This is despite the addition of the Gantt chart adding a redundancy as well as emphasizing temporal patterns in both the control and the label.

The third layer of the story was only partially conveyed. Although 62 percent understood what they were observing was based on real data, only 10 percent made any mention of genes. A third of that group thought what they saw in the animated ocean was a representation of what researchers see. This is not the case. For example:

Visitor8: They went on a boat and collected it in a bucket and
put it in a petri dish and put it under a microscope
and looked at it

For our next version we sought to improve how we convey the second and third stories. Visitors were able to track temporal changes such as noticing microbes are performing different abilities over a period. However, they did not notice the specific period when these behaviors were occurring, so we needed to put more consideration into making the time variable more evident. Finally we needed to highlight the underlying meta-genomics and meta-transcriptomics data, a fact missed by the majority of visitors.

4.3 Prototype 3: Final version

For the final iteration of Sea of Genes, we focused on conveying the second and third layers of the story by making stronger connections to the underlying metagenomic data .

Based on the Apprehension and Congruence principles [62] we hypothesized that the volume of microbes performing an action drew visitors attention much more than the animated background transition. To make the stories stand out even more we reduced noise again. We also note that the success of conveying the first layer of the story in the previous prototype may have been a result of interaction. We chose to eliminate the interaction which let visitors add microbes to the exhibit. The intuition was that the interaction distracted visitors from the animation preventing them from noticing the day to night transition. The current interaction component itself did not help to convey the story. The exhibit was now primarily an animation of microbes performing behaviors throughout a twenty four hour period.

We fixed the orientation and reduced the number of panels around the table to focus attention on the animation. We hypothesized that creating only one orientation for the exhibit might make it easier for visitors to use. We moved the label to the left side of the exhibit and removed all complex graphic elements [44, 46, 24] (Figure 7a).

Majority of our efforts were focused on conveying that this exhibit was based on genetic data. We included the DNA helix icon on our annotations as way to hint that genes play a role (Figure 7c). Prior evaluations suggested visitors were learning new things about microbes in the ocean and how they interact. However, two of the three layers were involved tying it to genes. These additional layers, that microbes behavior is driven by genes and that scientists collect data about their genes, needed to be emphasized further in the final version. We revised and added a number of elements to reinforce these two stories:

We hypothesized that a Gantt chart was not the most effective representation to convey the nature of the underlying data. We designed a new representation to show genomic data which was placed in the legend for each microbe (Figure 7b). The representation the team settled upon was similar to Circos visualizations. Circos visualizations have been used twice in the New York Times to supplement stories on metagenomics and comparative genomics, as well as a number of magazine covers [39]. Each gene related to a particular behavior was grouped around the circumference of the circle (all of the photosynthesis related genes under Preparing to Make Sugar). Each gene had an activity range and was displayed as a dynamic histogram. The more active a gene, the higher the amplitude of the bar. So through the course of twenty-four hours different areas of the circle would have waves of gene activity, similar to an equalizer. The new label focused on telling the third layer of the story of how the data was collected, and how the representations in the visualization were linked in four steps: (1) Collect Water; (2) Extract DNA; (3) Analyze Gene Activity; (4) Tell the story.

To reinforce genetics are a key component of what was being observed we made the following additions: A large title that included the word genes (A Sea of Genes), the legends and microbe annotations were adapted to refer to genes and the iconic DNA helix was added to the middle of the gene activity wheel and the annotation boxes.

4.3.1 Final Evaluation

For the final evaluation, we contracted with an external museum evaluation group, Inverness Research, to perform the study using mediated interviews. Their mediated interviews indicated that visitors could not figure out what to do or where to begin at the exhibit, and were often confused. For example, seven visitors from the 13 groups we conducted mediated interviews with said they were not sure where to start or what to do. As an example, A mother in Visitor Group A:

Visitor Group A: Honestly, its way over my head. I’m interested
in whats happening, I just don’t know what to
do and I can’t understand it. I guess there are
Sun Harvesters and Sugar Eaters? There is a
lot of empty space. I keep waiting for
something to happen. My instinct is to ask
how do I make it work so I can learn
something? But I can’t make it work. So there
are three types of genes?

In terms of the extent to which the exhibit helped visitors have an awareness that scientists study oceans by collecting and examining data, two visitors we spoke with in mediated interviews about the text understood that scientists collected data while on boats, and that something about what they collected is represented on the screen of the table. However, they did not identify the activation of genes as the data that was represented.

Despite our efforts at simplifying the exhibit, visitors were still extremely confused by what was happening or the intent of the exhibit. Usability problems made it difficult to determine if we could convey the last two stories.

5 Discussion

The intent of the Sea of Gene’s exhibit was to visualize different aspects of a complex metagenomic dataset to show the public new concepts about ocean ecosystems and how they are studied. We used narrative strategies described in our related works to construct and tell a set of stories. Overall, not all aspects of the story could be conveyed to visitors. We reflect on the strategies we applied to better understand visitors’ challenges with this exhibit.

Storytelling. Prototype 1 was based on classical approaches to developing a narrative [41, 38]. We worked with domain experts to extract important stories and insights. The purpose of this prototype was to tell the first layer of our story – that microbes interact with one another and that these interactions follow a daily pattern. This narrative was told over a twenty-four hour loop and time moved the story along. To emphasize microbial behaviors, at any given point in simulated time multiple identical behaviors could be observed. The random movements were needed to inject variation and enhance the perception of reality.

Evaluation of Prototype 1 showed that an animated story based on the microbial data would be a promising direction. Majority of the visitors recognized part of the first story microbes interact and the nature of the interactions and behaviors change over time. However, only a small minority noticed the the other layer of the story, interactions during the night were different from that during the day. Furthermore, in both prototype 2 and 3 the additional layer telling how genomics drives the interactions seen was not noticed. We hypothesize that since museum visitors are motivated and guided by personal interests [10, 12] they are more likely to not see more than the first layer of a story. It is likely that the viewers were overwhelmed with the other information and not observe more of the story. While they noticed that behaviors changed they may have required more cycles to recognize the daily pattern. In short, the layered story is too complicated for the viewers to identify in one cycle and is not sufficiently rich to hold attention over multiple cycles.


Figure 8: Circos-inspired visual representations of genomic expression over a 24 hour period. Illustrates that behind for every behavior observed, a set of genes are expressed causing it. Left shows genes activity is on for sugar eater’s dividing. Right shows genes related to copying are expressed for virus.

With prototype 3 our evaluations indicate that visitors found it unusable. Looking back at prototype 2 we improved assets again, streamlined it so the visualization could be seen from one direction, introduced 3 cards with a Circos visualization (Figure 8), and provided a side panel with information. Tapping on a card would highlight the microbes affected by the expression. Color was used to highlight and link the microbes to their corresponding Circos visualization (Figure 7a). When reading the evaluation we initially hypothesized we overwhelmed our users with the domain. The connection between the genes being expressed and the observed actions was too difficult to communicate. The evaluation reports show that within 10 seconds visitors were to overwhelmed and confused with what was presented. Many of them left. Visitors reported that it wasn’t clear what to do. The differences in prototype 2 and 3 are relatively small. Yet, we find significant differences in how these two prototypes were experienced. The biggest change was the introduction of a Circos visualization. We hypothesize that this additional animation was a factor in overwhelming visitors.

With prototype 3 we adhered to the Congruence principle [62] tightly. We reduced microbes to reduce redundant and uninformative animations. What remained were the minimal necessary set of animations and visuals based on our evaluations from prototype 2. In prototype one our animation was caught in the Gestalt principle of Common Fate [49]. By applying and addressing aspects about the Apprehension principle [28, 62, 59] in prototype 2 we were successful in having visitors follow the first layer of the story. Visitor frustrations did not cross the threshold where they deemed the exhibit unusable at this point. Prototype 3 introduced an animated Circos visualization which we believe could have been a factor that resulted in the exhibit to be unusable. However, when we used an animated Gantt chart we did not find a similar reaction. It is possible the amount of animated movement scaled from the Gantt to the Circos chart more than the amount decreased in the central animation. Even though the increased animated movement is subtle from Gantt to Circos. Both animations were separate from the larger and in similar locations.

From prototype 2 to 3, we removed considerable additional information that could confound our visitors and simplified the rest. Reduction of six cards to three, fixed orientation of visualization, cleaner more polished assets, fewer microbes on screen as well, and simpler text. Suggesting that there is a threshold for the number of unique animated transitions occurring at once that can be processed before viewers are overwhelmed and confused.

Data Authenticity. Prototype 1 was largely successful in conveying the targeted layer. A minor issue was its inability to convey clearly the daily pattern. Encouraged by these findings, in Prototype 2, we tried to add another layer of the story, while strengthening the daily connections in the first layer. The purpose of this layer was to convey was that the exhibit was portraying microbial data scientists had gathered from the ocean and not in a laboratory. To emphasize the daily pattern, we reduced the noise and complexity by eliminating some ancillary features and removing microbes. To aid comprehensions we incorporated annotations and other text [34, 29, 35, 25]. Finally, we added text to convey the provenance of the underlying data.

Even with these changes, we did not observe any change in the storylines that the visitors comprehended. Most visitors continued to recognize that microbes interact. Once again, they failed to observe the daily pattern in the behaviors. However, the changes made it easier for visitors to notice the interactions and behaviors. In short, the changes enhanced the comprehension of the same story lines that captured attention in prototype 1, but did not augment the set of ideas that visitors recognized. The variations between night and day clearly did not stand out. Finally, the visitors failed to infer the scientific process for this research from the exhibit.

Linking to Genomics. In the third prototype, since changes in behavior were being recognized, we sought to also convey the underlying genomics. Prototype 3 conveyed less than the previous prototype. The viewership declined and those who did visit the exhibit spent less time. The new features appear to have overwhelmed the observers. The exhibit seems to have failed the ”immediate apprehendability” criteria that [9] postulates as a critical requirement for exhibits in a museum. Apprehendabilty implies ” people introduced to it for the first time will understand its purpose, scope, and properties almost immediately and without conscious effort.”

Lessons. This exhibit was an ambitious effort to convey stories about the interactions and behaviors of microbes and the underlying genomics to the public. We were able to convey the first layer of our story; however, when we attempted to add layers to our story in parallel, we failed.

We see the need for additional research on how to to present a layered story in a busy public setting. In particular, further research on how to navigate between reader and author-driven methodologies in an interactive exhibit. When the content is complex and unfamiliar we find reader-driven approaches are less effective due to an increased cognitive load on visitors while navigating this material. We see this phenomena in our our first and second prototype. However, if we solely apply author-driven methodologies the visitor’s task is mostly observation. As a result, we find in our evaluation of the third prototype visitors experienced a loss of engagement and left the exhibit.

Narratives are predominantly linear and are most effective in conveying a single perspective. According to Spiro and Jehng [58], ”linearity of media is not a problem when the subject matter being taught is well structured and fairly simple. However, as content increases in complexity and ill-structuredness, increasingly greater amounts of important information are lost with linear approaches.” Can we create visual narratives that permit multiple perspectives and allow different narrative flows? As Hullman and Diakopolus [34] point out conveying a point of view requires careful over-emphasis. It is well known that multiple perspectives are needed to learn complex topics [20]

. We do not have good frameworks for classifying story complexity in a manner that can inform visualizations. A simple narrative has one causal pathway and is unidirectional. How do we characterize structures that are more complex? A taxonomy may allow us to develop visualization techniques. Effective storytelling is subject of interest to a diverse group of researchers in social sciences, computer science, and biological sciences. A taxonomy may allow us to map findings from these disparate domains and develop theories and guidelines. One approach could be to use graph theory. Here milestones, events, or information would be nodes and the flows or connections would be arcs. A simple story is a unidirectional planar graph with no branches. Let us call this a basic graph. In our narrative, we had a more general network. The activities of each microbe is close to a basic graph. The activities of different microbes interact creating a more general graph. Since these flows occurred in parallel and all share the same time dynamics we have a directed graph. The role of genes however changes the structure of the graph. We could view the genomics as nested information. Embedded in each node corresponding to an activity of a microbe, there was genomic data. In our visualization, we visualized the embedded data in a narrative that was occurring in parallel in a separate space. Our inability to connect effectively these two stories for the visitors suggests that for these types of graphs we need to consider other visualization techniques. In short, we contend that there is a need for a richer taxonomy. The graph approach appears to be promising and merits further exploration.

6 Conclusion and Further Work

In this paper, we reviewed our process for designing a museum exhibit to present complex data to the public. We reflected on our design choices and found reasons as to why it was unsuccessful. Despite multiple iterations of design and evaluation, we did not achieve all our goals. We created an animation of metagenomic data (i.e., interactions and behaviors of microbes) and embedded three narratives into it. However, as we adapted and iterated our exhibit to be closer to the complexity of the underlying data visitors became too overwhelmed. The few who understood what to do had difficulty linking animation to the underlying data. The introduction of an animated supplemental visualization crossed a threshold for what our visitors could comprehend. The use of narrative strategies to alleviate these issues worked for a single layer of the story. Yet, we were unable to bridge different layers to illustrate microbial behavioral patterns to their underlying genetics or how data is collected and used by scientists. We find animations with narration over it the ability to tell and convey layered stories. Where the audience is heavily guided by the narrator’s voice and is able to understand the nuances and depth in a story. Overall, visitors were simply overwhelmed by the amount of information being presented. Our team has successfully created interactive visualization exhibits in the past, but Sea of Genes presented us new challenges leading to new lessons learned.

Based on naturalistic observations and mediated interviews, we found that visitors could not understand where to start or how to interact with our exhibit. Similar to how interactive visualizations enable researchers to make discoveries, visualizations are a key component of scientific literacy. We hope this case study will stimulate additional research in approaches for visualizing complex data for the public to explore in physical settings like museum and visitors centers.


The authors wish to thank Elisha Wood-Charlson and Ed DeLong for sharing the SCOPE dataset and for their generous time and expertise collaborating on this exhibit. This exhibit would not have been possible without their assistance. We would also like to thank Nicolette Hayes, Eric Socolofsky, and Andrew Wong of Stamen Design for collaborating on the design and development of the prototypes, and Meghan Kroning,Tamara Kubacki, Katherine Nammacher, and Janine Penticuff for evaluation assistance. This work was supported by the National Science Foundation under grants DRL-1322828, DRL-1323214 and IIS-1528203, and by the Gordon and Betty Moore Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the view of the Foundations.


  • [1] Electron — build cross platform desktop apps with javascript, html, and css.
  • [2] Invisible you - the human microbiome exhibition. eden project.
  • [3] Science On a Sphere. National Oceanic and Atmospheric Administration (NOAA).
  • [4] The secret world inside you. American Museum of Natural History.
  • [5] Zoo in you. Oregon Museum of Science and Industry (OMSI).
  • [6] Living Liquid: Design and Evaluation of an Exploratory Visualization Tool for Museum Visitors. IEEE Transactions on Visualization and Computer Graphics, 18(12):2799–2808, 2012.
  • [7] Temporal Summary Images: An Approach to Narrative Visualization via Interactive Annotation Generation and Placement. IEEE Transactions on Visualization and Computer Graphics, 23(1):511–520, 2017.
  • [8] S. Ainsworth. How do animations influence learning. Current perspectives on cognition, learning, and instruction: Recent innovations in educational technology that facilitate student learning, pages 37–67, 2008.
  • [9] S. Allen. Designs for learning: Studying science museum exhibits that do more than entertain. Science Education, 88(1):S17–S33, 2004.
  • [10] S. Allen. Finding Significance. Exploratorium, 2004.
  • [11] F. O. Aylward, J. M. Eppley, J. M. Smith, F. P. Chavez, C. A. Scholin, and E. F. DeLong. Microbial community transcriptional networks are conserved in three domains at ocean basin scales. Proceedings of the National Academy of Sciences, 112(17):5443–5448, 2015.
  • [12] L. Bedford. Storytelling: The real work of museums. Curator: The Museum Journal, 44(1):27–34, 2001.
  • [13] S. Berney and M. Bétrancourt. Does animation enhance learning? a meta-analysis. Computers & Education, 101:150–167, 2016.
  • [14] F. Block, M. S. Horn, B. C. Phillips, J. Diamond, E. M. Evans, and C. Shen. IEEE Transactions on Visualization and Computer Graphics (TVCG), 18(12):2789 – 2798, 2012.
  • [15] J. Bruner. Actual minds, possible worlds. 1986.
  • [16] Y.-Y. Chan and H. Qu. Finavistory: Using narrative visualization to explain social and economic relationships in financial news. In International Conference on Big Data and Smart Computing (BigComp), 2016.
  • [17] L.-J. ChanLin. Animation to teach students of different knowledge levels. Journal of Instructional Psychology, 25(3):166, 1998.
  • [18] S. Claes and A. Vande Moere. The impact of a narrative design strategy for information visualization on a public display. In Proceedings of the 2017 Conference on Designing Interactive Systems, pages 833–838. ACM, 2017.
  • [19] P. Davis, M. Horn, L. Schrementi, F. Block, B. Phillips, E. Evans, J. Diamond, and C. Shen. Going deep: Supporting collaborative exploration of evolution in natural history museums. Proceedings of 10th International Conference on Computer Supported Collaborative Learning, 1(1):153–160, 2013.
  • [20] E. Einsiedel. The challenges of translating genomic knowledge. Clinical genetics, 70(5):433–437, 2006.
  • [21] J. H. Falk and L. D. Dierking. Learning from museums: Visitor experiences and the making of meaning. Altamira Press, 2000.
  • [22] E. L. Ferguson and M. Hegarty. Learning with real machines or diagrams: Application of knowledge to real-world problems. Cognition and instruction, 13(1):129–160, 1995.
  • [23] N. Gershon and W. Page. What storytelling can do for information visualization. Commun. ACM, 44(8):31–37, Aug. 2001.
  • [24] P. Ginns. Meta-analysis of the modality effect. Learning and instruction, 15(4):313–331, 2005.
  • [25] D. P. Groth and K. Streefkerk. Provenance and annotation for visual exploration systems. IEEE Transactions on Visualization and Computer Graphics, 12(6):1500–1510, 2006.
  • [26] J. Handelsman, J. Tiedje, L. Alvarez-Cohen, M. Ashburner, I. Cann, E. Delong, W. F. Doolittle, C. Fraser-Liggett, A. Godzik, J. Gordon, et al. The new science of metagenomics: revealing the secrets of our microbial planet. Nat Res Council Report, 13:47–84, 2007.
  • [27] P. M. Hart, S. R. Jones, and M. B. Royne. The human lens: How anthropomorphic reasoning varies by product complexity and enhances personal value. Journal of Marketing Management, 29(1-2):105–121, 2013.
  • [28] J. Heer and G. Robertson. Animated transitions in statistical data graphics. IEEE transactions on visualization and computer graphics, 13(6):1240–1247, 2007.
  • [29] J. Heer, F. B. Viégas, and M. Wattenberg. Voyagers and voyeurs: supporting asynchronous collaborative information visualization. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 1029–1038. ACM, 2007.
  • [30] M. Hegarty, S. Kriz, and C. Cate. The roles of mental animations and external animations in understanding mechanical systems. Cognition and instruction, 21(4):209–249, 2003.
  • [31] U. Hinrichs, H. Schmidt, and S. Carpendale. Emdialog: Bringing information visualization into the museum. IEEE transactions on visualization and computer graphics, 14(6):1181–1188, 2008.
  • [32] M. Horn, Z. Atrash Leong, F. Block, J. Diamond, E. M. Evans, B. Phillips, and C. Shen. Of bats and apes: an interactive tabletop game for natural history museums. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2059–2068. ACM, 2012.
  • [33] E. Hornecker. “i don’t understand it either, but it is cool”-visitor interactions with a multi-touch table in a museum. In Horizontal interactive human computer systems, 2008. TABLETOP 2008. 3rd IEEE International Workshop on, pages 113–120. IEEE, 2008.
  • [34] J. Hullman and N. Diakopoulos. Visualization rhetoric: Framing effects in narrative visualization. IEEE Transactions on Visualization and Computer Graphics, 17(12):2231–2240, Dec 2011.
  • [35] P. Isenberg and S. Carpendale. Interactive tree comparison for co-located collaborative information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6):1232–1239, 2007.
  • [36] G. Jacucci, A. Morrison, G. T. Richard, J. Kleimola, P. Peltonen, L. Parisi, and T. Laitinen. Worlds of information: designing for engagement at a public multi-touch display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2267–2276. ACM, 2010.
  • [37] S. Kalyuga. Relative effectiveness of animated and static diagrams: An effect of learner prior knowledge. Computers in Human Behavior, 24(3):852–861, 2008.
  • [38] R. Kosara and J. Mackinlay. Storytelling: The next step for visualization. Computer, 46(5):44–50, May 2013.
  • [39] M. I. Krzywinski, J. E. Schein, I. Birol, J. Connors, R. Gascoyne, D. Horsman, S. J. Jones, and M. A. Marra. Circos: An information aesthetic for comparative genomics. Genome Research, 2009.
  • [40] N. Lead States. Next generation science standards: For states, by states, 2013.
  • [41] B. Lee, N. H. Riche, P. Isenberg, and S. Carpendale. More than telling a story: Transforming data into visually shared stories. IEEE Computer Graphics and Applications, 35(5):84–90, Sept 2015.
  • [42] J. Ma. Visitors’ prior knowledge and interests in marine microbes and metagenomics: Living liquid front-end evaluation., 2011.
  • [43] J. Ma. Engaging museum visitors with scientific data through visualization: A comparison of three strategies. American Educational Research Association, 2013.
  • [44] R. E. Mayer and R. Moreno. A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of educational psychology, 90(2):312, 1998.
  • [45] R. Moreno and R. Mayer. Interactive multimodal learning environments. Educational psychology review, 19(3):309–326, 2007.
  • [46] R. Moreno, R. E. Mayer, H. A. Spires, and J. C. Lester. The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and instruction, 19(2):177–213, 2001.
  • [47] J. B. Morrison, B. Tversky, and M. Betrancourt. Animation: Does it facilitate learning. In AAAI spring symposium on smart graphics, volume 5359, 2000.
  • [48] E. A. Ottesen, C. R. Young, S. M. Gifford, J. M. Eppley, R. Marin, S. C. Schuster, C. A. Scholin, and E. F. DeLong. Multispecies diel transcriptional oscillations in open ocean heterotrophic bacterial assemblages. Science, 345(6193):207–212, 2014.
  • [49] S. E. Palmer. Vision science: Photons to phenomenology. MIT press, 1999.
  • [50] R. Ploetzner and R. Lowe. A systematic characterisation of expository animations. Computers in Human Behavior, 28(3):781–794, 2012.
  • [51] P. A. Rauschnabel and A. C. Ahuvia. You’re so lovable: Anthropomorphism and brand love. Journal of Brand Management, 21(5):372–395, 2014.
  • [52] J. Rounds. Storytelling in science exhibits. Exhibitionist, 21(2):40–43, 2002.
  • [53] R. C. Schank. Tell me a story. Northwestern University Press, 1990.
  • [54] W. Schnotz, J. Böckheler, and H. Grzondziel. Individual and co-operative learning with interactive animated pictures. European journal of psychology of education, 14(2):245–265, 1999.
  • [55] E. Segel and J. Heer. Narrative visualization: Telling stories with data. IEEE Transactions on Visualization and Computer Graphics, 16(6):1139–1148, Nov 2010.
  • [56] D. Smith, T. Grah, M. Murer, V. van Rheden, and M. Tscheligi. macroscope: First-person perspective in physical scale models. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction, pages 253–259, 2018.
  • [57] A. N. Spiegel, J. McQuillan, P. Halpin, C. Matuk, and J. Diamond. Engaging teenagers with science through comics. Research in science education, 43(6):2309–2326, 2013.
  • [58] R. Spiro and J. Jehng. Cognitive flexibility, random access instruction and hypertext: Theory and technology for the nonlinear and multi-dimensional traversal of complex subject matter. Cognition, education, and multimedia: Exploring ideas in high technology, pages 163–205, 1990.
  • [59] M. Suganuma and K. Yokosawa. Grouping and trajectory storage in multiple object tracking: Impairments due to common item motions. Perception, 35(4):483–495, 2006.
  • [60] S. G. Tringe, C. Von Mering, A. Kobayashi, A. A. Salamov, K. Chen, H. W. Chang, M. Podar, J. M. Short, E. J. Mathur, J. C. Detter, et al. Comparative metagenomics of microbial communities. Science, 308(5721):554–557, 2005.
  • [61] B. Tversky. What makes visualizations effective? In NIH/NSF Fall 2004 Workshop on Visualization Research Challenges., 2004.
  • [62] B. Tversky, J. B. Morrison, and M. Betrancourt. Animation: can it facilitate? International journal of human-computer studies, 57(4):247–262, 2002.