Narrative Transitions in Data Videos

09/11/2020 ∙ by Junxiu Tang, et al. ∙ Zhejiang University 0

Transitions are widely used in data videos to seamlessly connect data-driven charts or connect visualizations and non-data-driven motion graphics. To inform the transition designs in data videos, we conduct a content analysis based on more than 3500 clips extracted from 284 data videos. We annotate visualization types and transition designs on these segments, and examine how these transitions help make connections between contexts. We propose a taxonomy of transitions in data videos, where two transition categories are defined in building fluent narratives by using visual variables.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Related Work

In this section, we introduce related work with regard to animated transitions and data videos.

Animated transitions are commonly used in visualization presentations. Heer and Robertson[Heer2007] examined the effect of animated transitions between statistical charts and proposed detailed designs concerning the congruence of contexts. On the basis of an evaluation of those designs, they indicated that some animated transitions could keep viewers oriented. This notion means that well-designed transitions between visualizations can improve the viewer’s perception and cognition. Researchers have recently conducted more specific work on visualization transition designs for different presentation tasks, such as the analogy between visualizations [Ruchikachorn2015], aggregation operations[Kim2019], and visual grouping[Chalbi2020]. As for design details, Thompson et al.[Thompson2020]

proposed several dimensions of animated data graphics and classified transitions based on these dimensions.

Researchers also used animated transitions for data-driven storytelling. In Segel and Heer’s design space, as a type of visual narrative tactic, transition guidance promotes the use of conventional methods to achieve the continuity among different shots to keep viewers oriented when old scenes destruct and new scenes come[Segel2010]. Animated transitions can be a type of transition guidance when telling stories. Hullman et al. [Hullman2013] investigated the arrangements of narrative sequences based on transitions. Furthermore, Kim et al. [Kim2017] developed GraphScape, a synthetic model to evaluate the transition cost of visualization sequences. Researchers also use transitions to make stories cohesive in timelines [Brehmer2016] and slideshows [Wang2018]. Amini et al. [Amini2017] include Transition in the taxonomy of data clip types. They expanded a few transition designs in their proposed authoring tool, DataClips, from and to pictographs based on Heer’s work [Heer2007].

However, the discussed transition designs in the previous work have not well covered all transitions that connect contexts in the narrative of data videos. Data videos have a series of different forms, for example, animated graphics that consist of icons and characters, standard charts, pictographs. Given that not only visualizations exist in the data videos, but also iconic motion graphics, transition designs between scenes have a large space. In this paper, we expand the taxonomy to include specific designs and usage scenarios of transition clips. Additionally, we pay attention to how they cement the context in the linear narrative in data videos.

2 Content Analysis

We first state the definition of a transition in data videos in our research scope, followed by the adopted methodology.

2.1 Definition

Prior work has defined transitions in narrative visualizations[Hullman2013]. Hullman et al. [Hullman2013] consider the change between two independent visual expressions as a transition. In the previous investigation of data videos, Amini et al. [Amini2017] find that “most transitions in these videos are a combination of destroy/create clips rather than staged animations described in [Heer2007]”. However, after collecting and analyzing 284 data videos, we found many more different transition designs in data videos compared with the previous work. For example, transitions in Figure 1 connect the contents by using the shared visual elements of the successive scenes. The design of using shared visual elements is not strictly the staged animation [Heer2007], but they have similarities in reusing visual elements and keeping audience oriented in syntax and in semantic. In our work, we follow the prior definition of transitions  [Heer2007, Hullman2013, Amini2017] and expand it in a wider scenario. First, we identify clips in data videos, where a clip is an elemental unit of the data video sequence [Amini2017]. Each clip is considered containing two narrative states and one transition between these two states. A narrative state in a clip is defined as an informationally-distinct scene for presenting data facts or other video narrative, following the definition of narrative visualization state  [Hullman2013]. Therefore, we define the transition (also called narrative transition) in our paper as the change of two narrative states in a clip, where the narrative state can be 1) an animated content without visualizations or data-driven arguments; or 2) a visualization content that includes standard charts, pictographs, and other data-driven arguments.

2.2 Methodology

We conducted a content analysis [Krippendorff2018], which was also used in previous work  [Segel2010, Hullman2011, Byrne2015] to gain a comprehensive understanding of the transition design in data videos.

First, we collected 284 data videos from reputable sources, such as the leading media outlets, design associations’ portfolios, and video sites. We gathered these videos following the same criteria in Amini’s work [Amini2017]. The videos should convey data-driven arguments and contain related visualizations. The wide range of topics in the videos includes science, finance, politics, sports, and history. The video dataset includes animated motion graphics, photography videos, and combinations of them. We took apart these videos into single clips. We only focused on the clips that have at least one scene with visualizations or data-driven arguments. Finally, our dataset contains 3909 clips: 1644 vis-to-vis clips (42.1%), 1104 others-to-vis clips (28.2%) and 1161 vis-to-others clips (29.7%).

To examine the detailed transition designs in clips, we first reviewed selected samples and proposed an initial taxonomy considering the changing visual variables (e.g., position, color, and shape)  [Bertin1983] during the transition. We iteratively improved the taxonomy through multiple rounds of discussion among authors and attempts to label sampled transitions. Ultimately, we achieved an agreement on the final taxonomy (Sec. 3). We mainly considered the following questions when analyzing the transition design:

  • What are the narrative states (e.g., visualizations, animated icons) of the transition?

  • What visual variables have changed, and what unchanged during the transition?

  • How are the transitions visually presented?

  • What is the narrative relationship between two states?

Based on the taxonomy, two authors independently coded all the clips and reached an initial consensus on 87.7% (3430) transitions. The conflicts (12.3%, 479) were further resolved through discussion. Based on the agreement, we conducted a quantitative analysis on the visualization types and transition types in each clip. The complete results are attached to the supplemental material.

3 Taxonomy of Transitions

This section first presents an overview of our proposed taxonomy for narrative transitions in data videos and then introduces data-driven transitions in detail.

3.1 Overview

According to the definition of narrative transitions (Sec. 3.1), we examine the designs from two aspects, namely, data-driven animated transitions (e.g., staged animations [Heer2007], Figure 1(b), and Figure 3(a)(c)(d)) and non-data-driven ones (e.g., fade-in/off, Figure 1(a), and Figure 3(b)). We list five identified transition types in an order of the relevance to data, where the first three pertain to non-data-driven transitions and the last two belong to data-driven ones. The frequency of each transition type in the dataset (3909 clips) is also reported. Note that multiple types can be combined together to establish a transition in a clip. For example, Figure 3(b) uses a camera motion and a preserving guide in the case. We provide animated illustrations of these narrative transitions on the website: https://narrativetransitions.github.io/home/.

Refresh (53.0%, 2070) means a complete update of the previous scene (Figure 2(a)). In this type of transitions, no connection exists between the last frame of the previous scene and the first frame of the coming scene. Usually, this mechanism can be used to present a new topic or an abrupt turn. We place the combinations of the destruction of previous scenes and creation of coming scenes in this category, for example, Hard CutFade, and Wipe.

Halftime (2.5%, 98) adds a new scene between video context (Figure 2(b)). Such a scene is comparable to a quick half time or the stage curtain between the previous scene and the coming scene.

Camera Motion (14.9%, 583) updates the scene due to the changing viewpoints or screen focus. The seven subtypes of camera motion are[Rea2015, Storyblocks2019]: Pedestal, Truck, Tilt, Pan, Dolly, Zoom, and Rack Focus. Pedestal means moving the camera vertically. By contrast, Truck means moving the camera horizontally. Tilt and Pan also mean moving the camera in the vertical and horizontal direction, respectively. However, they both require the camera to keep a stable focus during the movement. Dolly means moving the camera forward or backward. Zoom means changing the focal distance. Rack Focus means changing the focus in the scene, for example, bokeh effects. Camera movements can create a sense of space over scenes, and usually, they are used to present spatial visualizations, such as maps. The changes in the focus of scenes can highlight the key points of narrations according to the video designer’s intention.

Preserving Guide (22.9%, 894) reuses elements in the previous scene as a visual guide that directs to the next scene (Figure 2(d)). For example, a preserving guide could be a flying icon (Figure 3(b)), a colored area, or a stable line, to lead the viewers’ attention between two consecutive scenes. This guide can be used with camera motions to construct fluid transformations (Figure 3(b)).

Narrative Agent (19.9%, 777) means substitutes for data during data-driven storytelling (Figure 2(e)). The transitioned elements could be regarded as agents of data attributes or data values. This transition illustrates the change of data such as Scaling and Merging.

Different transition designs are useful to connect diverse narrative states. For example, in bar charts, data-encoded bars can be used as Narrative Agents to present the change of data. Another example is that Camera Motion can be used to transform among different places on maps because this transition can generate a sense of space.

Figure 2: Taxonomy of transition designs in data videos (a-e). We also present changed and unchanged visual variables of Preserving Guide and Narrative Agent transitions (f).
Figure 3: Screenshots of the video Global Wealth Inequality[TheRulesOrg2013]. We added green marks on the screenshots to show the animation between two consecutive scenes. We presented the transition types, visual content and the corresponding transcripts (gray text) under each clip sequence.

3.2 Data-driven Narrative Transition

Transitions, such as Refresh, Halftime and Camera Motion, can be used not only in data videos but also in other video narratives. Refresh is the most popular transitions in the dataset. This kind of transition has been integrated in existing video authoring tools [Premiere2020, AfterEffects2020] and can be easily employed in videos. Preserving Guide and Narrative Agent need craft animations to conduct fluent narrative. These two transitions are usually used in data videos to preserve data attributes or encoding data-driven insights, which present narrative information of the visualization content of data videos.

We paid special attention to Preserving Guide and Narrative Agent transitions to understand the data-driven design patterns of transitions in data videos. They make up a large proportion of the studied clips, especially in the 1644 vis-to-vis transitions, among which, 477 are Preserving Guide (29.0%) and 617 are Narrative Agent (37.5%). Within these transitions, changed and unchanged visual elements both exist. We focus on the visual contents within clips in terms of Bertin’s variables of the image[Bertin1983]. Those changed variables consist of the animation of transitions, while unchanged ones preserve the narrative information to maintain the congruence of the previous scenes and coming scenes. The narrative information could be data attributes and values preserved in special visual variables. The constant variables, which are similar to data agents, help viewers understand what narrative is going on. We list them in Figure 2(f). Note that these agent elements are not isolated from each other in practice. In the following sections, we introduce the narrative transitions of data videos in detail. Illustrations about each type of transition are presented in Figure 2(d) and (e).

3.2.1 Preserving Guide

A single or a group of visual items in the previous scene are kept and reused in the coming scenes. However, the layout of these visual items, for instance, orientation, size, and position, may change. These visual elements maintain the narrative information in previous scenes by preserving the same appearance, and at the same time, guide viewers’ attention to a new scene by changing the layout. This transition is particularly useful in setting up a supplement, correlation, or a comparison. We abbreviate this transition as RST Guide.

This type means two related situations. The first situation is to expand the color of a specific item in the previous scene to the background color of the coming scene. Often, the coming scene introduces details about the item. The other situation is to shrink the background color of the previous scene to a specific item of the coming scene. This type of transition is usually used when presenting the relation between the whole and the parts. Color is an essential visual variable in this type of transition: the same color of adjacent scenes illustrates the same subject.

This type of transition means that the layout of the visual items in the previous scene remains the same, and additional items are interacting with the existed items. The stable visual items are unchanged cue in the scene, and they can be regarded as the basis for the additional items entering, growing, and leaving.

3.2.2 Narrative Agent

This type of transition means updating the visual content without changing the shapes, positions, and colors of the contents in the scenes. In this transition, the shape, position, and color maintain the narrative information; however, the number of the contents changes to show the differences. Such a mechanism is used in data videos for highlighting or presenting data sequentially.

These transitions change sizes to show the value change. Although the size changes, the other visual variables remain the same, thereby making the contents coherent in different scenes.

This type of transition mainly morphs icons or shapes from old ones to new ones. This transition conveys insights into the transformation between data.

Merging means combining separate contents into a group, while splitting means separating a group of contents. This type of transition describes insights into the gathering or scattering. The whole-part relationship of data can be clearly illustrated by using this type of transition.

4 Case Study of Narrative Transitions

In this section, we take clips from one data video as a case to illustrate the effect of transitions in connecting narratives and visualizations. We highlight characteristic clips of these videos and mark transitions in bold texts. This video is about global wealth inequality[TheRulesOrg2013]. It presents a series of data facts about the wealth comparison between the poor and the rich. Most facts are linked through crafted transitions rather than simply fading in/off. We introduce five clips (Figure 3) in detail.

The story starts with a pie chart, which shows the population distribution (Figure 3(a1)). The red color represents the richest people, which is only 1%, while the pewter represents the other 99% of the people. Then, the pie chart to present another data fact (Figure 3(a2-a5)), where the thin red sector expands, while the pewter part reduces. By establishing contrasts of wealth and popularity between two groups of people, this transition reveals that the 1% richest people own much more than the rest. Within the transition, the color encodes the data category (the richest people and the rest). The color encoding also preserves the narrative information during the transitions of two pie charts to keep viewers oriented.

The following footage consists of two icons and texts (Figure 3(b1)). The scene shows the fact that 3 billion poorest people and 300 richest people own the same wealth. Afterward, the narrator mentions that the number of people it takes to fill a mid-size commercial aircraft has more wealth than the combined populations of four countries. In the clip, an aircraft icon flies through the scene, and the previous pictograph quits the scene under the Truck transition (Figure 3(b2-b4)). The icon moves from left to right and guides the viewers’ attention to the following world map. After this contextual data fact, the topic turns from inequality over the population to that over regions (Figure 3(b5)). The aircraft in this clip is a that attracts viewers’ attention and shifts the presented topic insensibly.

Then, the world map splits into two parts by two colors (Figure 3(c1)). Developed regions and less developed regions are encoded with red and blue, respectively. These regions first from the whole world map and then into two different circular areas (Figure 3(c2-c4)). Each circular area represents the total wealth of the merged regions. The areas consist of a proportional area chart of global wealth measured by poor and rich regions. The chart then by to show the comparison of the rich regions and the poor regions over two hundred years (Figure 3(d1-d5)). During this process, the layout and the color of the two circles keep constant, whereas the sizes of the two circular areas change according to the data. The change in sizes creates a prominent contrast on the wealth of the rich and poor regions.

5 Discussion and Conclusion

In this study, we investigate the taxonomy of transition designs in data videos. First, we collect the dataset of 284 professional data videos with 3909 clips. These videos cover various visual styles of visualization contents and diverse transition designs. Based on a content analysis on these clips, we propose a taxonomy of narrative transitions in data videos. With regard to the change of visual variables, we conduct a more in-depth analysis of data-driven narrative transitions that preserve narrative meanings of contents, namely, Preserving Guide and Narrative Agent.

The proposed taxonomy for narrative transitions takes a step in this direction and hopes to encourage future research. First, following the taxonomy, evaluations on specific transition designs are needed to assess the effectiveness of engagement and memorability of data videos. Designers can take transition designs into consideration when building the attention cues of data videos, for example, highlighting data facts or guiding viewer’s attention. Second, our taxonomy provides a new way of inspecting the relationship between narrative transitions and story sequences in data videos. Narrative transitions can not only highlight visual changes in presentations but also enrich narrations. We plan to propose a comprehensive model that considers both narrative transitions and story sequences. Third, our taxonomy can inspire the transition design for other genres of narrative visualizations, for example, animated long-form web articles or data-GIFs[Shu2021]. Finally, except for narrative, future work can consider other messages that transition can present, for example, visualization rhetoric[Hullman2011] and acquire codes[Byrne2015].

Acknowledgements.
The authors wish to thank reviewers for their valuable comments and suggestions. The work was supported by NSFC (61761136020), National Key R&D Program of China (2018YFB1004300 ), NSFC-Zhejiang Joint Fund for the Integration of Industrialization and Informatization (U1609217), Zhejiang Provincial Natural Science Foundation (LR18F020001) and the 100 Talents Program of Zhejiang University. This project was also partially funded by Microsoft Research Asia. L. Yu is supported by XJTLU Research Development Funding RDF-19-02-11.

References