Storyline Visualizations with Ubiquitous Actors

by   Emilio Di Giacomo, et al.
Università Perugia

Storyline visualizations depict the temporal dynamics of social interactions, as they describe how groups of actors (individuals or organizations) change over time. A common constraint in storyline visualizations is that an actor cannot belong to two different groups at the same time instant. However, this constraint may be too severe in some application scenarios, thus we generalize the model by allowing an actor to simultaneously belong to distinct groups at any point in time. We call this model Storyline with Ubiquitous Actors (SUA). Essential to our model is that an actor is represented as a tree rather than a single line. We describe an algorithmic pipeline to compute storyline visualizations in the SUA model and discuss case studies on publication data.


page 1

page 2

page 3

page 4


Energy Data Visualizations on Smartphones for Triggering Behavioral Change: Novel Vs. Conventional

This paper conveys the importance of using suitable data visualizations ...

Net2Vis: Transforming Deep Convolutional Networks into Publication-Ready Visualizations

To properly convey neural network architectures in publications, appropr...

Viral Visualizations: How Coronavirus Skeptics Use Orthodox Data Practices to Promote Unorthodox Science Online

Controversial understandings of the coronavirus pandemic have turned dat...

Systematic Literature Review on Cyber Situational Awareness Visualizations

The dynamics of cyber threats are increasingly complex, making it more c...

A Dyadic IRT Model

We propose a dyadic Item Response Theory (dIRT) model for measuring inte...

Visualizing Webpage Changes Over Time

We report on the development of TMVis, a web service to provide visualiz...

1 Introduction

Storyline visualizations have been the focus of intense research in the last decade. Originally introduced to describe the narrative of a movie [Munroe], this visualization paradigm has been successfully used to represent the temporal dynamics of the interactions between actors (individuals or organizations) in a social network or in a working environment ([DBLP:journals/tvcg/LiuWWLL13, DBLP:conf/softvis/OgawaM10, DBLP:journals/cg/PadiaBH19, DBLP:conf/cw/QiangB16, Tanahashi2012, DBLP:journals/tvcg/TanahashiHM15, DBLP:journals/information/TongRBWLWLWQLM18]). In a storyline visualization, the narrative unfolds from left to right, each actor is represented as a line, and two lines may converge or diverge at a time instant based on whether the two corresponding actors interact or not at that instant; see Fig. 1. Since a group of lines bundled together usually reflects an in-person meeting, a common constraint in a storyline visualization is that an actor cannot belong to two different groups at the same point in time. However, this constraint represents a severe limitation for some application scenarios, for example when groups model associations that are not in-person meetings (e.g., paper co-authorships) or when each point in time of the storyline corresponds to a relatively long time interval (e.g., one year).

In this paper we generalize the classical storyline model by allowing an actor to simultaneously belong to distinct groups. We call this model Storyline with Ubiquitous Actors (SUA); see Fig. 1. Essential to our model is that an actor is represented as a tree rather than a single line. Our contribution is: We propose a visualization paradigm for the SUA model and identify quality metrics for it. We define an algorithmic pipeline for storyline visualizations in the SUA model. We provide a proof-of-concept implementation and apply it to produce visualizations in real-life scenarios.

Related Work. Tanahashi and Ma [Tanahashi2012] present a general framework for generating aesthetically pleasing storyline visualizations. Subsequent papers focus on specific optimization problems like crossing minimization [vanDijk2018, DBLP:conf/gd/GronemannJLM16, DBLP:conf/gd/KostitsynaNP0S15] and wiggle minimization [fn-mwsv-2017]. Padia et al. [DBLP:conf/graphicsinterface/PadiaBH18, DBLP:journals/cg/PadiaBH19] consider storyline visualizations with multiple timelines. Efficient approaches that compute storyline visualizations with hierarchical relationships or with streaming data are described by Liu et al [DBLP:journals/tvcg/LiuWWLL13] and by Tanahashi et al. [DBLP:journals/tvcg/TanahashiHM15], respectively. Qiang and Bingjie [DBLP:conf/cw/QiangB16] present a system that embeds storyline visualizations into a radial layout. For a broader dissertation on storytelling and visualization refer to the survey of Tong et al. [DBLP:journals/information/TongRBWLWLWQLM18]. We remark that our scenario is strongly related to the dynamic sets visualization; see, e.g., [DBLP:journals/cgf/MizunoWTI19, DBLP:journals/ivs/NguyenXWW16], and [DBLP:journals/cgf/AlsallakhMAHMR16] for a survey. In this regard, it is worth mentioning a recent work by Agarwal and Beck [doi:10.1111/cgf.13988], who adopt storylines for visualizing dynamic sets.

2 Storyline Visualizations and Ubiquitous Actors

We first recall basic definitions and principles of classical storyline visualizations and then define our visualization for the SUA model.

Classical storyline visualizations. A storyline consists of a set of actors and a set of groups. Each group is a triple , where is a subset of actors, is the begin-time of and is the end-time of . We say that is active at any time instant in the interval , and that each actor participates to . A common assumption is that an actor cannot participate to two distinct groups at the same point in time, i.e., if and are two distinct groups such that then .

In a storyline visualization, each actor is represented as a line that flows from left to right; see Fig. 1. Some basic principles are considered: For each group , the lines representing the actors in are adjacent, i.e., they run close together from the begin-time to the end-time of ; lines of actors that are not in the same group at the same time are depicted relatively far from one another; a line should not deviate unless it converges or diverges with another line.

Figure 1: Storyline visualization: (a) Classical model. (b) SUA model.

In addition, common quality metrics for the readability of storyline visualizations are: (a) Line or block crossings – a line crossing occurs when two lines intersect while a block crossing is caused by two blocks of parallel lines that pairwise intersect. (b) Line wiggles – line deviations that, when frequent, negatively affect the visual flow of the layout. (c) White space gaps – white areas used to separate lines of actors that do not participate to the same group.

Visualizations with ubiquitous actors. To support the visualization of ubiquitous actors, we represent an actor as a tree rather than as a line (see Section 3 for a formal definition of ). Informally speaking, when an actor simultaneously participates to different groups, the line of the actor branches out and forms a tree. For example, in Fig. 1 we see the trees of five actors . At time actor participates to group while at time it simultaneously participates to groups and . As a consequence, the line of at time is split into two branches. The choice of a tree is motivated by the fact that we want to represent each actor by a connected geometric feature (avoiding discontinuities); at the same time, we want to keep such geometric feature as simple as possible, since the addition of edges may increase the number of crossings. Such tree representations add new quality metrics:

Actor planarity. It is natural to require that each tree representing an actor is not self-intersecting. While this is trivially guaranteed when an actor is a line, it requires an algorithmic effort in the SUA model.

Branch continuity. To avoid interruptions in the continuity of the story, the number of branches of an actor at time that continue at time should be maximized. If an actor participates to groups at time and to groups at time , all branches at time should continue at time .

Branch degree. When an actor tree needs new branches at some time instant , it is desirable that the maximum number of branches that emanates from a common branch at time  is minimized.

We note that such new metrics may be in conflict with classical ones (see Fig. 2).

Figure 2: (a) A layout with optimal branch continuity. (b) Violating branch continuity for at and for at removes crossings and reduces line wiggles. (c) A layout with optimal branch degree. (d) Violating branch degree for at reduces line wiggles and removes crossings.

3 The SUA Algorithmic Pipeline

We compute storyline visualizations in the SUA model by means of an algorithmic pipeline based on the concept of actor-tree associated with an actor . The life-time of actor is the interval between the first and the last time instant at which belongs to some group. Tree is defined as follows. Node set – Tree has a root . For each time instant in the life-time of : If participates to at least one group () active at , has a node for each such group; otherwise has a single node that is not associated with any group. Edge set – The parent of a node () is assigned as follows: If is not in the life-time of , the parent of is the root . Else, if and is active before time , the parent of is . Else, the parent of is one of the nodes . Our algorithmic pipeline consists of four steps:

1. Actor-tree Initialization. It defines an initial actor-tree for each actor . Namely, given the nodes of , it assigns the parent to each node of .

2. Branch Permutation. For each time instant this step computes a permutation (i.e., a vertical order) of all nodes at time in the union of all actor-trees .

3. Actor-tree Untangling. For each actor-tree , it redefines the parent of some nodes, so to reduce self-intersections of without changing its node degrees.

4. Branch-coordinate Assignment. It assigns the -coordinates to actor-tree nodes.

In Step 1 we aim to optimize branch continuity and branch degree. Step 2 aims to minimize block or line crossings. Step 3 tries to enforce actor planarity. Step 4 aims to reduce line wiggles and space gaps. Different algorithmic strategies are applicable to each step. We briefly describe our solution; see also Fig. 4.

Actor-tree Initialization. For any actor-tree , let and be the sets of nodes at time instant and . The parents of the nodes in are chosen among the nodes in so that the distribution of the degrees in is as uniform as possible. For each node in that belongs to the same group of a node in , the parent of is . For the remaining nodes of , we adopt a round robin policy to assign children to the nodes in so that the difference of the degrees of any two nodes of is at most one.

Branch Permutation. We exploit a state-of-the-art algorithm for classical storyline visualizations, namely the algorithm by van Dijk et al. [vanDijk2018] based on a SAT formulation, which optimally solves the problem of minimizing block crossings. To this aim, we transform the output of Step 1 into an instance for a classical storyline visualization: Each actor tree is partitioned into a set of edge-disjoint paths by duplicating each node with children into nodes each having one child (see Fig. 3). Each path is processed by the algorithm in [vanDijk2018] as a distinct actor. All copies of the same node are then recombined into a single node to restore the tree. However, if disjoint paths originating from two copies of the same node are treated independently, they can create many crossings when recombined back into the tree. To alleviate this drawback, we let the initial node of each path belong to the same group of its original duplicate, unless this operation makes the path belonging to multiple groups at some other point in time. Moreover, when copies of the same node are recombined into a single node, two edges incident to this node may create a crossing, which is easily removed as depicted in Fig. 3. Hence, a crossing in an actor tree only involves independent edges.

Figure 3: (a) Transformation of an actor tree into a set of disjoint paths. (b) Crossing removal when merging two copies of the same node. (c) Actor-tree untangling. (d) Preservation of the edge order around a node when splitting it.

Actor-tree Untangling. For each actor-tree , we redefine the parent of some nodes to reduce the number of crossings between the edges of . If two edges and (, ) of cross, we replace them with two new edges and (see Fig. 3). This operation removes at least one self-intersection and does not create any new one. Also, the degree of , , , and does not change. We repeat this procedure until it is no longer possible to remove self-intersections from .

Branch-coordinate Assignment. As in the Branch-permutation step, we consider the set of paths that decompose the tree and make them disjoint by duplicating each node with children into nodes each having one child. The cyclic order of the edges around each node defines the permutation of the lines that correspond to these edges (see Fig. 3). Any technique that assigns coordinates to the paths, while reducing line wiggles and white space gaps can be applied (see, e.g., [fn-mwsv-2017, Tanahashi2012]). This assignment preserves the vertical permutations of the paths.

(a) Actor-tree Initialization: Input
(b) Actor-tree Initialization: Output
(c) Branch Permutation: Disjoint paths
(d) Branch Permutation: Algorithm in [vanDijk2018]
(e) Branch Permutation: Output
(f) Actor-tree Untang. + Branch-coord. Assign.
Figure 4: Illustration of the algorithmic pipeline.

4 Implementation and Case Studies

We developed a prototype web application, StoryTreeViewer, which implements the algorithmic pipeline of Section 3, see StoryTreeViewer offers a simple interactive interface, which we used to evaluate effectiveness and limits of our model through two case studies on publication data extracted from DBLP [dblp] and Scopus [scopus].

(a) Case Study 1
(b) Case Study 2
Figure 5: Visualizations of our case studies. See the appendix for larger images.

Case study 1. The first case study, see Fig. 5(a), describes scientific collaborations among the authors of this work in the various editions of the Graph Drawing Symposium (GD) since 1999. Each actor is an author and a group is a subset of actors who co-authored some papers. is active in if all their members co-authored at least one paper in each year from to . The layout reveals the following dynamic. In the first part of the story there is a strong collaboration between the three oldest actors (pink, green, and blue), in particular they form a group lasting from 2004 to 2009. In 2003, the pink actor was the chair of GD, which prevented him to publish together with the other two authors. In 2010 and 2011 the collaboration of the three actors is weaker, as they mainly collaborated with researchers outside their university. The dynamic becomes more involved in the last years, when two new members joined the group (cyan and orange), and new theoretical and application research topics were activated.

Case study 2. The second case study, see Fig. 5(b), describes scientific collaborations among five of the research teams (universities) with the highest number of papers published at GD. The actors are the teams and the groups are defined as in case study 1. Namely, a group is a subset of teams that appear together in some papers (in terms of author affiliations); is active in if all its teams appear together in at least one paper in each year from  to . The layout shows some interesting facts. From 1999 to 2002 there is a strong collaboration between Roma Tre and Perugia, witnessing that the group in Perugia stems from researchers coming from Rome. The collaboration between the five research teams increases since 2007 and becomes stronger since 2011. This is partly explained by the series of workshops started around 2006 (BWGD, HOMONOLO, GNV, etc.) that increased international collaborations.

Limits. Working on the case studies, we observed some limits of our approach: The implementation for the Branch Permutation step exploits the algorithm in [vanDijk2018], splitting each actor-tree into multiple disjoint paths. The size of this transformed instance raises some computational complexity issues. Our visualizations appear to be readable for relatively few actors and further work is needed to better evaluate the effectiveness of the SUA model on larger instances.

5 Conclusions and Future Work

We introduced the SUA model, which allows ubiquitous actors in storyline visualizations. This model extends the spectrum of applications for this type of representation and opens up to many intriguing research directions. Among them: Are there more effective ways of modeling ubiquitous actors other than using trees? Design and experiment different algorithms for the SUA pipeline.



(a) Case study 1
(b) Case study 2
Figure 6: Visualizations of our case studies.