Animated Drag and Drop Interaction for Dynamic Multidimensional Graphs

02/05/2019 ∙ by Benjamin Renoust, et al. ∙ Osaka University 0

In this paper, we propose a new drag and drop interaction technique for graphs. We designed this interaction to support analysis in complex multidimensional and temporal graphs. The drag and drop interaction is enhanced with an intuitive and controllable animation, in support of comparison tasks.



There are no comments yet.


page 1

page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

When facing multidimensional datasets, analysts working on non-trivial tasks systematically needs to jump between facets of their data [4, 5]. When facets could be different representations on the data, they may also encapsulate different aspects of the data, under conditional filters [14]. This brings obviously the necessity to put side by side different views (i.e. different representations of a same dataset), thus using in analysts the best of human’s perception to spot and understand the differences or similarities in pattern [7].

Figure 1: User select a subgraph from a view (a), then drag the selected subgraph to another view (b), then drop the subgraph (c). The same nodes and links will gradually move to the matching location and then be highlighted (d-e). User may go backward or forward in the animation holding a control key and positioning the mouse along an invisible line between the views (f).

It became completely natural to wish to find the same elements appearing in one view within another view. This requires interaction design, such that users may further search individual datum or groups of data. The multiple views with the interaction operating between them became known as linked views [15]. They became a paradigm early on in visualization [4] as well as most standard user interfaces from OS [1] to softwares [3].

The archetype of linked views certainly is represented by the coordinated scatter plot matrix [4], and this type of selectionhighlight coordination (a.k.a linked highlighting) later became standard in [5, 6]. It naturally combines with the decomposition of complex datasets in small multiple views (or small multiples [11]) and linked-highlighting became essential to all sort of manipulations in complex graphs whether from dimensionality [10], or temporality [9], or even combined [8].

2 Temporal Multidimensional Graphs

We consider a general class of graphs that captures a high level of complexity that we will refer to as temporal multidimensional graphs (weights and directions do not matter here). Our graphs are also multidimensional because links and nodes contain attributes. Our graphs are temporal such as nodes and links exist over time. This family of graphs spawns from real world complex datasets, for which comparison tasks are essential, so we put ourselves in a setting of small multiples, each view defined by time or by other properties.

The small multiples approach has demonstrated to be a good strategy for topology-based tasks on dynamic graphs [2]. In contrast, animation performs well for tasks related to the dynamic evolution of a graph such as addition and deletion of nodes and edges [2]. The use of animation is controversial, but animating transitions, especially with the support of interaction, may be useful [12].

3 Coordinating Temporal Multi Graphs

We place our work in a framework, where multiple subgraphs are defined, and each subgraph is coordinated with each other with linked highlighting. One of our tasks is to investigate the evolution of communities formed over time. These communities form around nodes and edges that exist in different time frames, but not necessary exist persistently across all timeframes: nodes and links may appear, disappear, and nodes may even switch community.

Linked highlighting across small multiples brings forward a selection in one view to all other views simultaneously. When users select nodes and links in a view, these appear in all other views. Although very convenient to dispatch a selection to many other views and understand a global trend, it is hard to follow which elements goes where. A task better suited for animated transitions.

3.1 A Drag and Drop Interaction for Graphs

As an alternative, we propose a drag and drop interaction (Fig. 1) which proposes a trade-off between the advantages of animation and those of small multiples with linked highlighting. The principle of this drag and drop interaction is to allow user to physically grab a part of their graph and observe how it arranges in a different view.

This interaction fits into the linked view paradigm [15], as we have the three elements necessary to its execution. 1) A view capable of interaction: the view from which we grab a subgraph. 2) A mechanism to transfer information: the drag that physically transfers a subgraph. 3) A view capable of visually respond: the animation occurring upon release.

More specifically, each view among the small multiples displays one subgraph . Similarly to linked-highlighting, users can select any subgraph in a view using their favorite selection mode. Similarly to a brushing interaction, they can then drag this selection across views to another view containing subgraph , and release the selection in it. Dragging the selection preserves the relative position of the selected elements which are only translated following the mouse position.

Upon release the selection , we check which nodes and edges belong to the target view’s graph , such as and

. Nodes and edges of the selection that do not satisfy this condition are faded out. Nodes and edges of the target view’s graph that do not satisfy this condition are grayed. The position of nodes and edges that satisfy the condition are then gradually interpolated from their release position to their position in the target graph. This interpolation forms the animation.

Following Tversky’s et al. recommendations [12] for control of animation, with the hold of a control key, we additionally provide a preview of animation to track individual elements (Fig. 1(f)). In this mode this animation is executed on mouseover a target view, before the drop is released. The position of animation is then controlled by the mouse position thus the user can move backward or forward in the animation time to track individual elements.

3.2 Implementation

We implemented this interaction on top of d3 and HTML’s drag and drop interaction events. But we used careful design to avoid conflict between all interactions. We want the drag and drop operate on a selection, but selections are usually implemented over a drag and drop interaction too (for example to create and brush a lasso selector).

In order to achieve this, upon selection, we create a new SVG layer on top of the graph view’s canvas, that is invisible to the user and recreate only the selected part of the graph. This SVG captures the drag interaction, and the user interaction actually drags this layer instead. Each other view’s canvas also presents a temporary SVG upper layer which listens to the drop event.

Finally, when the drop is called or the control key is pressed, the subgraph is copied at the drop position in the new layer and then we interpolate with animation to the positions as detailed in the previous section. The mouse position following an invisible line between the two views gives the percentage of progress of the animation. Therefore the user can advance to the full interpolation or go back in its early steps.

Note that color encoding at the end of interpolation is the same of the encoding from the target view. This is designed so users may use compare where the nodes were originated from in the previous view while helping them better track sub-elements of interest during the animation.

4 Discussion and Conclusion

We designed this drag and drop interaction in a highly multidimensional and multi-view context, for which linked was already serving different purposes, and more would have added confusion. The drag and drop interaction we introduce here is designed to support the analysis of dynamic communities. This is why we base our design on fixed choices such as the use of color encoding for categorical variables. Because our community structure is different for each of the small multiples, we cannot use the same encoding across all views. The animation becomes very helpful to match nodes and communities between views.

Although drag and drop is an interaction limited by definition to two views only, it gives a nice “physicality” to the interaction, especially with the mouse control. Animation works well in that case because it is exactly between two views limited to a user-defined point-of-interest. Informal feedbacks from users show promises in helping them engaging with the system.

In this work, we limited ourselves the use of our drag and drop interaction to the comparison task, but we only scratched the surface. The “drop” part actually captures the intent of a user (through the localization of the drop) and we have not exploited it yet. Small multiples are most appreciated from users [13] and when multiple separated views are provided, there is no doubt that drag and drop interactions make a great improvement so users may easily manipulate between data selections and data representations.


  • [1] I. Apple Computer. Macintosh human interface guidelines. Addison-Wesley Professional, 1992.
  • [2] D. Archambault, H. Purchase, and B. Pinaud. Animation, small multiples, and the effect of mental map preservation in dynamic graphs. IEEE TVCG, 17(4):539–552, 2011.
  • [3] D. Auber, D. Archambault, R. Bourqui, M. Delest, J. Dubois, A. Lambert, P. Mary, M. Mathiaut, G. Mélançon, B. Pinaud, B. Renoust, and J. Vallet. Tulip 5. In Encyclopedia of Social Network Analysis and Mining, pp. 1–28. Springer, 2017.
  • [4] R. A. Becker, W. S. Cleveland, and A. R. Wilks. Dynamic graphics for data analysis. Statistical science, pp. 355–383, 1987.
  • [5] N. Elmqvist, P. Dragicevic, and J.-D. Fekete. Rolling the dice: Multidimensional visual exploration using scatterplot matrix navigation. IEEE TVCG, 14(6):1539–1148, 2008.
  • [6] J. Heinrich, J. Stasko, and D. Weiskopf. The parallel coordinates matrix. EuroVis, pp. 37–41, 2012.
  • [7] K. Koffka. Perception: an introduction to the gestalt-theorie. Psychological Bulletin, 19(10):531, 1922.
  • [8] H. Ren, B. Renoust, M.-L. Viaud, G. Melançon, and S. Satoh. Generating “visual clouds” from multiplex networks for TV news archive query visualization. In CBMI, pp. 1–6. IEEE, 2018.
  • [9] B. Renoust, D.-D. Le, and S. Satoh. Visual analytics of political networks from face-tracking of news video. IEEE Transactions on Multimedia, 18(11):2184–2195, 2016.
  • [10] B. Renoust, G. Melançon, and T. Munzner. Detangler: Visual analytics for multiplex networks. In Computer Graphics Forum, vol. 34, pp. 321–330. Wiley Online Library, 2015.
  • [11] E. R. Tufte, N. H. Goeler, and R. Benson. Envisioning information, vol. 126. Graphics press Cheshire, CT, 1990.
  • [12] B. Tversky, J. B. Morrison, and M. Betrancourt. Animation: can it facilitate? International journal of human-computer studies, 57(4):247–262, 2002.
  • [13] S. van den Elzen and J. J. van Wijk. Small multiples, large singles: A new approach for visual data exploration. In Computer Graphics Forum, vol. 32, pp. 191–200. Wiley Online Library, 2013.
  • [14] A. Wilhelm. Linked views for visual exploration. In

    Handbook of data visualization

    , chap. II.8, pp. 199–216. Springer, 2007.
  • [15] G. Wills. Linked data views. In Handbook of data visualization, chap. II.9, pp. 217–241. Springer, 2007.