# Topological Tracking of Connected Components in Image Sequences

Persistent homology provides information about the lifetime of homology classes along a filtration of cell complexes. Persistence barcode is a graphical representation of such information. A filtration might be determined by time in a set of spatiotemporal data, but classical methods for computing persistent homology do not respect the fact that we can not move backwards in time. In this paper, taking as input a time-varying sequence of two-dimensional (2D) binary digital images, we develop an algorithm for encoding, in the so-called spatiotemporal barcode, lifetime of connected components (of either the foreground or background) that are moving in the image sequence over time (this information may not coincide with the one provided by the persistence barcode). This way, given a connected component at a specific time in the sequence, we can track the component backwards in time until the moment it was born, by what we call a spatiotemporal path. The main contribution of this paper with respect to our previous works lies in a new algorithm that computes spatiotemporal paths directly, valid for both foreground and background and developed in a general context, setting the ground for a future extension for tracking higher dimensional topological features in nD binary digital image sequences.

## Authors

• 10 publications
• 3 publications
• 3 publications
• ### Persistent Entropy for Separating Topological Features from Noise in Vietoris-Rips Complexes

Persistent homology studies the evolution of k-dimensional holes along a...
01/18/2017 ∙ by Nieves Atienza, et al. ∙ 0

• ### An efficient algorithm for 1-dimensional (persistent) path homology

This paper focuses on developing an efficient algorithm for analyzing a ...
01/27/2020 ∙ by Tamal K. Dey, et al. ∙ 0

• ### Visual Detection of Structural Changes in Time-Varying Graphs Using Persistent Homology

Topological data analysis is an emerging area in exploratory data analys...
07/20/2017 ∙ by Mustafa Hajij, et al. ∙ 0

• ### A New Run-based Connected Component Labeling for Efficiently Analyzing and Processing Holes

06/16/2020 ∙ by Florian Lemaitre, et al. ∙ 0

• ### Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting

There recently has been a surge of interest in developing a new class of...
05/10/2021 ∙ by Yuzhou Chen, et al. ∙ 0

• ### A Sheaf and Topology Approach to Generating Local Branch Numbers in Digital Images

This paper concerns a theoretical approach that combines topological dat...
11/27/2020 ∙ by Chuan-Shen Hu, et al. ∙ 0

• ### Informative Planning and Online Learning with Sparse Gaussian Processes

A big challenge in environmental monitoring is the spatiotemporal variat...
09/24/2016 ∙ by Kai-Chieh Ma, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Persistent homology ELZ00 ; ZC05 and zigzag persistence Cd10 provide information about lifetime of homology classes along a filtration of cell complexes. Such a filtration might be determined by time in a set of spatiotemporal data. Our general aim is to compute the “spatiotemporal” topological information of such filtration, taking into account that it is not possible to move backwards in time. This is not obvious if we use the known algorithms for computing (zigzag) persistent homology using time as filter function.

In the context of mobile sensor networks, dG06 is devoted to the problem of finding an evasion path that describes a moving intruder avoiding being detected by the sensors. In dG06 , the region covered by sensors at time is encoded using a Rips complex . A single cell complex is computed by stacking the complexes for all . Th. 7 of dG06 proves that there is no evasion path under a “homological” criterion. Using zig-zag persistent homology, an equivalent condition is provided in adams15 . A necessary and sufficient positive cohomological criterion for evasion in a general case is given in bib:GK17 . Finally, in GCK15 , the authors analyze time-varying coverage properties in dynamic sensor networks by means of zigzag persistent homology. In all the mentioned papers, vertices represent sensors and edges are provided whenever two sensors can detect each other but their specific locations are unknown.

We are concerned with the treatment of time-varying sequences of binary digital images and the tracking of homology classes over time inspired by persistent homology methods. We deal with vertices at exact positions in each image and adjacency relations between consecutive images are provided whenever there are cells in homologous positions (that is, the cells are in the same spatial position but at different times). Our general goal is to compute a spatiotemporal barcode storing the evolution of homology classes over time. In this paper, we concentrate our effort in 2D images and track connected components of both the background and the foreground over time. Roughly speaking, a spatiotemporal path will track a connected component over time and a spatiotemporal barcode will encode the evolution of homology classes over time. For example, the spatiotemporal barcode of the sequence of the four 2D binary digital images given in Fig.1(a)(a)-(d) reflects the fact that a connected component is born in the first image and dies in the third one, and another connected component is born in the second image and dies in the fourth one. This paper is an extension of our previous work iwcia2015 in which we also focused on computing a spatiotemporal barcode for a time-varying sequence of 2D binary digital images. There, we used the construction of an algebraic-topological model (AT-model) GR05 to compute spatiotemporal paths. In the present paper, though, we compute the spatiotemporal paths and barcode directly (without computing AT-models).

We also extend the definition of spatiotemporal filtration and spatiotemporal path to any dimension, what facilitates future extensions of our work to compute spatiotemporal -barcodes in any dimension .

Basics of persistent homology and AT-models are given in Section 2. We introduce the problem of computing the “correct” topological information of spatiotemporal data through two simple examples in Section 3. Formal definitions to deal with a temporal sequence of cubical complexes are set in Section 4. Our method to solve the problem is then introduced in Section 5. In Section 6, we extend the definition of spatiotemporal path to any dimension. We conclude in Section 7 and describe possible directions for future work.

## 2 Persistent Homology through AT-models

Consider as the ground ring throughout the paper ( i.e. ). Roughly speaking, a cell complex is a general topological structure by which a space is decomposed into basic elements (cells) of different dimensions that are glued together by their boundaries (see the definition of CW-complex in bib:12 ). The dimension of a cell is denoted by . A cell is a -face of a cell if lies in the boundary of and . The cell complex is built as follows: add a -cell of to together with all its faces if is face of exactly one -cell in .

If the cells in are -dimensional cubes then is a cubical complex. A -dimensional cube (-cube) is a product of elementary intervals . An elementary interval is defined as a unit interval , with or a degenerate interval . The number of non-degenerate intervals in such product is the dimension of the cube. -cubes, -cubes, -cubes and -cubes are vertices, edges, squares and 3D cubes (voxels) respectively. A cube is a face of a given cube if . Given two cubes, all the faces of a cube must also be a cube, as well as the intersection of any two cubes. A cubical complex has dimension if the cubes are all of dimension at most . The barycentric coordinates of a cube will be denoted by .

A -chain is a formal sum of -cells in . Since coefficients are either or , we can think of a -chain as a set of -cells, namely those with coefficients equal to . In set notation, the sum of two -chains is their symmetric difference. The -chains together with the addition operation form a group denoted as . Besides, the set , is denoted by . A set of homomorphisms , is called a chain map and denoted by . Given two -cells and , we say that if belongs to the -chain (in set notation). The boundary map is defined on a -cell as the sum of its -faces. The -chains with zero boundary form a subspace of . The -chains that are the boundary of -chains form a subspace of . The quotient group is the -th homology group of (with coefficients). The rank of , denoted by , is the -th Betti number of . For a deeper introduction of these concepts, see Mun84 ; kaz ; bib:12 .

A filtration of is an increasing sequence of cell complexes: . The partial ordering given by such a filtration can be extended to a total ordering of the cells of : , satisfying that for each , , the faces of lie on the set . The map is defined by . Informally, the -th persistent homology group ELZ00 ; ZC05 can be seen as a collection of -homology classes (representing connected components when , tunnels when , cavities when , …) that are born at or before we go from to and die after we go from to . A persistence -barcode bib:ghrist is a graphical representation of the -th persistent homology groups as a collection of horizontal line segments (bars) in a plane. Axis correspond to the indices of the cells in . For example, if a -homology class is born at time (i.e. when is added) and dies at time (), then a bar with endpoints and is added to the -barcode.

In bib:caip2011 the authors establish a correspondence between the incremental algorithm for computing AT-models GR05 and the one for computing persistent homology ELZ00 . More precisely, an AT-model for a cell complex is a quintuple , where:

• is the cell complex.

• is a set of cells of that describes the homology of , in the sense that it contains a distinct -cell for each -homology class of a basis (see next item), for all . The cells in are called surviving cells. For all , the set of all the surviving -cells together with the addition operation form the group which is isomorphic to .

• is a chain map that maps each -cell in to one representative cycle of the corresponding homology class . By a classical chain contraction property, it is true that is a -cell of is a basis for .

• is a chain map that maps each -cell in to a sum of surviving cells, satisfying that if are two homologous -cycles then .

• is a chain homotopy (see Mun84 ). Intuitively, for a -cell , returns a set of -cells needed to be contracted to “bring” to a surviving -cell contained in .

## 3 Stating the Problem

Our general goal is to compute, for a time-varying sequence of binary digital images, some kind of barcode that represents the evolution of homology classes over time.

Consider as the set of points with integer coordinates in space . An digital binary image is a set , where is the foreground, the background, and is the adjacency relation for the foreground and background, respectively. In this paper we will deal with points with integer coordinates in 2D space , that is, 2D digital binary images (or 2D images, for short), (or , for short), where is the adjacency relation for the foreground and background, respectively. All the 2D images considered here are finite, i.e., , where is a finite domain, so and are both finite.

In order to give some intuition about the problem we want to state, let us consider the simple examples given in Fig. 1, in which two sequences of a few -connected pixels appearing and disappearing over time, are shown.

To encode the spatiotemporal information of the two sequences, we construct two associated complexes (in fact, they are graphs) by replacing each pixel by a vertex and adding an edge between two vertices if:

• The corresponding pixels are -connected (in the same frame).

• The vertices correspond to the same pixel (homologous coordinates) at consecutive frames.

The resulting complexes and are shown in Fig. 2.

Now, to compute persistent homology on these two complexes and

, we should select an appropriate filtration. Since we want to capture the variation of homology classes over time, we first classify the cells of

and into spatial and temporal cells:

• All vertices are spatial (since vertices represent pixels).

• An edge is spatial if its endpoints (vertices) represent pixels of the same frame.

• If an edge is not spatial then it is temporal.

Therefore, we have the following spatial subcomplexes of : , , , . And the following sets of temporal cells: , , , where numbers correspond to the labels of cells shown in Fig 2(a). The filtration is obtained by interleaving the temporal cells after the correspondent spatial subcomplexes. That is, , , , , , and .

The filtration of is denoted with the same set of indices than the filtration of , where numbers now correspond to the labels of the cells shown in Fig 2(b).

If we compute persistent homology of and using the above filtrations, we will obtain, in both cases, that a connected component (-homology class) is born when cell is added and survives until the end. So, in both cases, a bar with endpoints and is added to the persistence -barcode.

However, we can observe that Fig. 1(a)-(d) cannot represent a connected component that is moving from the very beginning until the end while Fig. 1(e)-(h) can. So we wonder if we could modify the persistence -barcode of the first sequence (Fig. 1(a)-(d)) so that it codifies the connected components that can survive along time. The idea is to replace the bar with endpoints and by respective bars from to and from to , what will be formally described in next sections.

## 4 Spatiotemporal filtrations and paths

Going one step further, we introduce here the concept of spatiotemporal filtration for a more general setting of cubical complexes (in any dimension). The definition has been adapted from the construction described in dG06 ; adams15 for a sequence of simplicial complexes.

First, given a sequence of cubical complexes, each one embedded in , we want to construct a cubical complex embedded in , in which consecutive cubical complexes in are stacked along a new (temporal) dimension.

###### Definition 1.

Let be a sequence of cubical complexes embedded in . The stacked cubical complex is the cubical complex embedded in obtained as follows. Initially, . Now, if a -cell with barycentric coordinates belongs to (that is, denotes corresponding cells in homologous positions in and ), for some , , add the -cell to . This way, the barycentric coordinates of are .

See Fig. 3 as a toy example of stacked cubical complex. Since each cell can be identified by its barycentric coordinates , then is spatial if ; and it is temporal otherwise.

###### Definition 2.

Let be a stacked cubical complex for a sequence of (spatial) cubical complexes . Let denote the set of (temporal) cells with faces in both and . The spatiotemporal filtration is given by: ; if and ; and if and .

Once we have formally defined a spatiotemporal filtration for a sequence of cubical complexes in any dimension, we need to define spatiotemporal paths with the aim of setting down the restriction that it is not possible to move backwards in time.

First, recall that a path of edges in a cell complex from a vertex to a vertex is a chain of -cells (or in set notation) such that (1) only has as a face; (2) only has as a face; (3) each two edges and have a common face.

We have the following definition in the context of stacked cubical complexes.

###### Definition 3.

iwcia2015 Let be a stacked cubical complex with spatiotemporal filtration . A spatiotemporal path in is a path such that the number of edges in is less than or equal to , for any , .

Finally, two vertices are said to be spatiotemporally-connected if there is a spatiotemporal path between them.

Notice that, in a spatiotemporal path , there are not two temporal edges in connecting the same consecutive spatial complexes of the sequence, which follows from the idea that it is not possible to move backwards in time.

## 5 Topological Tracking of Connected Components in a 2D Image Sequence

In this section, which is the main section of the paper, we explain how to compute spatiotemporal paths and barcodes for both the background and the foreground of a given sequence of 2D images.

We first need an adequate representation of 2D image sequences that captures the time-varying nature of the sequence.

Let be a finite 2D image. We should deal with foreground and background differently, since pixels in the foreground are -connected while pixels in the background are -connected. Regarding the foreground of , a point can be interpreted as a unit closed square (called pixel) in centered at with edges parallel to the coordinate axes. The set of pixels centered at the points of together with their faces (edges and vertices) constitute a (2D) cubical complex denoted by . A cell in can be identified by its barycentric coordinates .

As for the background of , the cubical complex is computed as follows. Initially, , which corresponds to the set of vertices of . Each two -connected vertices in form a unit edge that is added to . Similarly, if four -connected vertices in form a unit square , then is also added to .

### 5.1 Computing Spatiotemporal Paths and Barcodes

Consider now a sequence of 2D images . Let S= and be the associated cubical complexes for, respectively, the foreground and the background of the 2D images in the sequence. The stacked cubical complexes and (both embedded in ) are computed as in Section 4. See Fig. 3 as an example of both and .

In iwcia2015 , given a spatiotemporal filtration, we designed an algorithm to compute the spatiotemporal barcode encoding lifetime of connected components on the 2D image sequence over time. The algorithm that we presented in that paper was based on the incremental algorithm for computing AT-models given in GR05 . In particular, the chain homotopy operator provides a path connecting each vertex to a distinguished vertex that represents the connected component that belongs to. More specifically, in the algorithm given in iwcia2015 , a path from to a surviving cell (vertex) was computed and, if is not a spatiotemporal path, then it is broken into pieces that are spatiotemporal paths. Regarding the spatiotemporal barcode, a bar was elongated at time if and only if and the connected component that represents the bar is spatiotemporally connected to some of the endpoints of the edge . Otherwise, the bar was not elongated. This differentiates from classical persistence barcodes in which, for example, the bar corresponding to a connected component that appears at time and does not merge to other connected component later, is elongated until the very end.

In this paper, given a spatial connected component (i.e., a connected component in or , ), we want to know in which previous frame it was born and track the connected component evolution along time. For this aim, we compute a spatiotemporal path for any vertex in or directly, i,e, without computing the chain homotopy . Notice that in order to pursue our goal in this paper, we only need the -skeleton (i.e. vertices and edges) of both and .

Now, let us explain how Alg. 1 works 111A naive implementation of the algorithm is available in http://grupo.us.es/cimagroup/. Let be a total ordering of the cells of the spatiotemporal filtration considered. We say that a cell is older than a cell if . Let us suppose that we are in step of the for-loop (line 4 in the algorithm). Then, is a collection of vertices of representing the spatiotemporally-connected components that were born and have survived until step . The map connects each vertex with the oldest vertex spatiotemporally connected to it. Besides, is a spatiotemporal path from to . At step , if is a vertex, then a new connected component is born (with only one vertex, ), so is added to , and is updated. If is an edge and , that means that where and are the endpoints of . Then, is connecting two paths, and , so no new connected component is created or destroyed. In fact, a new -homology class is born. See Fig. 4.a. Finally, if is an edge and , then is added to a set of edges that we will be used later for tracking. Observe that and for some (see Prop. 4). We can suppose that . Let . If then, we can not spatiotemporally connect with . See Fig. 4.b. If , then is spatial and the connected components represented by and are spatiotemporally connected. If and , then, again, the connected components represented by and are spatiotemporally connected. Therefore we can remove or from . We convene to remove the newest one which is (line 14 of the algorithm). See Fig. 4.c and 4.e. Finally, if and , then we cannot remove . Since is a spatiotemporal path, then there only exists an edge in . See Fig. 4.d. Now, we update the spatiotemporal paths of the vertices spatiotemporally connected to if and only if belongs to , to ensure that the updated path is also spatiotemporal (see lines 15-17 of the algorithm). See Fig. 4.c and 4.d. Observe that we do not compute the AT-model for the spatiotemporal filtration since we are only interested in spatiotemporal paths and barcode.

Regarding time complexity of the algorithm, let be the number of cells of dimension and in the stacked cubical complex. Due to the for-loops, the algorithm works in time. Regarding space, a spatiotemporal path must be stored for all the vertices in the filtration; potentially, in the spatiotemporal barcode , there may be as many bars as vertices in the stacked complex; also potentially, any of the edges of the stacked complex could be added to the set , so space complexity is .

###### Proposition 4.

If is a vertex in , then is a vertex in .

###### Proof.

We will prove it by induction on the steps of Alg. 1. Suppose we are in step . Then, by induction is a vertex if is a vertex, for all , . If is a vertex then is defined as , so the statement holds. If is an edge, we eventually update for some vertices , , by which is a vertex, so the statement holds. ∎

###### Proposition 5.

If is a vertex then is a spatiotemporal path.

###### Proof.

We prove the statement by induction. Suppose that at step and for any vertex , , is a spatiotemporal path connecting with another vertex , being . If , or and , then does not change for any vertex of . If , and then, for a vertex satisfying that either is a spatiotemporal path or . So the statement holds. ∎

Now, let us see how to “topologically” track a connected component, once Alg. 1 has been executed. Let be a vertex in for some , . Recall that the spatiotemporal path connects with the oldest vertex in the sequence that is spatiotemporally connected with . In particular, represents the connected component in that belongs to. That is, we can detect the time in which the connected component is created and we can follow it along the sequence. In fact, from the set of edges , we can obtain a directed tree containing all the vertices of that are spatiotemporally connected to other vertices in the sequence. Moreover, the directed paths in ending at a vertex connect the vertex with all the older vertices that are spatiotemporally connected to it. See Fig. 5.

Fig. 6 shows three simple examples of 2D image sequences. The associated spatiotemporal barcodes are computed using Alg. 1. From left to right, the first and second spatiotemporal barcodes have only one long bar, while the third one has two. The longest spatiotemporal paths are pictured in blue. Notice that the classical persistence -barcode would produce only one long bar in all of them.

## 6 Towards the Computation of Spatiotemporal d-Barcodes for nD Image Sequences

In this section, we generalize the concept of spatiotemporal path to higher dimension in order to set the ground for a future extension of Alg 1 for tracking higher dimensional topological features.

First, given a set of cells and a cubical complex , we denote by the subcomplex of obtained by taking all the cells of together with all their faces. Second, since the ground ring considered throughout the paper is , we should restrict ourself to cubical complexes with torsion-free homology groups in order to extend the definitions of spatiotemporal paths and barcodes to . Nevertheless, if the ground ring is , the definitions below are still valid for cubical complexes with non-torsion-free homology groups replacing by .

A “homological” -path in a cubical complex is defined as a subcomplex of formed by a set of edges of together with their faces such that

• 222Recall that denotes the disjoint union of the sets and . for some subcomplexes and of ,

• ,

• for .

###### Definition 6.

A spatiotemporal -path in a spatiotemporal filtration for a given sequence of cubical complexes is a homological -path in such that and for and for all , .

###### Proposition 7.

Def. 3 and Def. 6 are equivalent.

###### Proof.

Def. 6 Def. 3: is formed by a set of edges and vertices. Since , then has only one connected component. Since for , then does not contain any holes. So is a tree and is a set of vertices. Since then both and can contain only one vertex. Therefore, is a path. Besides, since , then the number of edges in is less than or equal to , for any (since edges in are disjoint).
Def. 3 Def. 6: Since is a path with no loops then , for and is formed by two vertices and . Then } and . Besides, since the number of edges in is less than or equal to , for any , then and for and for all , , what concludes the proof. ∎

The above definition has an easy generalization to any dimension. A “homological” -path in a cubical complex is defined as a subcomplex of formed by a set of -cells of together with their faces such that

• for some subcomplexes and of ,

• for ,

• for .

###### Definition 8.

A spatiotemporal -path in a spatiotemporal filtration for a sequence of cubical complexes is a homological -path in such that and for and for all , .

## 7 Conclusions and Future Work

In this paper, we have computed a modified persistence barcode, named spatiotemporal barcode, of a temporal sequence of 2D images reflecting the time nature of the data. The computation can be made both for the foreground and background of the given images, what enables the tracking of -holes of the foreground as (bounded) connected components of the background. We have simplified the algorithm presented in iwcia2015 for computing spatiotemporal paths, avoiding the computation of AT-models. Although we have presented our algorithm for computing spatiotemporal paths only for 2D image sequences, it can be extended to sequences of images of any dimension, once a spatiotemporal filtration is constructed for the given sequence. We have also extended the notion of spatiotemporal paths to any dimension. This is part of an ongoing project to define and compute spatiotemporal -barcodes (for any ) for sequences of images.

## Acknowledgments

This research has been partially supported by MINECO, FEDER/UE under grant MTM2015-67072-P.

We would like to thank the reviewers for valuable suggestions and comments.

## References

• (1) Adams, H., Carlsson, G.: Evasion paths in mobile sensor networks. I. J. Robotic Res. 34(1), 90–104 (2015)
• (2) Carlsson, G.E., de Silva, V.: Zigzag persistence. Foundations of Computational Mathematics 10(4), 367–405 (2010)
• (3) Edelsbrunner, H., Letscher, D., Zomorodian, A.: Topological persistence and simplification. FOCS2000, IEEE Computer Society, 454–463 (2000)
• (4) de Silva, V., Ghrist, R.: Coordinate-free coverage in sensor networks with controlled boundaries via homology. I. J. Robotic Res. 25(12), 1205–1222 (2006)
• (5) Edelsbrunner, H., Harer, J.: Computational Topology – An Introduction. American Mathematical Society (2010)
• (6) Gamble, J., Chintakunta, H., Krim, H.: Coordinate-free quantification of coverage in dynamic sensor networks. Signal Processing 114, 1–18 (2015)
• (7) Ghrist, R.: Barcodes: The persistent topology of data. Bulletin of the American Mathematical Society 45, 61–75 (2008)
• (8) Ghrist, R., Krishnan, S.: Positive Alexander Duality for Pursuit and Evasion. SIAM J. Appl. Algebra Geometry, vol. 1, pp. 308–327 (2017)
• (9) Gonzalez-Diaz, R., Ion, A., Jimenez, M.J., Poyatos, R.: Incremental-decremental algorithm for computing AT-models and persistent homology. In: CAIP2011, LNCS, vol. 6854, pp. 286–293 (2011)
• (10) Gonzalez-Diaz, R., Real, P.: On the cohomology of 3D digital images. Discrete Applied Math 147 (2-3), 245–263 (2005)
• (11) Gonzalez-Diaz R., Jimenez M.J., Medrano B.: Spatiotemporal barcodes for image sequence analysis. In: IWCIA, LNCS, vol. 9448, pp. 61–70 (2015)
• (12) Hatcher, A.: Algebraic Topology. Cambridge University Press (2002)
• (13) Kaczynski T., Mischaikow K., Mrozek M.: Computational homology, Appl. Math. Sci., vol. 157, Springer-Verlag (2004)
• (14) Munkres, J.: Elements of Algebraic Topology. Addison-Wesley Co. (1984)
• (15) Zomorodian, A., Carlsson, G.: Computing persistent homology. Discrete and Computational Geometry 33 (2), 249–274 (2005)