A persistent homology-based topological loss for CNN-based multi-class segmentation of CMR

07/27/2021 ∙ by Nick Byrne, et al. ∙ King's College London 25

Multi-class segmentation of cardiac magnetic resonance (CMR) images seeks a separation of data into anatomical components with known structure and configuration. The most popular CNN-based methods are optimised using pixel wise loss functions, ignorant of the spatially extended features that characterise anatomy. Therefore, whilst sharing a high spatial overlap with the ground truth, inferred CNN-based segmentations can lack coherence, including spurious connected components, holes and voids. Such results are implausible, violating anticipated anatomical topology. In response, (single-class) persistent homology-based loss functions have been proposed to capture global anatomical features. Our work extends these approaches to the task of multi-class segmentation. Building an enriched topological description of all class labels and class label pairs, our loss functions make predictable and statistically significant improvements in segmentation topology using a CNN-based post-processing framework. We also present (and make available) a highly efficient implementation based on cubical complexes and parallel execution, enabling practical application within high resolution 3D data for the first time. We demonstrate our approach on 2D short axis and 3D whole heart CMR segmentation, advancing a detailed and faithful analysis of performance on two publicly available datasets.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

page 8

page 12

page 16

page 20

page 21

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Medical image segmentation is a prerequisite of many pipelines dedicated to the analysis and visualisation of clinical data. A pervasive example from cardiac magnetic resonance (CMR) divides the 2D short axis image into myocardial, and ventricular components. Associated segmentations permit quantitative assessment of important clinical indices Ruijsink et al. [2019] and the analysis of cardiac physiology Lopez-Perez et al. [2015]. Further to their quantitative application, segmented CMR images find a qualitative role within the visualisation of 3D data, including the planning of structural intervention Valverde et al. [2017]. These successes have been achieved in spite of the operator burden of manual image segmentation, a task that demands specialist expertise and which in 3D, can take hours to complete Byrne et al. [2016].

In response, deep learning, and in particular convolutional neural networks (CNNs), have fostered significant gains

Chen et al. [2020]. Key to their success has been the design of specialised architectures for image segmentation, U-Net Ronneberger et al. [2015] being the prime example. Implicit within this multi-scale architecture is an acknowledgement that image segments are discriminated by pixel, local and global image features, including anatomical morphology. Whilst considerable effort has been devoted to the extraction of multi-scale image features, less attention has been paid to their role in optimisation Duan et al. [2019]. For the most part, CNNs have been trained using pixel-wise loss functions such as cross-entropy (CE) or the Dice similarity coefficient (DSC). Whilst easily implemented and possessing favourable numerical properties, their treatment of pixels as independent from one another renders them insensitive to higher order features of anatomy. This is in contrast with the spatially extended features that we anticipate are learned during training.

Without explicitly considering such features, CNN optimisation can predict segmentations which lack spatial coherence, including spurious connected components or holes Painchaud et al. [2019]. To the operator, such errors appear nonsensical, violating fundamental properties of anatomy. Since they are often small and constrained to object boundaries, their associated segmentation may remain suitable for the assessment of certain clinical indices. However, for a wider array of downstream applications it is crucial to represent such features faithfully Byrne et al. [2016].

Historically, and before the advent of deep learning, state of the art cardiac segmentation methods depended on strong priors describing anatomical configuration, such as atlas-based approaches or those based on deformable or statistical shape models. Ideally, however, such prior information is abstract and adaptable to the variety of cases encountered in the clinic, not dependent on its appearance within exemplar training data. In the simplest case, abstract priors might specify the configuration of the heart’s chambers, valves and associated vasculature. For example, in short axis CMR the right ventricular cavity is bound to the left ventricular myocardium, which in turn surrounds the left ventricular blood pool.

Whether describing normal or pathological anatomy, and in contrast to the pixel-wise losses mentioned, anatomical priors provide a global description of segmentation coherence. However, whilst these constraints are simple to express qualitatively, their quantitative expression is not trivial. Furthermore, the opacity of CNNs has hindered efforts to explicitly utilise such priors in parameter optimisation. Addressing these limitations, we present a topological loss function for multi-class image segmentation, leveraging an explicit anatomical description that is independent of exemplar training data.

2 Previous work

2.1 Anatomical priors within cardiac segmentation

CNNs have emerged as the state of the art reference standard for cardiac image segmentation, achieving notable success within: 2D short axis CMR segmentation Bernard et al. [2018]; and 3D CT/CMR multi-class labelling Zhuang et al. [2019]. We consider closely related tasks in the remainder of this paper (see Figure 1).

Figure 1: (a) Short axis segmentation of 2D data into right () and left () ventricular cavities, and myocardium (). (b) Whole heart segmentation of 3D data into right () and left () atrial classes in addition to those listed prior.

2.1.1 2D short axis segmentation

Short axis segmentation has received the greatest interest of any cardiac task Chen et al. [2020]. In studies of healthy subjects, this has culminated in a level of performance consistent with interobserver variation Bai et al. [2018]. However, in studies of cardiovascular disease, a performance deficit remains. This gap is partially characterised by the type of anatomically implausible error described in section 1.

To address these modes of failure, authors have sought to leverage prior information. One approach has combined CNNs with conventional methods such as atlas-based segmentation Duan et al. [2019], active contours Ngo et al. [2017] or statistical shape models Tóthová et al. [2018]. Other works have attempted to inject prior information into CNN optimisation directly, via a learned, latent representation of anatomically plausible shapes Oktay et al. [2018]. These implicit embeddings, however, make it difficult to understand the extent to which such priors are related to morphology or topology as claimed. Work in Painchaud et al. [2019] bridged this gap, augmenting the latent space by only maintaining anatomically plausible cases.

2.1.2 3D whole heart segmentation

Before the shift to deep learning, whole heart segmentation made use of strong prior knowledge, with statistical shape models Wierzbicki et al. [2008] and in particular atlas-based segmentation Zuluaga et al. [2013] representing the state of the art. In contrast, exponents of CNN-based segmentation have been necessarily preoccupied with solutions for handling large 3D volumes. Work has focused on architectural modifications including cascaded spatial processing, patch-based inference and slice-by-slice or 2.5D segmentation Isensee et al. [2018].

There are exceptions to this trend, however: Wang et al. [2017] incorporated the results of statistical shape modelling as an additional CNN input channel; Xu Xiaowei et al. [2020] used a graphical representation of the great vessels to improve diagnostic classification and segmentation. Works combining multi atlas-based registration and CNNs have also been presented Luo and Zhuang [2020]. Most recently, Wang et al. [2021] employed a latent representation of cardiac anatomy within a few shot learning framework, achieving impressive results.

2.2 Segmentation topology

In other clinical settings, priors specifying the adjacency Ganaye et al. [2018], containment BenTaieb and Hamarneh [2016] and hierarchy He et al. [2019] of anatomical components have been used to build associated loss functions. We observe that across applications BenTaieb and Hamarneh [2016], Ganaye et al. [2018], Painchaud et al. [2019]

, criteria defining anatomical plausibility can frequently be summarised by label map topology. Recently, persistent homology (PH), an emerging tool for topological data analysis, has been combined with machine learning

Hofer et al. [2017], Brüel-Gabrielsson et al. [2019]. Associated topological features have been employed within image classification and segmentation by -means clustering Assaf et al. [2016] and -nearest neighbour Qaiser et al. [2016] search. They have also been leveraged to distill dermoscopic images Vandaele et al. [2020]. Bespoke input layers have been used to feed PH features to CNNs, improving electroencephalogram classification Hofer et al. [2019], and the detection of COVID-19 Hajij et al. [2021].

Pertinently, PH has been employed not only as a source of features for classification, but also to extract a supervisory signal for optimisation; including within binary segmentation tasks. Reference Hu et al. [2019] established a PH-based loss according to the Wasserstein distance, applying their approach to microscopy and within the natural image domain. Other exponents of PH-based losses have optimised segmentation topology against an explicit prior. Works have considered the topologies of the branching murine neurovasculature Haft-Javaherian et al. [2020], the cylindrical small bowel Shin et al. [2020] and the toroidal appearance of the myocardium in short axis CMR Clough et al. [2019, 2020].

3 Contributions

To the best of our knowledge, and outside of our preliminary work Byrne et al. [2021], the application of PH-based loss functions to the task of multi-class segmentation is yet to be explored. In the multi-class setting, we make the following contributions:

  1. A formalism for the construction of PH-based loss functions for optimising CNN-based segmentation topology.

  2. Their application within a post-processing framework, demonstrating significant topological improvements across 2D and 3D CMR segmentation tasks.

  3. An efficient implementation based on cubical complexes and parallel computation, admitting 3D application at full spatial resolution for the first time.

4 Theory

4.1 Betti numbers, homology and cubical complexes

In D, objects with differing topology can be distinguished by the first Betti numbers: in 2D and in 3D. Intuitively, counts the connected components which make up an object, , the number of 2D holes or loops and , the number of 3D holes or voids contained Otter et al. [2017]. Whilst this characterisation is simple to express in abstraction, its practical computation relies upon homology, a branch of algebraic topology concerned with procedures to compute the topological features of objects. The mathematical machinery underlying homology depends on the representation of objects by combinatorial structures called simplicial and cubical complexes. Provided its representation as a valid complex, homology returns the Betti numbers of a binary segmentation by linear algebra alone.

Cubical complexes are well suited to data structured on a rectangular lattice, constructing 3D objects as the combination of points (0-cells), and unit line intervals (1-cells), squares (2-cells) and cubes (3-cells). For image data, this representation is not only more computationally efficient and elegant than its simplicial equivalent (employed in Brüel-Gabrielsson et al. [2019], Byrne et al. [2021], Clough et al. [2019]), but also affords control over the connectivity of pixels. As illustrated by Figure 2, there are two approaches for the construction of cubical complexes from 2D data Garin et al. [2020]. These differ in their treatment of pixels as either -cells or -cells of the resulting complex, representing a 4-connected and an 8-connected object, respectively. Despite this difference, in initial experiments (see section 10) we found an equivalence between topological loss functions when applied in conjunction with either mode of construction. Accordingly, we consider foreground image pixels to be 4-connected (or 6-connected in 3D), and as 0-cells in the remainder of this work.

4.2 Persistent homology (PH)

Figure 2: Construction of a cubical complex from 2D data. Pixel intensities in (a) exceeding an arbitrary threshold of three appear bright in the binary image (b). In (c), these are considered 0-cells, representing a 4-connected foreground, including three components. In (d) these are considered 2-cells, representing an 8-connected foreground component, containing a hole.

In general, PH describes a scheme for the particular application of algebraic topology which exposes the topological features of data (their Betti numbers) at multiple scales Otter et al. [2017]. We use the 2D example of Figure 3 to illustrate the associated persistence barcode, providing a practical explanation as it relates to our application. This considers the probabilistic segmentation

. Critically, rather than considering topology at a single probability threshold

, PH captures the topology of at all possible thresholds. Descending from the maximal threshold , this amounts to computing the homology of a nested sequence of binary segmentations: . The barcode, therefore, is a dynamic characterisation of probabilistic segmentation topology, tracking its evolution against this threshold, .

Figure 3: The PH barcode. Bars are labelled by topological dimension, and ranked in order of descending persistence: .

An exemplar sequence is demonstrated in Figure 3. At each threshold, the number of vertically arranged bars indicates the topological features of . The presentation of each bar reveals the dimensionality of the feature described: solid bars indicate connected components; open bars indicate loops. As such, the Betti numbers of are returned by counting the number of each type of bar vertically intersected.

A bar extends horizontally over the probability interval for which its associated feature is maintained. Critical values of admit changes in the topological features of and are indicated by the endpoints of each bar. Accordingly, the persistence of a topological feature is the horizontal extent of its associated bar. Persistent bars are considered robust to small perturbations, suggesting that they are true topological features of . Hence, in Figure 3, we arrange bars in order of descending persistence after grouping by topological dimension. From the persistence barcode of the probabilistic segmentation , we write the persistence of the most persistent feature of dimension as . Topological persistence is a differentiable quantity that is consistent with gradient-based learning Brüel-Gabrielsson et al. [2019], Hofer et al. [2017].

5 Methods

5.1 Notation

We address the generic multi-class segmentation task, seeking a meaningful division of the D CMR image into meaningful anatomical segments. In the tasks considered, image segments describe the chambers of the heart, their association, and their relationship to the myocardium (see Figure 1). We use the following shorthand: myocardium (), left atrium (), left ventricle (), right atrium () and right ventricle (). We denote the ground truth image segmentation by , being made up by mutually exclusive class label maps: , , … , including foreground classes. We consider the background.

In each task we optimise the parameters, , of a CNN to infer the probabilistic segmentation, , a distribution over per-class segmentation maps: , , … . We write inference as . In all cases, we achieve a discrete prediction as the segmentation which maximises pixel-wise probability: , with mutually exclusive classes: , , … . Given the success of CNN-based solutions, we assume that, at least with respect to spatial overlap,

is a reasonable estimate of

.

Within our formalism, we consider the topology, not only of individual segmentation objects, but also their combination: by and we refer to the Boolean union of classes and ; is the pixel-wise probability of class or . We consider the union of a class with itself to be the segmentation of the single class: , and .

5.2 Multi-class topological priors

The Betti numbers are topological invariants permitting the specification of priors for the description of foreground image segments. Consider our 2D, short axis example (see Figure 1):

(4)
(5)
(6)

Equations (1)-(3) specify that the , and each comprise a single connected component, and that the contain a loop. However, these equations only provide a topological specification in a segment-wise fashion: they fail to capture inter-class relationships. For instance, they make no specification that the surround the or that the and be adjacent. By the inclusion-exclusion principle, the topology of a 2D multi-class segmentation is determined by all foreground objects and all object pairs Bazin et al. [2007]. In 3D, this group includes all possible object triples. Hence, a richer topological description also considers the combined foreground classes set out in Equations (4)-(6).

For convenience, we collect Equations (1)-(6) into a task-specific Betti array . Each element denotes the Betti number of dimension for the ground truth segmentation . Vitally, even in the absence of the ground truth, can be determined by prior knowledge of the anatomy to be segmented.

5.3 Topological loss function

We exploit the differentiable properties of the persistence barcode to construct a loss that exposes the differences between and our prior specification . However, in contrast with previous works Clough et al. [2020], Shin et al. [2020], which considered binary segmentation problems, our loss is informed by the topological features of multiple probabilistic segmentations. Critically, segmentation selection and construction defines the topological features to which our formulation is sensitive. Given the theory of subsection 5.2, we consider the persistence barcode for all foreground class labels and class label pairs (see Figure 4):

(7)

Recalling that is the persistence of the most persistent topological feature of dimension , evaluates the total persistence of the longest -dimensional bars for the probabilistic union of classes and , . Assuming that the inferred segmentation closely approximates the ground truth, measures the presence of anatomically meaningful topological features: the extent to which the topological features anticipated by are present within . evaluates the persistence of features that are superfluous to : it is sensitive to spurious connected components or holes. Summing over all topological dimensions , and considering all classes and class pairs , optimising maximises the persistence of topological features which match the prior specification, and minimises those which do not.

Figure 4: Construction of the loss . Each probabilistic segmentation ( or ), is accompanied by its associated persistence barcode (only bars with are shown).

5.4 CNN-based post-processing framework

We employ our topological loss function in a CNN post-processing framework Byrne et al. [2021], Clough et al. [2020], seeking an improvement in segmentation topology by fine tuning a pre-trained CNN, . This achieves a new set of network parameters , optimised to correct the topology of the specific test case .

However, since topology is a global property, many segmentations potentially minimise . Hence, where is the number of pixels in , limits test time adaptation to the minimal set of modifications necessary to align the segmentation and the topological prior,

. The hyperparameter

controls the influence of the associated similarity constraint:

(8)
(9)

5.5 Computation of the PH barcode

Computation of the persistence barcode is an intensive procedure, introducing significant inefficiencies within CNN training Shin et al. [2020]

. This is particularly problematic in 3D and within our loss formulation, demanding the extraction of multiple barcodes per gradient update. In this work, we have implemented a thin wrapper around the open source “CubicalRipser” library

Kaji et al. [2020]

, integrating its functionality with the automatic differentiation engine of PyTorch. As discussed in

section 7, this brings about dramatic performance improvements when compared with previous works Clough et al. [2020], Byrne et al. [2021] reliant on the “TopologyLayer” library Brüel-Gabrielsson et al. [2019]. Performance further benefits from the parallel computation of persistence, making trivial use of multiprocessing from the Python Standard Library.

6 Experiments

6.1 Experimental methods and baselines

We compare performance with the following baselines: (1) -, The discrete segmentation maximising pixel-wise probability, conventionally inferred by U-Net; and (2) , Connected component analysis: the discrete segmentation composed by the per-class largest components of -.

We test the performance of two flavours of topological post-processing. Both deploy a variant of Equation 9: (1) , post-processing according to per-class topological priors specified for individual foreground labels only; and (2) , post-processing according to multi-class topological priors, specified for all individual and paired foreground labels.

6.2 Metrics and statistical analysis

We use the generalised DSC, , as an objective measure of spatial overlap performance Zhuang et al. [2019], and note the change in spatial overlap induced by post-processing as .

We characterise topological accuracy using two metrics. The Betti error measures the total deviation between inferred and ground truth topology. In 2D, this assessment is made according to the Betti numbers of all individual foreground classes and all pairs of foreground classes:

(10)

In 3D, also considers class triples, . To understand whether improvements in segmentation topology translate into anatomically meaningful segmentations we also make a binary assessment of topological success:

(11)

Spatial overlap results are described by their median and interquartile range. To account for large differences in performance, Betti error is described by the percentile () and percentile ranges indicated. Results for and are compared using Wilcoxon signed rank test with Bonferroni correction. is described by sample proportion (

) and its associated standard deviation (

), and compared using exact binomial test after Bonferroni correction.

6.3 Experiment 1: 2D short axis segmentation

6.3.1 Experimental setting

In this experiment we demonstrate our approach on multi-class, 2D short axis segmentation, using a subset of the publicly available ACDC training dataset Bernard et al. [2018]. Ignoring irregular appearances at apex and base, we extract the three mid ventricular slices from each short axis stack, including diastolic and systolic frames from all 100 patients. This achieves 600 short axis images, all sharing the topological prior given in subsection 5.2

. All image-label pairs were resampled to an isotropic spacing of 1.25 mm and normalised to have zero mean and unit variance

Isensee et al. [2018]. Subjects were randomly divided between training, validation and test sets in the ratio 2:1:1, stratified by ACDC diagnostic classification. We trained a 2D U-Net Ronneberger et al. [2015] using CE loss and the combined training and validation set of 450 images, for 16,000 iterations. We employed the Adam optimiser with a learning rate of . Each minibatch contained ten patches of size 352 by 352, randomly cropped from ten different patients. Data augmentation applied random rotation.

Topological post-processing was performed on the inferred multi-class segmentations of the held-out test set. This sought to minimise for the priors expressed in subsection 5.2, with . Test time adaptation used the Adam optimiser with a learning rate of for 100 iterations. Hyperparameters for both supervised training and topological post-processing were optimised using the validation set.

6.3.2 Results

Spatial overlap performance Topological performance
-
Reporting: ; ; ; .
Table 1: Performance on the test set, held out from the ACDC training data: segmentation of 2D short axis CMR.
(a) (b) (c)
Figure 5: Where CCA is limited to the discrete removal of superfluous regions, topological post-processing rectifies errors by expressive modification. (c) suppression of the anomalous component is accompanied by consolidation of the cavity and completion of the myocardial torus.
(a) (b) (c)
Figure 6: Specification of topological priors for all pairs of foreground classes captures the interaction between classes.
Figure 7: Optimisation of in the task of 2D short axis segmentation for the entire ACDC test set.

Row one of Table 1 demonstrates that U-Net segmentations exhibit topological errors in approximately 15% of cases. Note, that these failings present despite strong spatial overlap performance that is consistent with the state of the art. This suggests that measures of spatial overlap do not reliably predict topological performance.

It is apparent from row two of Table 1 that approximately two fifths of individual Betti errors incurred by U-Net are resolved by rudimentary CCA (). This translates into a significant increase in (), accounting for approximately one half of the cases predicted with incorrect topology by U-Net. Sensitive only to 0D topological features, CCA cannot rectify errors related to the presence of loops. Moreover, as most frequently employed, CCA takes a discrete approach to segmentation post-processing considering connected components of the thresholded segmentation. In contrast, our topological loss functions modify probabilistic segmentation topology via CNN optimisation and are conditioned on the test image in question. This admits topological improvement via expressive modification (see Figure 5).

The superior topological performance of our loss functions is quantified in Table 1. Row three reflects the naive extension of previous work Clough et al. [2019] to the multi-class setting, specifying a single prior per class: . This setup admits improved topological performance compared with CCA, resolving all topological errors in over 97% of cases. This reflects substantial but insignificant improvements in the average (), and (), compared against CCA.

Row four outlines the performance of the full realisation of our approach, specifying topological priors per class and per pairwise combination of classes: . This provides topologically accurate results in almost 99% of test images, failing in just two, and returns the lowest . Compared with the naive specification of per-class topological priors only, pairwise combinations expose a richer description of multi-class segmentation topology (see Figure 6). Compared with CCA, makes significant improvements in () and ().

Figure 7 illustrates the evolution of the different terms of Equation 9, demonstrating that the smooth improvement in is balanced by a small increase in the similarity constraint . The latter ensures that post-processed segmentations cannot deviate dramatically from the initial U-Net prediction and maintains spatial overlap performance. This is borne out in Table 1: on average, all post-processing methods tested achieved marginal improvement in the compared with U-Net. However, any gain made was not substantial when compared with the DSC between manual segmentations of different observers: estimated to be around 0.9 Bai et al. [2018].

6.4 Experiment 2: 3D whole heart segmentation

In this experiment, we apply our methods to the task of segmenting multi-class, whole heart anatomy from isotropic, high spatial resolution image data. We consider a semantic subset of the MM-WHS Challenge task, seeking a segmentation of the CMR volume into , , , and Zhuang et al. [2019]. In the context of topological optimisation, this task is made particularly challenging by the semantics of the publicly available ground truth training data. Primarily concerned with measures of spatial overlap and surface error, these segmentations need not and do not convey clinically meaningful topology. Hence, this application of our method seeks to not only refine, but also to impose meaningful segmentation topology. Our specification reflects: (1) continuity of the atrioventricular junctions; (2) isolation of the left and right heart; and (3) adjacency of the blood pool and epicardium (see section 11).

6.4.1 Experimental setting

We employed all twenty publicly available training cases and performance was assessed against the encrypted test set of forty examples. Prior to experimentation, all data were resampled to an isotropic resolution of 1.00 mm and normalised to have zero mean and unit variance. We elected to crop all volumes tightly around the foreground region of interest. This allows us to maintain high spatial resolution (and sensitivity to topological features associated with thin tissue interfaces) without extreme increases in computational demand.

For our baseline we trained a 3D U-Net Çiçek et al. [2016]

using CE loss for 40000 iterations. We employed the stochastic gradient descent optimiser, using a learning rate of 0.01 and momentum of 0.99. Each minibatch contained two 3D image patches of size 192 by 160 by 160. We made use of intensive data augmentation including rotation about all three spatial axes, scaling and non-rigid deformation. Topological fine tuning was mediated by the Adam optimiser using a learning rate of

for 100 iterations and with . All hyperparameters were established by five fold cross-validation over the training set.

6.4.2 Results

Spatial overlap performance Topological performance
-
Reporting: ; ; ; .
Table 2: Performance on the encrypted MM-WHS test set: multi-class segmentation of high resolution 3D CMR.
Figure 8: In the setting of high topological post-processing corrects segmentation topology by minimal adaptation. (a) U-Net segmentation presents a number of topological errors. (b) CCA successfully removes superfluous connected components, but the anomalous association of the left and right heart remains. (c) successfully corrects all errors. Topological features of interest are indicated in red.
Figure 9: The prediction of spurious connected components with high probability can confound topological post-processing. (a) - segmentation includes significant spatial overlap and topological errors. (b) Naive misidentifies the . (c) is confounded by topologically credible candidates, maintaining an anomalous association with the . Topological features of interest are indicated in red.

Table 2 demonstrates that with respect to spatial overlap, our CCA results are consistent with the leading submissions to the MM-WHS challenge Zhuang et al. [2019]. However, further assessment of topological performance requires careful consideration. Since the training data do not reflect clinically meaningful topology, it might be argued that evaluation against our prior has limited interpretability. To inform this comparison, we examined the effective performance of the ground truth training labels. Having a median of , these examples do not include a single topologically accurate segmentation.

It is clear from these results that the topology of the ground truth training data not only diverges from our clinically relevant specification, but is meaningless. In this context, topological post-processing attempts to impose unseen topology at test time. However, it is not alone in this regard: the same can be said for CCA. Though widely employed as a practical approach to reduce false positives and improve spatial overlap, it must be acknowledged that here, CCA applies a limited, 0D topological prior, that conflicts with the features of training data. Despite this, Table 2 endorses the former rationalisation: coupled with a significant improvement in the average (), CCA modifies segmentation topology, reducing the by just over 85% ().

However, CCA remains insensitive to both multi-class and high dimensional topology. Sensitivity to such features is achieved by our topological losses (see Figure 8). Both and adapt segmentation topology, significantly reducing when compared with CCA ( and , respectively). It is telling, however, that this improvement constitutes a topologically accurate segmentation in only two of forty cases. Coupled with the deleterious effect of topological post-processing on spatial overlap this goes some way to characterise the challenge of this task and the limitations of its CNN-based solution. Degradation in spatial ovelap is significant: for both and .

Primarily, such degradation occurs where - segmentation includes multiple, probabilistically credible candidates that, to some extent, are consistent with the topological prior. Figure 9 demonstrates a case in which - localises the left atrium to three distinct regions of the image. Since each component has a persistence greater than 0.9, separating true from anomalous components is probabilistically equivocal. In this context, results in a segmentation with two left atrium components. Remote from its true location and of significant size, a spurious component inferior and posterior to the apex limits spatial overlap performance.

Such errors are problematic in and of themselves. However, given its combinatorial nature, they can also confound multi-class topology. The prior seeks to isolate from by specifying that their combination comprise two connected components. Implicitly, however, our specification assumes that both structures demonstrate correct topology individually. Provided a probabilistic segmentation that equivocally suggests multiple atrial components (see Figure 9), CNN optimisation can reduce

by reinforcing the communication between atria: a modification at odds with anticipated anatomy.

7 Discussion

7.1 Context

Frequently concerned with the delineation of structured anatomical targets, medical image segmentation often benefits from the incorporation of prior information. Our work adds to a rich body of research concerned with methods to leverage and make best use of such priors. Historically, a motivation for their use has been a desire to constrain predicted segmentations to anatomically credible morphology and multi-class configuration. We interpret these ambitions through the lens of topology, a property that, whilst long acknowledged as critical to anatomical plausibility, has rarely been considered explicitly. Instead, topology has more often been captured implicitly, within the examples comprising an atlas or statistical shape model, for example. In contrast, our work exploits PH, an increasingly popular tool from topological data analysis, exposing the topological features of image data. Accordingly, topology provides both the motivation for, and unlike the majority of previous work, the mathematical basis of our methodology and assessment.

Most recently, given the CNN-based state of the art, authors have considered means to inject prior information into parameter optimisation. Popularised by Oktay et al. [2018]

, anatomically constrained neural networks are optimised against a compact, latent representation of anatomically plausible segmentations. However, this approach assumes that the properties which characterise anatomy (morphology, topology etc.) can be implicitly encoded. Since then, it has been shown that CNNs optimised against such a latent representation can still make predictions with anatomically implausible features

Painchaud et al. [2019].

Compared with this family of methods, our topological loss functions permit optimisation against an explicit topological prior. This is beneficial to interpretability, making no assumption as to the faithful representation of anatomically relevant features within a learned representation. However, perhaps its greatest strength is drawn from the abstract quality of the prior information employed. Those methods based on a learned representation are necessarily biased on training data. By contrast, topological priors are abstracted from the expert’s knowledge of segmentation targets, rather than their appearance within exemplar data. This enhances the generalisability of our approach, extending its application to the low data setting. Consider the twenty training cases available in subsection 6.4: it is unlikely that an effective latent representation of highly variable, 3D anatomy could be established from such a sample. Moreover, by decoupling prior information from data, our approach is less susceptible to performance losses in the presence of out of sample test cases, including pathological structural variation.

Finally, we stress that topology is just one component of anatomical plausibility. In other respects, such as morphology, our approach is complementary to those based on a learned representation of anatomical segmentations.

7.2 Computational performance

Compared with our previous work Byrne et al. [2021], adopting the CubicalRipser algorithm Kaji et al. [2020] and integrating its functionality with PyTorch conferred significant computational gains. Basing our previous implementation on Brüel-Gabrielsson et al. [2019], post-processing a single, short axis slice required over six minutes Byrne et al. [2021]. Here, topological refinement had a mean execution time of only 7.12 s.

In 3D, these gains are not only convenient, but arguably enable practicable scientific investigation. By our previous implementation, computing the barcode for a single MM-WHS volume took over an hour. Given that our formulation demands the computation of multiple barcodes at each iterative step, such execution times are impractical. By contrast, in this work, 3D topological post-processing (including all iterations) required a mean of 15.7 minutes. Practical post-processing times for 3D data permit the specification of topological priors truly related to anatomy rather than its appearance in 2D cross-section. This averts the technical challenge of anticipating the slice-wise topological changes associated with tomographic reconstruction and makes our formulation generalisable to many multi-class segmentation tasks.

7.3 Limitations

We have demonstrated our approach within two CMR segmentation tasks, exhibiting varying degrees of success. Whilst the proposed loss functions reliably improve segmentation topology, they can also be associated with degradation in other metrics of performance when compared against U-Net prediction. Prior to topological fine tuning, CNN pre-training establishes an optimal set of parameters with respect to spatial overlap. Hence by introducing topological priors, some compromise is to be expected. Our experiments demonstrate that the extent to which this trade off is felt depends strongly on the task considered. In 2D short axis segmentation, topological post-processing coincided with a small increase in spatial overlap performance. However, in 3D, topological improvement was associated with a statistically significant drop in . Aside from the limitations of our approach, this result indicates the inability of pixel-wise loss functions to promote topological feature learning.

Figure 10: The relationship between topological performance and spatial overlap, prior to post-processing. For insepction, metrics are normalised across their respective test sets.

We assert that this difference is strongly associated with pre-trained U-Net performance. A rich body of research indicates the successes of CNN-based short axis segmentation Chen et al. [2020]. Compared with the majority of 3D applications, this task benefits from numerous training data, reduced structural variability and often a surplus of computational resource in relation to adequate model capacity. This culminates in vastly improved CNN-based segmentation of the 2D short axis image compared with the 3D whole-heart. Qualitatively, we suggest that the former task is more closely aligned with our contextual assumption for the application of topological post-processing: that pre-trained CNN segmentation closely approximates the ground truth aside from a small number of topological errors of limited spatial extent. Quantitatively, we speculate that the applicability of our approach is related to the extent to which pre-trained CNN topological error is explained by spatial overlap performance. As illustrated in Figure 10, Spearman’s Rho suggests that in the 3D task, there is greater association between U-Net spatial overlap and topological performance (-0.364 versus -0.228). This suggests that the topological errors presented by U-Net are more likely to be associated with significant deficits in spatial overlap. Equivocation between true and spurious topological features results. Validation of this assessment across a range of tasks will be necessary to fully understand the generalisability of our approach.

Finally, whilst we remain concerned by degradation in , we stress the importance of anatomically meaningful segmentation to an array of downstream tasks, including surgical planning Valverde et al. [2017]. For these purposes, we think it important that diverse aspects of performance, including topology, are represented and considered alongside spatial overlap.

7.4 Future work

Our formulation presents a general framework that is applicable to any multi-class image segmentation task. In particular, we are keen to explore its application to problems in which anatomical topology varies from case to case, possibly indicating clinically relevant structural disease. A patient-specific topological prior lends itself to the treatment of congenitally malformed cardiac anatomy, for example.

More generally, we speculate that topological loss functions may enhance a range of CNN training paradigms. Given our efficient implementation, our approach could be incorporated into conventional CNN optimisation. Moreover, the abstract and explicit priors on which such losses are based could plausibly provide a supervisory signal for weakly or semi-supervised learning.

8 Conclusion

We have extended PH-based loss functions to multi-class image segmentation. In the context of state of the art spatial overlap performance, our novel approach made statistically significant and predictable improvements in label map topology within 2D and 3D tasks. Our approach is theoretically founded, building a multi-class prior as the collection of individual and paired label map topologies. Compared with the naive consideration of singular labels (the natural extension of previous work), the superiority of this scheme is borne out experimentally. Crucially, we adopted a highly efficient algorithmic backbone for the computation of cubical PH, achieving dramatic improvements in execution time and permitting practicable extension to the 3D setting. A careful analysis of both quantitative and qualitative performance allowed us to reflect the limitations of our approach and consider its wider application. Whilst demonstrated in the field of CMR image analysis, our formulation is generalisable to any multi-class segmentation task: we envisage many applications across a diverse range of anatomical and pathological targets.

9 Code availability

Our wrapper harmonising CubicalRipser with PyTorch automatic differentiation will be made public at the point at which this work is published. For the time being, please contact Nick Byrne with any queries relating to implementation.

10 Appendix A: Performance by complex construction

Spatial overlap performance Topological performance
Reporting: ; ; ; .
Table 3: Performance on the test set, held out from the ACDC training data: segmentation of 2D short axis CMR.

Table 3 extends Experiment 1: segmentation of 2D short axis data to consider modes of cubical complex construction. We use superscripts to differentiate: and , each reflecting topological post-processing in conjunction with cubical complexes in which pixels are considered 0- and 2-cells, respectively. We refer to these as the 0- and 2-construction. For each, we consider topological performance as measured by the Betti Error () and Topological Success () metrics, with the appropriate pixel connectivity relation (see section IV in the main text).

In the context of post-processing according to individual class labels only (), rows one and two compare the performance of each mode of complex construction. Whilst these suggest no difference in the median (1.000 in both cases), marginal improvement in (0.980 - 0.973) is associated with the 2-construction. This pattern is largely replicated in rows three and four, which reflect the application of topological priors for class labels and their pairwise combination (). In these rows, the 2-construction is associated with small improvement in (0.000 - 0.510). This translates into a marginal gain in (0.993 - 0.987).

However, irrespective of the choice of topological prior, there are no statistically significant differences in topological (or spatial overlap) performance between either the 0- or 2-construction. In the absence of any appreciable difference, we employ the 0-construction throughout this work (the default within the CubicalRipser package), and the account provided in the main text. We anticipate that broadly, our findings were not affected by this choice.

11 Appendix B: Topological priors for 3D whole heart segmentation

The topological prior relevant to segmentation of data from the MM-WHS Challenge is as follows:

(12)

Acknowledgments

This work was supported by the EPSRC programme Grant ‘SmartHeart’ (EP/P001009/1), the Wellcome Trust IEH Award [102431], the Wellcome EPSRC Centre for Medical Engineering at School of Biomedical Engineering and Imaging Sciences, King’s College London (WT 203148/Z/16/Z) and by the National Institute for Health Research (NIHR) Biomedical Research Centre at Guys and St Thomas NHS Foundation Trust and King’s College London.

Nick Byrne is funded by a National Institute for Health Research (NIHR), Doctoral Research Fellowship for this research project. This report presents independent research funded by the NIHR. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

The authors would like to thank Shizuo Kaji for his development of, and support in relation to, the excellent CubicalRipser software package. The authors would like to thank Pablo Lamata for providing access to computational resources. The authors have no conflicts of interest to disclose.

References

  • Ruijsink et al. [2019] Bram Ruijsink, Esther Puyol-Antón, Ilkay Oksuz, Matthew Sinclair, Wenjia Bai, Julia A. Schnabel, Reza Razavi, and Andrew P. King. Fully Automated, Quality-Controlled Cardiac Analysis From CMR. JACC Cardiovasc. Imaging, 2019. ISSN 1936878X. doi: 10.1016/j.jcmg.2019.05.030. URL http://10.0.3.248/j.jcmg.2019.05.030https://dx.doi.org/10.1016/j.jcmg.2019.05.030.
  • Lopez-Perez et al. [2015] Alejandro Lopez-Perez, Rafael Sebastian, and Jose M Ferrero. Three-dimensional cardiac computational modelling: methods, features and applications. Biomed. Eng. Online, 14(1):35, 2015. ISSN 1475-925X. doi: 10.1186/s12938-015-0033-5. URL https://doi.org/10.1186/s12938-015-0033-5.
  • Valverde et al. [2017] Israel Valverde, Gorka Gomez-Ciriza, Tarique Hussain, Cristina Suarez-Mejias, Maria N. Velasco-Forte, Nicholas Byrne, Antonio Ordoñez, Antonio Gonzalez-Calle, David Anderson, Mark G. Hazekamp, Arno A.W. Roest, Jose Rivas-Gonzalez, Sergio Uribe, Issam El-Rassi, John Simpson, Owen Miller, Enrique Ruiz, Ignacio Zabala, Ana Mendez, Begoña Manso, Pastora Gallego, Freddy Prada, Massimiliano Cantinotti, Lamia Ait-Ali, Carlos Merino, Andrew Parry, Nancy Poirier, Gerald Greil, Reza Razavi, Tomas Gomez-Cia, and Amir Reza Hosseinpour. Three-dimensional printed models for surgical planning of complex congenital heart defects: An international multicentre study. Eur. J. Cardio-thoracic Surg., 52(6):1139–1148, 2017. ISSN 1873734X. doi: 10.1093/EJCTS/EZX208. URL http://10.0.4.69/ejcts/ezx208https://dx.doi.org/10.1093/ejcts/ezx208.
  • Byrne et al. [2016] N Byrne, M Velasco Forte, A Tandon, I Valverde, and T Hussain. A systematic review of image segmentation methodology, used in the additive manufacture of patient-specific 3D printed models of the cardiovascular system. JRSM Cardiovasc. Dis., 5(0):204800401664546, 2016. ISSN 2048-0040. doi: 10.1177/2048004016645467. URL http://10.0.4.153/2048004016645467https://dx.doi.org/10.1177/2048004016645467.
  • Chen et al. [2020] Chen Chen, Chen Qin, Huaqi Qiu, Giacomo Tarroni, Jinming Duan, Wenjia Bai, and Daniel Rueckert. Deep Learning for Cardiac Image Segmentation: A Review, 2020. URL https://www.frontiersin.org/article/10.3389/fcvm.2020.00025.
  • Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M Wells, and Alejandro F Frangi, editors, Med. Image Comput. Comput. Interv. – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4.
  • Duan et al. [2019] Jinming Duan, Ghalib Bello, Jo Schlemper, Wenjia Bai, Timothy J.W. Dawes, Carlo Biffi, Antonio de Marvao, Georgia Doumoud, Declan P. O’Regan, and Daniel Rueckert. Automatic 3D Bi-Ventricular Segmentation of Cardiac Images by a Shape-Refined Multi- Task Deep Learning Approach. IEEE Trans. Med. Imaging, 38(9):2151–2164, 2019. ISSN 1558254X. doi: 10.1109/TMI.2019.2894322. URL http://10.0.4.85/tmi.2019.2894322https://dx.doi.org/10.1109/TMI.2019.2894322.
  • Painchaud et al. [2019] Nathan Painchaud, Youssef Skandarani, Thierry Judge, Olivier Bernard, Alain Lalande, and Pierre-Marc Jodoin. Cardiac MRI Segmentation with Strong Anatomical Guarantees. In Dinggang Shen, Tianming Liu, Terry M Peters, Lawrence H Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan, editors, Med. Image Comput. Comput. Assist. Interv. – MICCAI 2019, pages 632–640, Cham, 2019. Springer International Publishing. ISBN 978-3-030-32245-8.
  • Bernard et al. [2018] Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng-Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, and Miguel Angel Gonzalez Ballester. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging, 37(11):2514–2525, 2018. ISSN 0278-0062.
  • Zhuang et al. [2019] Xiahai Zhuang, Lei Li, Christian Payer, Darko Štern, Martin Urschler, Mattias P Heinrich, Julien Oster, Chunliang Wang, Örjan Smedby, Cheng Bian, Xin Yang, Pheng-Ann Heng, Aliasghar Mortazi, Ulas Bagci, Guanyu Yang, Chenchen Sun, Gaetan Galisot, Jean-Yves Ramel, Thierry Brouard, Qianqian Tong, Weixin Si, Xiangyun Liao, Guodong Zeng, Zenglin Shi, Guoyan Zheng, Chengjia Wang, Tom Macgillivray, David Newby, Kawal Rhode, Sebastien Ourselin, Raad Mohiaddin, Jennifer Keegan, David Firmin, and Guang Yang. Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge. Med. Image Anal., 58:101537, 2019. ISSN 1361-8415. doi: 10.1016/j.media.2019.101537. URL http://10.0.3.248/j.media.2019.101537https://dx.doi.org/10.1016/j.media.2019.101537.
  • Bai et al. [2018] Wenjia Bai, Matthew Sinclair, Giacomo Tarroni, Ozan Oktay, Martin Rajchl, Ghislain Vaillant, Aaron M. Lee, Nay Aung, Elena Lukaschuk, Mihir M. Sanghvi, Filip Zemrak, Kenneth Fung, Jose Miguel Paiva, Valentina Carapella, Young Jin Kim, Hideaki Suzuki, Bernhard Kainz, Paul M. Matthews, Steffen E. Petersen, Stefan K. Piechnik, Stefan Neubauer, Ben Glocker, and Daniel Rueckert. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson., 20(1), 2018. ISSN 1532429X. doi: 10.1186/s12968-018-0471-x. URL http://10.0.4.162/s12968-018-0471-xhttps://dx.doi.org/10.1186/s12968-018-0471-x.
  • Ngo et al. [2017] Tuan Anh Ngo, Zhi Lu, and Gustavo Carneiro. Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance. Med. Image Anal., 35:159–171, 2017. ISSN 13618423. doi: 10.1016/j.media.2016.05.009. URL http://10.0.3.248/j.media.2016.05.009https://dx.doi.org/10.1016/j.media.2016.05.009.
  • Tóthová et al. [2018] Katarína Tóthová, Sarah Parisot, Matthew C H Lee, Esther Puyol-Antón, Lisa M Koch, Andrew P King, Ender Konukoglu, and Marc Pollefeys. Uncertainty Quantification in CNN-Based Surface Prediction Using Shape Priors. In Shape Med. Imaging, pages 300–310. Springer International Publishing, 2018. ISBN 0302-9743. doi: 10.1007/978-3-030-04747-4_28. URL http://10.0.3.239/978-3-030-04747-4{_}28https://dx.doi.org/10.1007/978-3-030-04747-4{_}28.
  • Oktay et al. [2018] Ozan Oktay, Enzo Ferrante, Konstantinos Kamnitsas, Mattias Heinrich, Wenjia Bai, Jose Caballero, Stuart A Cook, Antonio De Marvao, Timothy Dawes, Declan P O’Regan, Bernhard Kainz, Ben Glocker, and Daniel Rueckert. Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation. IEEE Trans. Med. Imaging, 37(2):384–395, 2018. ISSN 0278-0062. doi: 10.1109/tmi.2017.2743464. URL http://10.0.4.85/tmi.2017.2743464https://dx.doi.org/10.1109/TMI.2017.2743464.
  • Wierzbicki et al. [2008] Marcin Wierzbicki, John Moore, Maria Drangova, and Terry Peters. Subject-specific models for image-guided cardiac surgery. Phys. Med. Biol., 53(19):5295, 2008. ISSN 0031-9155.
  • Zuluaga et al. [2013] Maria A Zuluaga, M Jorge Cardoso, Marc Modat, and Sébastien Ourselin. Multi-atlas propagation whole heart segmentation from MRI and CTA using a local normalised correlation coefficient criterion. In Int. Conf. Funct. Imaging Model. Hear., pages 174–181. Springer, 2013.
  • Isensee et al. [2018] Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, and Sebastian Wirkert. nnu-net: Self-adapting framework for u-net-based medical image segmentation. arXiv Prepr. arXiv1809.10486, 2018.
  • Wang et al. [2017] Chunliang Wang, Qian Wang, and Örjan Smedby.

    Automatic Heart and Vessel Segmentation Using Random Forests and a Local Phase Guided Level Set Method.

    In Reconstr. Segmentation, Anal. Med. Images, pages 159–164. Springer International Publishing, 2017. ISBN 0302-9743. doi: 10.1007/978-3-319-52280-7_16. URL http://10.0.3.239/978-3-319-52280-7{_}16https://dx.doi.org/10.1007/978-3-319-52280-7{_}16.
  • Xu Xiaowei et al. [2020] Xu Xiaowei, Tianchen Wang, Zhuang Jian, Yuan Haiyun, Huang Meiping, Cen Jianzheng, Jia Qianjun, Dong Yuhao, and Shi Yiyu. ImageCHD: A 3D Computed Tomography Image Dataset for Classification of Congenital Heart Disease. In Martel Anne L., Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, A. Zuluaga Maria, Kevin Zhou S, Daniel Racoceanu, and Leo Joskowicz, editors, Med. Image Comput. Comput. Assist. Interv. – MICCAI 2020, pages 77–87, Cham, 2020. Springer International Publishing. ISBN 978-3-030-59719-1.
  • Luo and Zhuang [2020] Xinzhe Luo and Xiahai Zhuang. MvMM-RegNet: A New Image Registration Framework Based on Multivariate Mixture Model and Neural Network Estimation. In Anne L Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A Zuluaga, S Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz, editors, Med. Image Comput. Comput. Assist. Interv. – MICCAI 2020, pages 149–159, Cham, 2020. Springer International Publishing. ISBN 978-3-030-59716-0.
  • Wang et al. [2021] W Wang, Q Xia, Z Hu, Z Yan, Z Li, Y Wu, N Huang, Y Gao, D Metaxas, and S Zhang. Few-shot Learning by a Cascaded Framework with Shape-constrained Pseudo Label Assessment for Whole Heart Segmentation. IEEE Trans. Med. Imaging, page 1, 2021. ISSN 1558-254X. doi: 10.1109/TMI.2021.3053008.
  • Ganaye et al. [2018] Pierre-Antoine Ganaye, Michaël Sdika, and Hugues Benoit-Cattin.

    Semi-supervised Learning for Segmentation Under Semantic Constraint.

    In Med. Image Comput. Comput. Assist. Interv. – MICCAI 2018, pages 595–602. Springer International Publishing, 2018. ISBN 0302-9743. doi: 10.1007/978-3-030-00931-1_68. URL http://10.0.3.239/978-3-030-00931-1{_}68https://dx.doi.org/10.1007/978-3-030-00931-1{_}68.
  • BenTaieb and Hamarneh [2016] Aïcha BenTaieb and Ghassan Hamarneh. Topology aware fully convolutional networks for histology gland segmentation. In Int. Conf. Med. Image Comput. Comput. Interv., pages 460–468. Springer, 2016.
  • He et al. [2019] Yufan He, Aaron Carass, Yihao Liu, Bruno M Jedynak, Sharon D Solomon, Shiv Saidha, Peter A Calabresi, and Jerry L Prince. Fully Convolutional Boundary Regression for Retina OCT Segmentation. In Dinggang Shen, Tianming Liu, Terry M Peters, Lawrence H Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan, editors, Med. Image Comput. Comput. Assist. Interv. – MICCAI 2019, pages 120–128, Cham, 2019. Springer International Publishing. ISBN 978-3-030-32239-7.
  • Hofer et al. [2017] Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl. Deep learning with topological signatures. In Adv. Neural Inf. Process. Syst., volume 2017-Decem, pages 1635–1645, 2017.
  • Brüel-Gabrielsson et al. [2019] Rickard Brüel-Gabrielsson, Bradley J Nelson, Anjan Dwaraknath, Primoz Skraba, Leonidas J Guibas, and Gunnar Carlsson. A topology layer for machine learning. arXiv Prepr. arXiv1905.12200, 2019.
  • Assaf et al. [2016] Rabih Assaf, Alban Goupil, Valeriu Vrabie, and Mohammad Kacim. Homology functionality for grayscale image segmentation. J. Informatics Math. Sci., 8(4):281–286, 2016. ISSN 0975-5748.
  • Qaiser et al. [2016] Talha Qaiser, Korsuk Sirinukunwattana, Kazuaki Nakane, Yee-Wah Tsang, David Epstein, and Nasir M Rajpoot. Persistent homology for fast tumor segmentation in whole slide histology images. Procedia Comput. Sci., 90:119–124, 2016. ISSN 1877-0509.
  • Vandaele et al. [2020] Robin Vandaele, Guillaume Adrien Nervo, and Olivier Gevaert. Topological image modification for object detection and topological image processing of skin lesions. Sci. Rep., 10(1):1–15, 2020. ISSN 2045-2322.
  • Hofer et al. [2019] Christoph Hofer, Roland Kwitt, Marc Niethammer, and Mandar Dixit. Connectivity-optimized representation learning via persistent homology. In Int. Conf. Mach. Learn., pages 2751–2760. PMLR, 2019. ISBN 2640-3498.
  • Hajij et al. [2021] Mustafa Hajij, Ghada Zamzmi, and Fawwaz Batayneh. Tda-net: Fusion of persistent homology and deep learning features for covid-19 detection in chest x-ray images. arXiv Prepr. arXiv2101.08398, 2021.
  • Hu et al. [2019] Xiaoling Hu, Fuxin Li, Dimitris Samaras, and Chao Chen. Topology-preserving deep image segmentation. In Adv. Neural Inf. Process. Syst., pages 5658–5669, 2019.
  • Haft-Javaherian et al. [2020] Mohammad Haft-Javaherian, Martin Villiger, Chris B Schaffer, Nozomi Nishimura, Polina Golland, and Brett E Bouma. A Topological Encoding Convolutional Neural Network for Segmentation of 3D Multiphoton Images of Brain Vasculature Using Persistent Homology. In

    Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Work.

    , pages 990–991, 2020.
  • Shin et al. [2020] Seung Yeon Shin, Sungwon Lee, Daniel C Elton, James L Gulley, and Ronald M Summers. Deep Small Bowel Segmentation with Cylindrical Topological Constraints. arXiv Prepr. arXiv2007.08674, 2020.
  • Clough et al. [2019] James R. Clough, Ilkay Oksuz, Nicholas Byrne, Julia A. Schnabel, and Andrew P. King. Explicit Topological Priors for Deep-Learning Based Image Segmentation Using Persistent Homology. In Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), volume 11492 LNCS, pages 16–28. Springer International Publishing, 2019. ISBN 9783030203504. doi: 10.1007/978-3-030-20351-1_2. URL http://10.0.3.239/978-3-030-20351-1{_}2https://dx.doi.org/10.1007/978-3-030-20351-1{_}2.
  • Clough et al. [2020] James Clough, Nicholas Byrne, Ilkay Oksuz, Veronika A Zimmer, Julia A Schnabel, and Andrew King. A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Trans. Pattern Anal. Mach. Intell., 2020. ISSN 0162-8828.
  • Byrne et al. [2021] Nick Byrne, James R Clough, Giovanni Montana, and Andrew P King. A Persistent Homology-Based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI. In Esther Puyol Anton, Mihaela Pop, Maxime Sermesant, Victor Campello, Alain Lalande, Karim Lekadir, Avan Suinesiaputra, Oscar Camara, and Alistair Young, editors, Stat. Atlases Comput. Model. Hear. M&Ms EMIDEC Challenges, pages 3–13, Cham, 2021. Springer International Publishing. ISBN 978-3-030-68107-4.
  • Otter et al. [2017] Nina Otter, Mason A. Porter, Ulrike Tillmann, Peter Grindrod, and Heather A. Harrington. A roadmap for the computation of persistent homology. EPJ Data Sci., 6(1), 2017. ISSN 21931127. doi: 10.1140/epjds/s13688-017-0109-5. URL http://10.0.4.116/epjds/s13688-017-0109-5https://dx.doi.org/10.1140/epjds/s13688-017-0109-5.
  • Garin et al. [2020] Adélie Garin, Teresa Heiss, Kelly Maggs, Bea Bleibe, and Vanessa Robins. Duality in Persistent Homology of Images. arXiv Prepr. arXiv2005.04597, 2020.
  • Bazin et al. [2007] Pierre-Louis Bazin, Lotta Maria Ellingsen, and Dzung L Pham. Digital Homeomorphisms in Deformable Registration. In Nico Karssemeijer and Boudewijn Lelieveldt, editors, Inf. Process. Med. Imaging, pages 211–222, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg. ISBN 978-3-540-73273-0.
  • Kaji et al. [2020] Shizuo Kaji, Takeki Sudo, and Kazushi Ahara. Cubical Ripser: Software for computing persistent homology of image and volume data. arXiv Prepr. arXiv2005.12692, 2020.
  • Çiçek et al. [2016] Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In Med. Image Comput. Comput. Interv. – MICCAI 2016, pages 424–432. Springer International Publishing, 2016. ISBN 0302-9743. doi: 10.1007/978-3-319-46723-8_49. URL http://10.0.3.239/978-3-319-46723-8{_}49https://dx.doi.org/10.1007/978-3-319-46723-8{_}49.