Persistence Lenses: Segmentation, Simplification, Vectorization, Scale Space and Fractal Analysis of Images

04/25/2016 ∙ by Martin Brooks, et al. ∙ 0

A persistence lens is a hierarchy of disjoint upper and lower level sets of a continuous luminance image's Reeb graph. The boundary components of a persistence lens's interior components are Jordan curves that serve as a hierarchical segmentation of the image, and may be rendered as vector graphics. A persistence lens determines a varilet basis for the luminance image, in which image simplification is a realized by subspace projection. Image scale space, and image fractal analysis, result from applying a scale measure to each basis function.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

page 7

page 8

page 9

page 10

page 11

page 15

page 16

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Variational and morphological image processing [43, 66, 27, 32] employ diverse types of image segmentation for subsequent piecewise approximation by smooth or constant functions. The varilet transform’s lens parameter [10] plays this same role: a lens comprises a hierarchy of nested facets, each bounded in the image plane by one or more Jordan level sets, thereby defining a multiresolution segmentation.

A persistence lens uses the the level sets of the critical points of persistence birth-death pairs [22]. Thus, regions of lesser contrast are contained in regions of greater contrast; the nesting structure is a contrast-invariant topological representation of contrast variation with the image.

When augmented with a local geometric, topological or image measure, a persistence lens’s segmentation hierarchy becomes a scale space, which for natural images may exhibit power law distributions over large regions.

Varilet transforms apply to a continuous interpolation of the image’s luminance channel, where the scalar pixel values are considered as a grid of samples on the image plane continuum. Subsequent analysis remains in the continuous domain, using vector graphics to realize image segmentation and simplification.

2 Example

This section provides an annotated example, intended to create motivation and perspective for the technical sections that follow.

Figure 1 is the original image from which the subsequent images are derived. Figure 2 is the luminance image, which defines the continuous interpolation acted upon by varilet transforms. The figures represent the images in png format, at their native size for 72 dpi.

We will use a persistence lens generated by the algorithm described in section 3.6.4. The lens has 29,298 lens regions, for a total of 43,572 facets; the first level of the hierarchy has 6,308 facets; the hierarchy has maximum lens region nesting depth 19.

Figure 3 visualizes the lens by its segmentation of the continuous image, with each each lens facet filled gray; figure 4 shows each facet filled with color. Figure 5 is close-up of a portion of figure 4; figure 6 is a further magnification. Figure 7 shows all facets at the first level of the lens hierarchy, figure 8 shows their boundary components, and figure 9 highlights a single facet.

The images shown in figures 3 - 9 are vector graphics; however, the figures represent the vector images in png format with sufficient resolution for modest enlargement.

Figure 10 indicates image regions having fractal structure, by identification of a power law distribution of their facets’ contrast. In section 3.8 we discuss other measures that may also generate fractal structure, including facet area and topological total variation.

Figures 11 shows three images from an image scale space based on topological total variation.

Figure 1: Original image, a 481x321 8-bit RGB jpeg, number 385028 from the Berkeley Segmentation Database [39].
Figure 2: Luminance image derived from the original. The varilet transform acts on scalar fields, thus the luminance image will be interpolated to a continuous function. The luminance values are treated as samples of a continuous function on the underlying plane continuum; the samples are located at the corners of a 480 x 320 grid of squares. Interpolation of each square patch is bilinear, except when bilinear interpolation creates a saddle point in the interior of the patch; in this case the square patch is replaced with four triangular patches having a common vertex at the saddle point location, so that every patch is topologically monotone.
Figure 3:

Persistence lens visualized by its segmentation of the continuous image domain. Each of the 43,572 lens facets is filled with a single shade of gray, the luminance of the facet’s boundary components. Each of the lens’s facets is a connected open set; when it has more than one boundary component then the facet has holes. Each boundary component is a Jordan (simple closed) curve upon which the luminance is constant, with all components having this same value. As a vector image, each facet is represented as an SVG path element comprising of one or more closed polylines; they are correctly filled by the SVG even-odd fill rule.

Figure 4: Each lens facet filled with a single color, selected from the original image colors. As does figure 3, this figure has unfilled regions, seen as the checkered background, which we now describe: The root of the lens hierarchy is the entire image; the unfilled regions of the image are the portion of the root not contained in any other facet.
Figure 5: Low magnification close-up of a portion of figure 4, showing lens facet nesting.
Figure 6: High magnification close-up of a portion of figure 5, showing the level of detail arising from continuous interpolation of the image.
Figure 7: Filled facets for the first level of the lens’ hierarchy. These 6,308 facets are pairwise disjoint.
Figure 8: Close-up of lens facet boundaries for the first level of the lens’ hierarchy. We have not drawn the boundary components with the hyperbolic segments that result from bilinear interpolation; instead we have drawn straight line segments.
Figure 9: Close-up of of a single facet at the first level of the lens’ hierarchy, shown with pink outline.
Figure 10: First level facets of figure 7

, overlaid with brightly colored regions each having a distinct power law distribution of its sub-facets’ contrasts. The power law exponents typically lie in the range 2-4, estimated by the maximum likelihood method of Clauset

[19].
Figure 11: Segmentation of lenses of successively coarser scale space images, using topological total variation as scale measure.

3 Theory

We provide an overview of varilet analysis for image processing.

Varilet analysis applies to real-valued continuous functions on compact metric space . For image analysis, is the image plane and is an interpolation of the images’s luminance channel111Future research includes the possibility of using more advantageous “scalarization” of color images, such as Gooch et al. [26].

All spaces in this paper are compact metric spaces and all functions are continuous. For a subset of a topological space, we denote the interior by , the closure by , the boundary by , and set difference by .

3.1 Varilets

We briefly review the results of [10], starting with a classic result of analytic topology.

Continuous function is monotone when is connected for all ; thus carries connected sets to connected sets. is light when is totally disconnected for all .

The monotone-light factorization [63, 61, 37] states that for every continuous function there exists a unique compact metric space , called ’s middle space, such that , where is monotone and is light.

(1)

We restrict our attention to scalar fields, i.e. . We denote ’s monotone-light factorization by .

3.1.1 Piecewise Monotone Functions

Definition 3.1.

Scalar field is piecewise monotone when is a finite graph.

For piecewise monotone , the middle space is graph-theoretically and topologically identical to ’s Reeb graph [49]. The monotone-light factorization is a useful context for the Reeb graph.

For an image, where interpolates the luminance channel, can always be chosen to piecewise monotone222A forthcoming paper will discuss interpolation schemes for varilet analysis..

Most varilet analysis is couched in terms of and , subsequently using monotone factor inverse to get back to domain . This is powerful, because enjoys the simple topology of graph continua [30]. Monotonicity of makes the theory oblivious to the many complexities of continua.

’s critical points are the vertices of ; points within an edge are regular points.

The light factor is numerically monotone along each edge of . If vertex terminates both an increasing and a decreasing edge, then it is a saddle; in this case terminates three or more edges. A vertex at which all edges have the same direction is an extremum, either a maximum or minimum. ’s global extrema are the points of mapped by to ’s maximum and minimum values; all other extrema are local.

An interpolated image’s domain is typically a rectangular region, for which the Reeb graph is a tree, also know as the contour tree [13]. However, data missing from the image may result in holes in the domain, or the image may have a topologically more complex domain, in which case the graph will have loops. Varilet analysis applies equally well in all cases.

3.1.2 Varilets and Varilet Transforms

We use the light factor to measure length of edges in : for edge terminated by vertices , the length of is .

Definition 3.2.

Topological total variation is the sum of all ’s edge lengths.

Definition 3.3.

A varilet basis is a collection of piecewise monotone functions , called varilets, such that every linear combination has topological total variation

(2)

Varilets are independent in the sense that only when all . Please note that equation (2) differs by a normalization factor from the formulation of [10].

Topological total variation’s relation to a varilet basis is analogous to that of energy for a finite Fourier basis. The name varilet stems from this partitioning of topological total variation.

Definition 3.4.

A varilet transform for is a varilet basis for which

(3)

In [10], I provide a simple mathematical algorithm that produces many different varilet transforms for any piecewise monotone . Each transform is specified by a special type of finite hierarchy on ’s middle space, which we call a lens.

3.1.3 Varilet Lens

A varilet lens for scalar field is defined in terms of ’s monotone-light factorization . A lens is a subset of the middle space .

Definition 3.5.
  • A subset is a lens region for when is closed, connected, has nonempty interior, and is constant on .

  • A lens facet is a connected component of a lens region’s interior.

  • A varilet lens for is a finite collection of lens regions, with , such that any two are either nested or disjoint.

Let be a varilet lens for . Each lens region and each facet may be pulled back to ’s domain by .

Because is connected and open, the boundary components of are equal-luminance Jordan curves.

The lens hierarchy also organizes a lens’s facets: Let be ’s facets. When another lens region , then each of ’s facets is a subset of some  .

3.1.4 Varilet Supports

Given a varilet lens for , the varilet transform produces one varilet basis function for each lens region . Then each basis function ’s support is the following subset :

(4)

Each varilet is non-constant only on its support’s inverse image .333“Support” traditionally means “non-zero”, but here means “non-constant”.

For , support always contains at least one point of .

Suppose is an immediate successor in the lens hierarchy, i.e. there does not exist lens region with . Then contains at least one point of .

This discussion enables us to define the half-open support :

(5)

The collection partitions the middle space .

3.2 Image Segmentation

A varilet lens’ supports cover , intersecting only at their boundaries. This makes the supports’ inverse images a natural candidate for segmentation.

However, for image processing we take a different approach. The image segmentation for lens is defined as the collection of inverse images of all of ’s facets; we will also refer to these as facets.

As discussed above, each lens facet is bounded by one or more constant-luminance Jordan curves, and the facets inhabit ’s lens region hierarchy.

The figures of the section 2 show the unfilled regions resulting from segmentation incompleteness.

The unfilled regions include image domain points that do not lie in the first level of the segmentation. These will always exist, because the first level lens regions are closed and disjoint.

Additionally, for any first level lens region , its boundary’s inverse image may properly contain the Jordan boundary components of its facets; this difference will also be part of the unfilled region. The unfilled points of include various contour phenomena, including junctions, crack tips [43], and regions of nonempty interior.

Varilet analysis articulates multiscale Jordan boundary components, at the cost of not articulating more complex phenomena. This may offer practical advantages for image processing:

  • By working exclusively with Jordan boundary components, we avoid the full complexity of image contour lines.

  • The image topology data required for the hierarchy of Jordan boundary components is acquired as part of the construction of the monotone-light factorization, leveraging a rich literature on image Reeb graph construction, e.g. [13, 15].

  • Truncation of recursion depth corresponds to image simplification (section 3.4).

  • Vector graphics realization of filled segments is immediate, with lens facet drawing order reflecting the lens hierarchy from root down to leaves.

    • Vector graphics admits application of continuous mathematics.

    • Vector graphics supports exploratory data analysis by zooming.

3.3 Vectorization of Segmentation

Vectorization of Jordan boundaries is accomplished as follows: ’s middle space is constructed by sweeping a plane through ’s graph. For every interval of luminance values not containing a sample or critical point, we save the sequences of pixel coordinates through which the plane passes. Subsequently, when we need to vectorize a lens facet boundary having luminance value , we use to lookup the coordinate sequence, then calculating the interpolated sub-pixel path of the boundary as an SVG polyline.

The main complication stems from boundaries that intersect the image frame; these are completed to a simple closed curve by continuing around the frame until meeting the other end. Boundaries are oriented, thereby providing disambiguation at several algorithmic junctures.

A lens facet is rendered as an SVG draw command comprising the collection of closed polylines of its boundary components.

Lens facets are correctly filled, including holes, by SVG’s even-odd fill rule. Fill color may be chosen as the gray level of the facet’s boundary (figure 3), or may be selected from the pixel color statistics of the facet region.

3.4 Image Simplification

Given interpolated luminance image and lens , the varilet transform yields a varilet basis such that . From this basis we may construct filtered , where each .

We restrict our attention to binary filters, where either or . A binary filter is proper when at least one, but not all, .

Suppose is a continuous luminance image having monotone-light factorization . Let be a lens for , and let be the varilet basis resulting from the varilet transform.

Definition 3.6.

An image simplification of is a proper binary filter such that implies for all .

The filtered function defines a gray vector image, using a continuous gray scale .

In the following subsections we discuss characteristics of simplified images.

3.4.1 Simplification Kernel and Cokernel

Suppose lens and binary coefficients define simplified image . Letting denote the varilet supports, we define the simplification’s kernel and cokernel as the following subsets of ’s middle space :

and cover ’s middle space; they intersect only at their boundaries, which are equal.

From definition 3.6 it follows that the components of are lens regions.

3.4.2 Simplification as Quotient

Simplification ’s monotone-light factorization has middle space which is a topological quotient of . The quotient map is monotone.

is a homeomorphism on the cokernel’s interior ; whereas each kernel component is mapped to a distinct point.

’s monotone-light factorization is diagrammed in (6), wherein function is identical to except that is constant on each kernel component, and denotes any right inverse of .

(6)

3.4.3 Varilet Transform of Simplification

From ’s lens we may construct a simplified lens for simplification  :

(7)

Using this simplified lens, the varilet transform of yields varilet basis , i.e. the basis of the simplification is the obvious subset of the original image’s basis, here expressed in terms of varilet supports in the cokernel:

(8)

Comparing equations (8) and (3), we see that the simplification is simply the original sum without the terms for varilets having support in the kernel. This provides a useful characterization of the varilet transform: By transforming ’s representation to varilet basis functions , simplification corresponds to deletion of terms from the sum  ; i.e.  is the projection of to the subspace spanned by  .

When drawing varilet segmentations as vector graphics, the segmentation of a simplification is simply a truncation of the hierarchy of nested facets.

3.5 Critical Point Trackability and Stability

The early scale space papers of Witkin [64] and Koenderink [31] identify desiderata for scale space, paraphrased and contextualized as follows.

  • Critical points can be tracked across scales.

  • No new critical points are created as scale increases.

  • Critical points have stable magnitude and location across scales.

By trackability we mean #1 & 2; by stability we mean #3.

We discuss relationships between the critical points of and those of simplified image , using lens , quotient map , kernel and cokernel .

  • [A] Each critical point in the interior of kernel is removed. In other words, no critical point persists through this simplification; these critical points track to nowhere.

  • [B] Each critical point in the interior of cokernel is retained. In other words, every critical point persists, without change, through this simplification; these critical points track one-to-one.

  • [C] For any critical point , whether, and in what form, persists through this simplification depends on information not found in the general case.

Statements [A] and [B] make critical point tracking easy, whereas [C] requires an additional definition in order to ensure desideratum #2.

Definition 3.7.

Lens is trackable when each lens region contains a critical point in its boundary.

Definition 3.7 ensures desiderata #1&2.

Regarding desideratum #3, when lens region is a kernel component, then has constant value on , and thus critical point values are stable. Critical point “location” can be considered stable in the limited sense that implies , with equality holding in case [B].

3.6 Persistence Lens

The notion of lens is quite general, but for image processing we focus on persistence lenses, utilizing the contours of critical points of persistence birth-death pairs [22]. In this section we describe the construction and properties of persistence lenses.

The power of persistent homology [22] for topological data analysis [11] derives in part from its context in algebraic topology; however in this paper take a simpler approach for more limited results.

3.6.1 Persistence

Suppose scalar field has monotone-light factorization . Working exclusively in ’s middle space :

Definition 3.8.

Local maximum ’s persistence region is the largest lens region containing such that:

  •  and

  •  .

The definition is symmetric for minima. For global extremum , we define .

Local extremum ’s persistence is

(9)

For global extrema, is equal to the length of ’s image.

Persistence regions of same-sense extrema are either disjoint or nested. On the other hand, persistence regions of opposite-sense extrema may have intersecting interiors without being nested. This situation is closely related to the problem of simplification conflicts described by Edelsbrunner et al. [24].

As in Agarwal et al. [1] and Bauer et al. [5], persistence regions can be computed in two independent global sweeps of ’s middle space, with the up-sweep capturing minima and the down-sweep capturing maxima.

We will use paths in . A path has no self-intersections. A path has a direction, starting at some point and ending at another. Concatenation of paths and is denoted .

For local maximum (with a symmetric statement for minima), maximality of implies that there exists a critical point and a path from to a point such that:

  •  and

  •  and

  •  .

We call an apogee, a dominator, and a dominator path. The designation “apogee” is a many-to-many generalization of persistence pairing; “dominator” corresponds to the Elder Rule of Edelsbrunner [22].

The collection of all of ’s apogees is denoted . Not necessarily every point of is an apogee, but there exists at least one, and (by definition) . The possibility of multiple apogees follows from the possibility of equal values. One may easily construct examples for which two local extrema share an apogee. Global extrema have no apogees.

Any path from to an apogee is an apogee path. Concatenation of an apogee path and an dominator path is a persistence path , denoted , thereby indicating an extremum , apogee , and dominator .

3.6.2 Persistence Lens Definition

Consider varilet lens having half-open varilet supports .

Definition 3.9.

Lens region is persistence closed when local extremum implies .

Definition 3.10.

Lens region is externally dominated when local extremum implies that has a dominator such that for all .

Definition 3.11.

Varilet lens is a persistence lens when is trackable, and every lens region is persistence closed and externally dominated.

3.6.3 Extremal Tracking and Persistence Semi-stability

A persistence lens ensures the following extremal tracking and persistence semi-stability properties, proved in appendix A.

Proposition 3.12.

When using a persistence lens, then for each extremum of any simplification , there exists a same sense extremum of such that , and for every such we have .

In the proposition, is the monotone quotient map of diagram (6).

This is a “sense making” result, extending trackability to extrema and providing a limited form of stability of persistence: Every extremum of a simplified image is tracked to by one or more extrema in the original image, and persistence does not grow. The particulars of definition 3.11 (persistence lens) were chosen specifically for this reason.

3.6.4 Construction of Persistence Lens by Conflict Resolution

Let be the collection of all persistence regions for minima of ; and similarly for maxima444The arrows reflect the sweep direction.. Each of and is a persistence lens.

However, is not in general a varilet lens. We say that overlaps when they have non-empty intersection but are not nested.

We construct a persistence lens by an iterative process of conflict resolution for overlapping regions. There are multiple ways to resolve conflicts; this is a source of multiplicity of persistence lenses.

We construct a persistence lens by first constructing , and then removing or substituting for every pair of overlapping regions, iterating this process until no conflicts remain.

Suppose overlaps . One may resolve this conflict by simply omitting both regions from the final result. However, we want lenses to have more structure, not less, and therefore are motivated to find alternatives. Two solutions that we currently use for images include the following555Addition additional conflict resolutions are possible.:

  • : Choose one of and .

  • : When , substitute lens region .

Without getting into the details of the iteration, we can sketch an inductive proof that iterated conflict resolution results in a persistence lens. Starting with , all persistence lens requirements except nesting are met. This situation obtains after applying the conflict resolution rules. The iteration halts when the lens regions are nested.

3.7 Image Scale Space

In this section we endow varilet basis functions with scale measures. The smallest detail appearing in when viewed through lens is expressed in terms of varilet transform as the minimum value of  . Simplification can increase scale by removing small detail.

3.7.1 Scale Measures

A scale measure assigns to each piecewise monotone function a nonnegative number . Scale measures will be applied to varilets.

We currently use the following scale measures; many more are possible. This list comprises one geometric, one topological, and one image measure.

  • The area of ’s support.

  • Topological total variation  .

  • Contrast .

3.7.2 Scale Space Simplification

Thresholding scale measure at value provides the cokernel of a scale space simplification by pruning the hierarchy of persistence lens  :

(10)

As expressed by equation (8), the resulting simplified image has a naturally induced varilet lens for which the varilet basis is a subset of ’s varilet basis, each having .

By varying the threshold we may generate the collection of all distinct scale threshold simplifications of ; this is ’s scale space for persistence lens and scale measure .

Not every simplification is a scale space simplification. We measure the scale measure’s fit to lens as the fraction of simplifications that are scale space simplifications; this is typically in the range .

3.8 Image Fractal Regions

A fractal function’s graph has non-integral Hausdorff dimension, the function’s fractal dimension [38]. Fractal analysis of multiscale data makes use of a variety of empirical fractal indices, some of which correspond to fractal dimension, whereas others stand on their own merit [38, 56].

We use the following size counting paradigm for fractal analysis:

  • Let be a multiset of nonnegative numbers, each of which is considered to be a measurement of the “size” of a “feature” of .

  • Create the empirical distribution of feature size counts from .

  • Determine whether has a power law distribution, and if so, estimate the exponent.

For step 3, Clauset et al. [19, 60] provide a maximum likelihood estimator for the power law exponent, using the Kolmogorov-Smirnov statistic for goodness of fit.

We determine the fractal characteristics of image using a persistence lens and scale measure .

Suppose lens gives varilet basis having supports . To view a lens region as a fractal, we use Clauset’s maximum likelihood estimator on the distribution of varilet scales .

We calculate the power law exponent and goodness of fit for every lens region , and then aggregate the largest-possible regions having consistent exponent and high goodness of fit, resulting in analysis as shown in figure 10 of section 2.

Varying the choice of lens and/or scale measure causes minor variation of the fractal regions, an area of ongoing research.

3.9 Theory Summary

Varilet analysis is a novel and elementary extension of classic results of analytic topology [63]. Varilet analysis may have benefits as part of the larger image processing toolbox. These include theoretically and algorithmically elementary forms of:

  • multiresolution analysis,

  • vector graphics display,

  • scale space and fractal analysis.

Varilet analysis focuses on monotonicity in several guises, e.g. topological monotonicity of ; numerical monotonicity of along the graph edges of ; monotone quotient map , and piecewise monotonicity of . In this respect varilet analysis may be complementary to existing methods.

By transforming ’s representation to varilet basis functions , image simplification corresponds to deletion of terms from the sum .

Because varilet image analysis happens in the middle space, it relies on data collected during the monotone-light factorization in order to compute and thereby pull simplifications back to the image space. The data for each varilet can be coded in various ways, and may be complete or incomplete. In the present application to image analysis, we chose to code Jordan boundaries only, leaving as unmodelled the more complex aspects of the image (section 3.2)666It is possible to collect additional data for in order to have more options and control of the rendered image representation, but we have not done so in this paper..

Varilet analysis relies on finiteness of the Reeb graph and continuity of the luminance image. This is a drastic mathematical simplification when compared to, say, functions of bounded variation. However, we see images as fundamentally finite, and we are content to allow closely spaced contours to straddle the ambiguity between continuous and discontinuous.

As an overall summary: Varilet image analysis provides many capabilities that are also provided by existing techniques, but that varilet analysis does so in a mathematically and algorithmically elementary way. This viewpoint is further explored in the next section.

4 Related Work, Contributions and Discussion

Varilet image analysis has commonalities with many theories. In this section we compare selected image processing approaches to varilets.

4.1 Reeb Graph & Simplification

Simplification of sampled two-dimensional scalar fields has appeared in work by Carr [12], Carr et al. [14], Weber et al. [62], Bremer et al. [9], Edelsbrunner et al. [24, 21], Gyulassy et al. [28], Bauer et al. [6], and Tierny et al. [57, 58].

Each of these references uses the topological structure of the scalar field to guide simplification. Carr et al. [12, 14] and Weber et al. [62] use the Reeb graph [49], Bremer et al. [9] and Gyulassy et al. [28] use the Morse-Smale complex [23], and Edelsbrunner et al. [24, 21] use the persistence diagram. The Reeb-based techniques are concerned with removing extrema; the Morse-Smale and persistence-diagram methods may also remove critical points related to the genus of isosurfaces.

Varilet’s contribution to Reeb graph analysis is formulation of simplification as a quotient (diagram 6). This is accomplished by transforming ’s representation to a varilet basis , in which simplification corresponds to deletion of terms from the sum .

In Carr et al. [14], the order in which extrema are removed is determined by pruning contour tree leaves in preference order, using any of a variety of local geometric measures; this reference motivated varilets’ use of geometric, topological and image measures (section 3.7).

Piecewise monotone functions have identical middle space and Reeb graph. The monotone-light factorization entails additional information in the form of the monotone and light factors, as reflected in our notation . There is nothing that prevents Reeb graph analysis from recognizing these functions. For example, light factor (differently named) was used to show stability of the Reeb graph under perturbations of by Bauer et al. [5] and Di Fabio et al. [4].

Bauer et al. [5] simplify by removing features having persistence below a threshold. Although varilets use persistence to define lens structure, varilets do not use persistence to drive simplification, due to a “type” mismatch: We have assigned persistence to individual extrema, but we measure scale on a basis function. Contrast is the luminance image measure (section 3.7.1) most closely related to persistence; e.g. contrast is persistence for extremal persistence regions (section 3.6.1).

Bauer et al. [6] combine discrete Morse theory and persistent homology for function simplification guided by discrete vector fields. As do varilets, as well as [12, 57], they simplify by flattening. Tierny et al. [57, 58] simplify scalar fields by working directly in the image space with guidance from the middle space topology, whereas varilets work directly in the middle space, lifting the result back to the image with only at the end. Image-space simplification in [6, 57, 58] is powerful because it exercises full control over all details of the image. As discussed in sections 3.2 & 3.9, this is in contrast to the varilet image processing, where we incompletely model image structure.

Computational methods for simplification of three-dimensional visualization geometry use edge contraction in a triangular mesh [29]. Some approaches include topological considerations based on the Reeb graph, e.g. Takahashi et al. [54]. These works differ from varilet simplification, because they focus on simplifying the domain geometry of triangulated surfaces rather than simplifying scalar fields on a fixed domain.

4.2 Image Segmentation

Mumford et al. [43] define a regularized equation whose solution provides a complete global image segmentation. Their approach has been refined and solutions have been explored; for a review see [34].

Hierarchical image segmentation is also well-developed, including e.g. Abelaez et al. [3], who provide the Berkeley Segmentation Data Set [39]. Guigues et al. [27, 55] use scale sets, utilizing piecewise constant segmentation by regularizing within a hierarchy defined by persistence of regions. A similar approach is taken by Xu et al. [66], with additional guidance from shape space semantics.

Varilets’ contribution to image segmentation is its mathematically elementary formulation of image segmentation as a hierarchy of open regions (lens facets) of the image space, each bounded by Jordan level sets.

4.3 Image Scale Space

Starting with Witkin [64] and Koenderink [31], scale space has provided a parameterized family of smoothed images. For an overview, see Lindeberg [36].

Scale space theory includes both linear and nonlinear scale spaces. Linear scale spaces result from Gaussian smoothing, equivalently formulatied as heat diffusion [31]. Nonlinear scale spaces have various motivations, including the fact that certain types of multiresolution sensing systems are built around non-Gaussian filters [67], a desire to extend the scale space concept to morphological filtering [18, 42], and dissatisfaction with Gaussian filtering for vision applications [53, 47].

Varilet scale space looks different than these references, but shares their basic intent: a sequential removal of detail, where at each stage the removed detail has smaller scale than what remains. Varilet scale space fully embraces the semantics of “causality” [31], using simplification’s monotone quotient map to track identity (section 3.5).

Reininghous et al. [50] define persistence scale space, conceptually similar to varilets, but based on persistent homology and discrete Morse theory, whereas varilets are based on more elementary analytic topology.

Chen et al. [17] study persistence diagrams norms as a function of the degree of scale space diffusion. Their experience of rapidly decreasing norms is consistent with proposition 3.12. Their figure 2 shows a linear log-log relationship between scale and number of extrema, indicating the possibility of a power law distribution; this would be similar for many natural images, due to naturally-occurring fractal content. We note that power laws constitute a relatively slow decay; one may expect non-fractal images to exhibit exponential decay, further supporting the reference’s observation.

Monasse et al. [41] construct a scale space from level sets by representing them with all holes filled in. By comparison, varilet analysis retains the holes of a level set’s interior.

Chao et al. [15] construct a topologically multiscale scale space Reeb graph for content matching applications.

4.4 Image Fractal Analysis

Fractal structure of textures and natural images is well known; for example  [51, 59, 20], and see the image processing applications at FracLab [35]. Blondeau et al. [16] measure fractal color distribution. Various measurement and estimation schemes are applied, including wavelets, box-counting, energy and pixel methods.

Varilet analysis is complementary to these methods, utilizing a different measurement and counting domain for power law estimation (section 3.8): For any choice of persistence lens, together with any choice of scale measure, the count of varilets by scale is input to Clauset’s maximum likelihood estimator for power law exponent [19] (figure 10). Again, this approach is simpler and more direct than many, and may therefore be useful.

4.5 Jordan Boundaries

It has been recognized, e.g. by Ambrosio et al. [2, 40], that a level set’s interior’s boundary components are Jordan curves; these same references construct a hierarchy of Jordan curves boundaries with similarlies to varilet’s. Varilet analysis differs from this work in two ways: (1) Whereas the references work in the image space, varilets work in the middle space; and (2) the references merge two distinct hierarchies of Jordan curves to get the region hierarchy, whereas varilets utilize an externally supplied hierarchy in the form of a lens parameter (section 3.6.4). Different lenses may be used, in accordance with differing requirements and preferences.

4.6 Image Vectorization

Varilet image analysis provides vector representation of hierarchical image segmentation, by vectorizing the Jordan lens facets’ boundary components. The luminance value is identical for all Jordan boundaries of the same lens region.

Image vectorization methods typically use image segmentation and/or edge detectors, e.g. Selinger [46]. Birdal et al. [8] merge regions of similar color. Kopf et al. [33]

use heuristics to group pixels into cells.

Orzan et al. [45], Xie et al. [65], and Olsen et al. [44] combine image simplification and vectorization using multiscale diffusion curves.

Fuchs et al. [25] provide a level-of-detail approach to progressive SVG imagery. Whereas their SVG is the entire image, varilets’ SVG is the segmented image (section 3.2); therefore the two methods are not addressing the same problems. However, varilets’ lens hierarchy does provide a natural source for level-of-detail SVG streaming of the the segmented image.

4.7 Image Total Variation

Varilet analysis takes a new view by defining topological total variation as the sum of the Reeb graph’s arc lengths (section 3.1.2). is fundamental; the varilet basis functions partition in analogy to Fourier and wavelet partitions of energy. serves as a topological measure for scale space and fractal analysis.

Total variation image denoising is well known [52].

Bauer et al. [7] link total variation and persistence in denoising. Plonka et al. [48] apply a variant of total variation denoising employing a regularization term having persistence-derived coefficients. Further research may show relationships to varilets’ use of a persistence lenses for image processing.

5 Conclusion

We have presented a novel image processing approach having very direct expressions of image segmentation, simplification, vectorization, scale space and fractal analysis. The purpose of our exposition is to create awareness of varilet analysis as a basis for additional image processing tools.

The author wishes to thank Dr. Michael Stieber at Apollo Systems Research Corporation, and also National Research Council Canada, for support of this research.

Appendix A Appendix: Proof of Extremal Tracking and Persistence Stability

Consider lens and simplification having quotient map , kernel , cokernel , and monotone-light factorization .

We prove proposition 3.12 in two parts in the following two subsections.

a.1 Extremal Tracking

Proposition A.1.

Let be a persistence-closed kernel component containing no global extrema in its interior. Then is an extremum only when contains a same-sense extremum.

Proof.

Suppose is a maximum. Then for each boundary point , every edge that does not lie entirely in terminates at a critical point , with .

Assume is comprised entirely of saddles and regular points; we will derive a contradiction. From this assumption we have:

  • Property X: For every there exists a maximum and a path from to upon which is only increasing.

We now proceed with the proof. There exists at least one maximum such that each of its dominators ; for example, may be chosen by the condition .

Let be such a maximum, chosen with the additional constraint that is minimal over all such maxima.

Choose any dominator path for , ending at a dominator .

Let be the last boundary point along ; then along the subpath of that starts at and continues outside , let a point for which is least. Then , and by definition 3.8 we know .

Let be the extremum stipulated by property ; we claim . We know , because otherwise would be a dominator for ; therefore . Thus, if not , then and ; this contradiction proves the claim.

Now consider any maximum such that , and such that there exists a path from to having . Such maxima exist, because the maximum and path stipulated by Property X satisfy the criteria. Then , since otherwise would be a dominator of . It follows that .

Now choose such a maximum , with the additional constraint that is maximal over all such extrema. We claim that by maximality, for every dominator . The claim follows because the stipulated path from to can be concatenated with the persistence path path from through some to same-sense extremum dominator ; this concatenated path satisfies , and therefore does not lie in .

Finally, this last conclusion causes to contradict the minimality of , thereby contradicting property X and the assumption that does not contain a maximum. ∎

a.2 Non-Increasing Persistence

Proposition A.2.

for every local extremum and each extremum .

Proof.

Choose any local extremum , and then choose any same-sense extremum such that . Then and .

Let be a persistence path having dominator with . Then , where .

From and ’s images , we define path , from to , and then to . Since each kernel component is a constant-boundary region , there can be a loop within or only if or crosses multiple times in and out of ; paths and result from excising all such loops.

We cannot derive a lower bound for , because we have no information about persistence paths for . But we can use to prove by showing that implies . Assume to be a maximum.

Neither nor lie in the interior of any kernel component, but points along path , including apogee , may do so. Consider any kernel component such that path intersects  ; the following argument applies also when intersects  : Because , we have . In other words, every point has

(11)

Let be the point along path at which takes its least value; if there are several points having this value, take to be the last along the path. Let be the subpath of from to , and let be the subpath from to . Then by equation (11), and  . Thus only when , in which case . ∎

References

  • [1] Pankaj K. Agarwal, Herbert Edelsbrunner, John Harer, and Yusu Wang. Extreme elevation on a 2-manifold. Discrete & Computational Geometry, 2006.
  • [2] Luigi Ambrosio, Vicent Caselles, Simon Masnou, and Jean-Michel Morel. Connected components of sets of finite perimeter and applications to image processing. Journal of the European Mathematical Society, 3(1):39–92, Feb 2001.
  • [3] Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 33(5):898–916, May 2011.
  • [4] Claudia Landi Barbara Di Fabio. The edit distance for reeb graphs of surfaces. arXiv:1411.1544, 2014.
  • [5] Ulrich Bauer, Xiaoyin Ge, and Yusu Wang. Measuring distance between reeb graphs. In Proceedings of the Thirtieth Annual Symposium on Computational Geometry, SOCG’14, pages 464–473, New York, NY, USA, 2014. ACM.
  • [6] Ulrich Bauer, Carsten Lange, and Max Wardetzky. Optimal topological simplification of discrete functions on surfaces. Discrete & Computational Geometry, 47(2):347–377, 2012.
  • [7] Ulrich Bauer, Carola-Bibiane Schönlieb, and Max Wardetzky. Total variation meets topological persistence: A first encounter. In Proceedings of ICNAAM 2010, pages 1022–1026, 2010.
  • [8] Tolga Birdal and Emrah. Bala. A novel method for vectorization. ArXiv e-prints.
  • [9] P.-T. Bremer, B. Hamann, H. Edelsbrunner, and V. Pascucci. A topological hierarchy for functions on triangulated surfaces. IEEE Transactions on Visualization and Computer Graphics, 10(4):385 – 396, 2004.
  • [10] Martin Brooks. Varlets: Additive decomposition, topological total variation, and filtering of scalar fields, 2015. arXiv:1503.04867.
  • [11] Gunnar Carlsson. Topology and data. Bull. Amer. Math. Soc., 46:255–308, 2009.
  • [12] Hamish Carr. Topological manipulation of isosurfaces. PhD Thesis, University of British Columbia, 2004.
  • [13] Hamish Carr, Jack Snoeyink, and Ulrike Axen. Computing contour trees in all dimensions. In Proceedings of the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’00, pages 918–926, Philadelphia, PA, USA, 2000. Society for Industrial and Applied Mathematics.
  • [14] Hamish Carr, Jack Snoeyink, and Michiel van de Panne. Simplifying flexible isosurfaces using local geometric measures. In IEEE Visualization, pages 497–504. IEEE Computer Society, 2004.
  • [15] Jinhui Chao and Shintaro Suzuki. A scale-space reeb-graph of topological invariants of images and its applications to content identification. In Fiorella Sgallari, Almerico Murli, and Nikos Paragios, editors,

    Scale Space and Variational Methods in Computer Vision

    , volume 4485 of Lecture Notes in Computer Science, pages 338–349. Springer Berlin Heidelberg, 2007.
  • [16] François Chapeau-Blondeau, Julien Chauveau, David Rousseau, and Paul Richard. Fractal structure in the color distribution of natural images. Chaos, Solitons & Fractals, 42(1):472 – 482, 2009.
  • [17] Chao Chen and Herbert Edelsbrunner. Diffusion runs low on persistence fast. In IEEE International Conference on Computer Vision (ICCV), 2011.
  • [18] M. H. Chen and P. F. Yan. A multiscanning approach based on morphological filtering. IEEE Trans. Pattern Anal. Mach. Intell., 11(7):694–700, July 1989.
  • [19] A. Clauset, C. Rohilla Shalizi, and M. E. J. Newman. Power-law distributions in empirical data. SIAM Rev., 51(4):661–703, 2009.
  • [20] A. F. Costa, G. Humpire-Mamani, and A. J. M. Traina. An efficient algorithm for fractal analysis of textures. In 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images, pages 39–46, Aug 2012.
  • [21] H. Edelsbrunner, D. Morozov, and V. Pascucci. Persistence-sensitive simplification of functions on 2-manifolds. Proc. 22nd ACM Symp. Computational Geometry, 2006.
  • [22] Herbert Edelsbrunner and John Harer. Computational Topology - an Introduction. American Mathematical Society, 2010.
  • [23] Herbert Edelsbrunner, John Harer, and Afra Zomorodian. Hierarchical morse-smale complexes for piecewise linear 2-manifolds. Discrete Comput. Geom., 28:87 – 107, 2003.
  • [24] Herbert Edelsbrunner, David Letscher, and Afra Zomorodian. Topological persistence and simplification. Discrete Comp. Geo., 28:511 – 533, 2002.
  • [25] Georg Fuchs, Heidrun Schumann, and René Rosenbaum. Progressive imagery with scalable vector graphics. volume 7881, pages 788108–788108–12, 2011.
  • [26] Amy A. Gooch, Sven C. Olsen, Jack Tumblin, and Bruce Gooch. Color2gray: Salience-preserving color removal. ACM Trans. Graph., 24(3):634–639, July 2005.
  • [27] Laurent Guigues, Jean Pierre Cocquerez, and Hervé Men. Scale-sets image analysis. International Journal of Computer Vision, 68(3):289–317, 2006.
  • [28] Attila Gyulassy, Vijay Natarajan, Valerio Pascucci, Peer Timo Bremer, and Bernd Hamann. A topological approach to simplification of three-dimensional scalar functions. IEEE Transactions on Visualization and Computer Graphics, pages 474 – 484, 2006.
  • [29] H. Hoppe. Progressive meshes. SIGGRAPH, 1996.
  • [30] Sam B. Nadler Jr. Continuum Theory, An Introduction. Marcel Dekker, Inc., New York, 1992.
  • [31] Jan J. Koenderink. The structure of images. Bilo. Cybern., 50:363–370, 1984.
  • [32] G. Koepfler, C. Lopez, and J. M. Morel. A multiscale algorithm for image segmentation by variational method. SIAM Journal on Numerical Analysis, 31(1):282–299, 1994.
  • [33] Johannes Kopf and Dani Lischinski. Depixelizing pixel art. ACM Trans. Graph., 30(4):99:1–99:8, July 2011.
  • [34] Antoine Lemenant. A selective review on mumford–shah minimizers. Bollettino dell’Unione Matematica Italiana, 9(1):69–113, 2016.
  • [35] Jacques Lévy Véhel and Pierrick Legrand. Signal and Image processing with FracLab. In FRACTAL04, Complexity and Fractals in Nature, 8th International Multidisciplinary Conference, volume Thinking in Patterns : fractals and related phenomena in nature, pages 321–322, Vancouver, Canada, April 2004.
  • [36] Tony Lindeberg. Generalized axiomatic scale-space theory. Advances in Imaging and Electron Physics, 178:1–96, 2013.
  • [37] Harriet Lord. Monotone-light factorizations: a brief history. Preprint available at /www.csupomona.edu/~hlord/hh60.ps.
  • [38] B. Mandelbrot. The fractal geometry of nature. Macmillan, 1983.
  • [39] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.
  • [40] P. Monasse and F. Guichard. Fast computation of a contrast-invariant image representation. IEEE Transactions on Image Processing, 9(5):860–872, May 2000.
  • [41] Pascal Monasse and Frédéric Guichard. Scale-space from a level lines tree. Journal of Visual Communication and Image Representation, 11(2):224 – 236, 2000.
  • [42] A. Morales and R. Acharya. Nonlinear multiscale filtering using mathematical morphology. In

    Computer Vision and Pattern Recognition, 1992. Proceedings CVPR ’92., 1992 IEEE Computer Society Conference on

    , pages 572–578, Jun 1992.
  • [43] David Mumford and Jayant Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42(5):577–685, 1989.
  • [44] Sven Olsen and Bruce Gooch. Image simplification and vectorization. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, NPAR ’11, pages 65–74, New York, NY, USA, 2011. ACM.
  • [45] Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. Diffusion curves: A vector representation for smooth-shaded images. ACM Trans. Graph., 27(3):92:1–92:8, August 2008.
  • [46] Selinger P. Potrace: a polygon-based tracing algorithm. http://potrace.sourceforge.net. Accessed: 2016-04-20.
  • [47] P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 12(7):629–639, Jul 1990.
  • [48] Gerlind Plonka and Yi Zheng. Relation between total variation and persistence distance and its application in signal processing. Advances in Computational Mathematics, pages 1–24, 2015.
  • [49] Georges Reeb. Sur les points singuliers d’une forme de Pfaff complètement intégrable ou d’une fonction numérique. C. R. Acad. Sci. Paris, 222:847–849, 1946.
  • [50] Jan Reininghaus, Natallia Kotava, David Günther, Jens Kasten, Hans Hagen, and Ingrid Hotz. A scale space based persistence measure for critical points in 2d scalar fields. IEEE Trans. Vis. Comput. Graph.IEEE Trans. Vis. Comput. Graph., 17(12):2045–2052, 2011.
  • [51] Daniel L. Ruderman and William Bialek. Statistics of natural images: Scaling in the woods. Phys. Rev. Lett., 73:814–817, Aug 1994.
  • [52] Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60:259 – 268, 1992.
  • [53] P. Saint-Marc, J.-S. Chen, and G. Medioni. Adaptive smoothing: a general tool for early vision. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 13(6):514–529, Jun 1991.
  • [54] Shigeo Takahashi, Yuriko Takeshimab, and Issei Fujishirob. Topological volume skeletonization and its application to transfer function design. Graphical Models, 66(1), January 2004.
  • [55] et al Teixidó N, Albajes-Eizagirre A. Hierarchical segmentation-based software for cover classification analyses of seabed images (seascape). Marine Ecology Progress Series, 33(5):45–53, 2011.
  • [56] James Theiler. Estimating fractal dimension. J. Opt. Soc. Am. A, 7(6), June 1990.
  • [57] J. Tierny and V. Pascucci. Generalized topological simplification of scalar fields on surfaces. Visualization and Computer Graphics, IEEE Transactions on, 18(12):2005–2013, Dec 2012.
  • [58] Julien Tierny, David Günther, and Valerio Pascucci. Optimal general simplification of scalar fields on surfaces, 2013.
  • [59] Antonio Turiel and Nestor Parga. The multifractal structure of contrast changes in natural images: from sharp edges to textures. Neural Computation, 12(4):763–793, 2000.
  • [60] Yogesh Virkar and Aaron Clauset. Power-law distributions in binned empirical data. Ann. Appl. Stat., 8(1):89–119, 03 2014.
  • [61] JR Walker. A simplicial monotone-light factorization theorem. Fundamenta Mathematicae, 85(3):229–233, 1974.
  • [62] G.H. Weber, S.E. Dillard, H. Carr, V. Pascucci, and B. Hamann. Topology-controlled volume rendering. IEEE Transactions on Visualization and Computer Graphics, 13(2), 2007.
  • [63] Gordon Thomas Whyburn. Analytic Topology. American Mathematical Society Colloquium Publications, v. 28. American Mathematical Society, New York, 1942.
  • [64] Andrew P. Witkin. Scale-space filtering. In IJCAI, pages 1019–1022, 1983.
  • [65] Guofu Xie, Xin Sun, Xin Tong, and Derek Nowrouzezahrai. Hierarchical diffusion curves for accurate automatic image vectorization. ACM Trans. Graph., 33(6):230:1–230:11, November 2014.
  • [66] Y. Xu, T. Géraud, and L. Najman. Hierarchical image simplification and segmentation based on Mumford-Shah-salient level line selection. ArXiv e-prints, March 2016.
  • [67] B. Zuerndorfer and G.H. Wakefield. Extensions of scale-space filtering to machine-sensing systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 12(9):868–882, Sep 1990.