A Note on Topology Preservation in Classification, and the Construction of a Universal Neuron Grid

08/07/2013 ∙ by Dietmar Volz, et al. ∙ GMX 0

It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constructable in three space dimensions. Conclusions will be drawn for established neuron grid types as well as neural fields.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Mathematical Preliminaries

Topology is one of the basic branches of mathematics. It is sometimes also referred to as qualitative geometry, in a way that it deals with the qualitative properties and structure of geometrical objects. The geometrical objects of interest in this paper are vector spaces, manifolds, and curves. These form the basis of the presented mathematical treatment of classification with neuron grids. Consequently, the paper has to begin with some mathematical preliminaries.

1.1 Manifolds

In the following, a n-dimensional vector space will be identified with a subspace of . It is assumed that is equipped with a topology which in turn is induced by a metric. Mappings between subspaces of are called homeomorphic or topology preserving if they are one-to-one and continuous in both directions. It is known from the theorem of dimension invariance (Brouwer, 1911) that mappings between non-empty open sets and for are never homeomorphic. In the light of Brouwer’s theorem it is the open sets that fix the topological dimension of a subset of . As the is introduced as a metric space, the open sets are given by open balls that formalize the concept of distance between points of this metric space. Homeomorphic mappings preserve the neighborhood relationship between points of . Amongst the huge variety of subsets of the metric space , the n-dimensional manifolds (or n-manifolds) have turned out to be of interest as these describe solution spaces of equations or geometrical entities. Manifolds are parameterized geometrical objects, parameters could describe e.g. the coordinates of a physical space. As n-manifolds are locally Euclidean of dimension n they are subject to Brouwer’s theorem and therefore of fixed topological dimension. As a consequence, dimension reducing mappings between manifolds will not be able to transfer mutual topological structures. A vivid example of the dimension conflict of two manifolds of different dimension is given by a surjective and continuous mapping of onto which is also called a ’Peano curve’ (Peano, 1890). The convoluted structure of a Peano curve does not reflect the neighborhood relationship of elements of the underlying manifold , not even locally: There is no topology preserving mapping of onto . An illustration of a Peano curve to iteration is given in figure 1.

Figure 1: A Peano curve (source: Wikipedia)

1.2 Curves

The Peano curve introduced in the preceding section is based on the conventional definition of a compact curve as a continuous mapping of . The following definition according to (Menger, 1968) renders the definition of a compact curve more precisely. Beforehand, a definition of a continuum is required.
Definition A compact, connected set having more than one element is a continuum.
A set which contains no continuum is called discontinuous.
Definition A continuum as a subset of a metric space is called a curve if every point of is contained in arbitrary small neighborhoods having discontinuous intersects with .
From Menger’s definition of a curve the following theorem results:
Theorem[Menger] Every curve defined in a metric space is homeomorphic to a curve defined in .

The proof of this theorem is left here. The interested reader is referred to Menger’s textbook (Menger, 1968). The theorem states that with regard to topological aspects, the transition from curves defined in three-dimensional Euclidean space to curves defined in an arbitrary metric space doesn’t give any generalization.
Menger’s definition of a curve even leads to another theorem (Menger, 1968).
Theorem[Menger] Every compact curve in a metric space can be mapped to a subset of a so-called universal curve .

The universal curve is constructable and up to homeomorphism uniquely defined. The construction starts with a cube . is the set of all points of such that at least two of the three coordinates have triadic expansions that do not contain a 1. The hereby constructed set fulfills the definition of a curve (Menger, 1968) and leads to a self-similar structure. The construction is shown in figure 2.

Figure 2: Menger’s universal curve in perspective view

Summarizing, the topological relationships that arise from Brouwer’s and Menger’s theorems are as follows:

if and


hereby X, Y, Z are (n,m,3)-manifolds, a curve, identifies a homeomorphic mapping.
Subsets of the universal curve as introduced in this section usually are no manifolds. The only exception are the trivial cases of subsets that are homeomorphic to an interval or .

2 Application to Neuron Grids

2.1 Vector Quantization

The method of Vector Quantization (VQ) (Linde et al., 1980)classifies elements of a vector space

by approximating the probability density function

of elements of . Hereby, the elements of are mapped iteratively to a set of weight vectors such that respective a suitable metric the quantization error functional gets minimal. An extension of the VQ method introduces a topology on the set of weight vectors such that constitutes a topological space which is then called a neuron grid. By means of the topology of the neuron grid, elements of

are mutually adjacent if they are adjacent on the neuron grid. Using algorithms of machine learning, for example Kohonen’s feature maps

(Kohonen, 1990) and several variants thereof (Fritzke, 1992; Oja et al., 1999) try to transfer the topology of the vector space to the topology of the neuron grid . Hereby, the topology of the neuron grid as adapted to the vector space topology is representative of the topology of . The following section will provide some insight to the restrictions of topology preservation in classification of a vector space with VQ methods.

2.2 Identification of a Neuron Grid with a Curve

Assuming Menger’s curve definition, the following identification is made:

Definition A neuron grid is a curve

It is required that neurons of a neuron grid constitute a discrete point set of a curve. Subsets of a neuron grid which contain no neurons are called links between neurons.
Consequently, the topology of the defined neuron grid corresponds to a curve topology. The identification of a neuron grid with a curve as motivated in this section is coherent and straightforward for neuron grids whose topologies are identifiable with a subset of Menger’s universal curve. This is the case for neuron grids of the Kohonen type (Kohonen, 1990) as well as some variants thereof e.g. (Fritzke, 1992; Oja et al., 1999) that constitute topological spaces of a discrete point set together with a set of links between the point set. The presented identification with a curve produces the following results:

  1. All neuron grid topologies are uniquely definable in . There’s no need to introduce ’hypercubes’ or ’high-dimensional’ grid topologies.

  2. Given the neuron grid topology is adapted to a vector space topology , then the neuron grid topology will generally be retained in . Classification results are always visualizable in .

  3. Up to homeomorphism it exists a uniquely defined universal neuron grid such that every connected neuron grid is a subset of the universal neuron grid.

These results were inferred directly from Menger’s curve theorems.
It should be mentioned though that Brouwer’s theorem precludes a direct topology preserving mapping between the vector space topology which is induced by the metric of and if : The topology of a neuron grid that is identifiable with a subset of the universal curve is weaker than the topology of , metrical properties of generally won’t be reproduced on the classification result.
It should also be mentioned that similar results are not possible for a neuron grid if it is defined by a two- or higher-dimensional manifold. The well-known ’Klein bottle’ (Klein, 1923) provides a vivid example what might happen to a two-dimensional surface that is defined in , and shows an apparent self-intersection if mapped to . This mapping is not homeomorphic, although from Brouwer’s theorem homeomorphic mappings of a two-dimensional surface from to would be allowed.

Discussion

The presented theorems of Brouwer and Menger illustrate an alternative though simple and straightforward approach to the topological foundation of neuron grids. The identification of a neuron grid with a subset of the universal curve at a first or second glance may be seen as trivial. This is in particular due to the presented definition of a curve which is intuitive and comprehensible, also from a naive standpoint. The presented topological framework of neuron grids restricts neuron sets to discrete point sets. Alternative approaches (Amari, 1977; Bressloff, 2005; Cottet, 1995) introduce continuous models of neural fields. As neural fields per definition provide open sets in their domain, the present paper also incloses the caveat that neural fields might produce unusable results if applied to classification tasks.

Acknowledgements

I would like to thank Catalin Dartu for the nice visualization of Menger’s universal curve.

References

  • Amari (1977) Amari, S. (1977). Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern, 27, 77 – 87.
  • Bressloff (2005) Bressloff, P. C. (2005). Spontaneous symmetry breaking in self-organizing neural fields. Biol Cybern, 93(4), 256 – 274.
  • Brouwer (1911) Brouwer, L. E. J. (1911). Beweis der Invarianz der Dimensionszahl. Math. Ann., 70, 161 – 165
  • Cottet (1995) Cottet, G. (1995). Neural networks: Continuous approach and applications to image processing. Journal of Biological Systems, 3, 1131 – 1139.
  • Fritzke (1992) Fritzke, B. (1992). Growing Cell Structures - A Self-Organizing Network in k Dimensions. Artificial Neural Networks. North-Holland. Amsterdam, Vol. II, 1051 – 1056.
  • Klein (1923) Klein, F. (1923). Gesammelte Mathematische Abhandlungen. Springer. Berlin, Bd.III, 571
  • Kohonen (1990) Kohonen, T. (1990).

    The self-organizing map.

    Proceedings of the IEEE, 78(9), 1464 – 1480.
  • Linde et al. (1980) Linde, Y., Buzo, A., & Gray, R. M. (1980). An algorithm for vector quantizer design. IEEE Trans. Comm., 28, 84 – 95.
  • Menger (1968) Menger, K. (1968). Kurventheorie. Chelsea Pub Co, 347 – 360.
  • Oja et al. (1999) Oja, E., Laaksonen, J., & Koskela, M. (1999).

    Application of Tree Structured Self-Organizing Maps in Content-Based Image Retrieval.

    Proc. of the 9th Int. Conf. on ANN (ICANN’99), Vol. 1, 174 – 179.
  • Peano (1890) Peano, G. (1890). Sur une courbe, qui remplit toute une aire plane. Math. Ann., 36, 157 – 160.