Non-constant bounded holomorphic functions of hyperbolic numbers - Candidates for hyperbolic activation functions

06/07/2013 ∙ by Eckhard Hitzer, et al. ∙ 0

The Liouville theorem states that bounded holomorphic complex functions are necessarily constant. Holomorphic functions fulfill the socalled Cauchy-Riemann (CR) conditions. The CR conditions mean that a complex z-derivative is independent of the direction. Holomorphic functions are ideal for activation functions of complex neural networks, but the Liouville theorem makes them useless. Yet recently the use of hyperbolic numbers, lead to the construction of hyperbolic number neural networks. We will describe the Cauchy-Riemann conditions for hyperbolic numbers and show that there exists a new interesting type of bounded holomorphic functions of hyperbolic numbers, which are not constant. We give examples of such functions. They therefore substantially expand the available candidates for holomorphic activation functions for hyperbolic number neural networks. Keywords: Hyperbolic numbers, Liouville theorem, Cauchy-Riemann conditions, bounded holomorphic functions

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

For the sake of mathematical clarity, we first carefully review the notion of holomorphic functions in the two number systems of complex and hyperbolic numbers.

The Liouville theorem states that bounded holomorphic complex functions are necessarily constant [1]. Holomorphic functions are functions that fulfill the socalled Cauchy-Riemann (CR) conditions. The CR conditions mean that a complex -derivative

(1)

is independent of the direction with respect to which the incremental ratio, that defines the derivative, is taken [5]. Holomorphic functions would be ideal for activation functions of complex neural networks, but the Liouville theorem means that careful measures need to be taken in order to avoid poles (where the function becomes infinite).

Yet recently the use of hyperbolic numbers

(2)

lead to the construction of hyperbolic number neural networks. We will describe the generalized Cauchy-Riemann conditions for hyperbolic numbers and show that there exist bounded holomorphic functions of hyperbolic numbers, which are not constant. We give a new example of such a function. They are therefore excellent candidates for holomorphic activation functions for hyperbolic number neural networks [2, 3]. In [3]

it was shown, that hyperbolic number neural networks allow to control the angle of the decision boundaries (hyperplanes) of the real and the unipotent

-part of the output. But Buchholz argued in [4], p. 114, that

Contrary to the complex case, the hyperbolic logistic function is bounded. This is due to the absence of singularities. Thus, in general terms, this seems to be a suitable activation function. Concretely, the following facts, however, might be of disadvantage. The real and imaginary part have different squashing values. Both component functions do only significantly differ from zero around the lines111Note that we slightly correct the two formulas of Buchholz, because we think it necessary to delete in Buchholz’ original , etc. and .

Complex numbers are isomorphic to the Clifford geometric algebra

which is generated by a single vector

of negative square , with algebraic basis . The isomorphism is realized by mapping .

Hyperbolic numbers are isomorphic to the Clifford geometric algebra which is generated by a single vector of positive square , with algebraic basis . The isomorphism between hyperoblic numbers and is realized by mapping .

2 Complex variable functions

We follow the treatment given in [5]. We assume a complex function given by an absolute convergent power series.

(3)

where are real functions of the real variables . Since are obtained in an algebraic way from the complex number , they cannot be arbitrary functions but must satisfy certain conditions. There are several equivalent ways to obtain these conditions. Following Riemann, we state that a function is a function of the complex variable

if its derivative is independent of the direction (in the complex plane) with respect to which the incremental ratio is taken. This requirement leads to two partial differential equations, named after Cauchy and Riemann (CR), which relate

and .

One method for obtaining these equations is the following. We consider the expression only as a function of , but not of , i.e. the derivative with respect to shall be zero. First we perform the bijective substitution

(4)

based on . For computing the derivative

with the help of the chain rule we need the derivatives of

and of (4)

(5)

Using the chain rule we obtain

(6)

Requiring that both the real and the imaginary part of (6) vanish we obtain the Cauchy-Riemann conditions

(7)

Functions of a complex variable that fulfill the CR conditions are functions of and , but they are only functions of , not of .

It follows from (7), that both and fulfill the Laplace equation

(8)

and similarly

(9)

The Laplace equation is a simple example of an elliptic partial differential equation. The general theory of solutions to the Laplace equation is known as potential theory. The solutions of the Laplace equation are called harmonic functions and are important in many fields of science, notably the fields of electromagnetism, astronomy, and fluid dynamics, because they can be used to accurately describe the behavior of electric, gravitational, and fluid potentials. In the study of heat conduction, the Laplace equation is the steady-state heat equation [6].

Liouville’s theorem [1] states, that any bounded holomorphic function , which fulfills the CR conditions is constant. Therefore for complex neural networks it is not very meaningful to use holomorphic functions as activation functions. If they are used, special measures need to be taken to avoid poles in the complex plane. Instead separate componentwise (split) real scalar functions for the real part , and for the imaginary part , are usually adopted. Therefore a standard split activation function in the complex domain is given by

(10)

3 Hyperbolic numbers

Hyperbolic numbers are also known as split-complex numbers. They form a two-dimensional commutative algebra. The canonical hyperbolic system of numbers is defined [5] by

(11)

The hyperbolic conjugate is defined as

(12)

Taking the hyperbolic conjugate corresponds in the isomorphic algebra to taking the main involution (grade involution), which maps .

The hyperbolic invariant (corresponding to the Lorentz invariant in physics for ), or modulus, is defined as

(13)

which is not positive definite.

Hyperbolic numbers are fundamentally different from complex numbers. Complex numbers and quaternions are division algebras, every non-zero element has a unique inverse. Hyperbolic numbers do not always have an inverse, but instead there are idempotents and divisors of zero.

We can define the following idempotent basis

(14)

which fulfills

(15)

The inverse basis transformation is simply

(16)

Setting

(17)

we get the corresponding coordinate transformation

(18)

as well as the inverse coordinate transformation

(19)

The hyperbolic conjugate becomes, due to (15), in the idempotent basis

(20)

In the idempotent basis, using (20) and (15), the hyperbolic invariant becomes multiplicative

(21)

In the following we consider the product and quotient of two hyperbolic numbers both expressed in the idempotent basis

(22)

and

(23)

Because of (23) it is not possible to divide by if , or if . Moreover, the product of a hyperbolic number with (on the axis) times a hyperbolic number with (on the axis) is

(24)

due to (15). We repeat that in (24) the product is zero, even though the factors are non-zero. The numbers along the axis are therefore called divisors of zero. The divisors of zero have no inverse.

The hyperbolic plane with the diagonal lines of divisors of zero (b), and the pairs of hyperbolas with constant modulus (c), and (a) is shown in Fig. 1.

Figure 1: The hyperbolic number plane [9] with horizontal -axis and vertical -axis, showing: (a) Hyperbolas with modulus (green). (b) Straight lines with modulus (red), i.e. divisors of zero. (c) Hyperbolas with modulus (blue).

4 Hyperbolic number functions

We assume a hyperbolic number function given by an absolute convergent power series

(25)

where are real functions of the real variables . An example of a hyperbolic number function is the exponential function

(26)

with

(27)

Since are obtained in an algebraic way from the hyperbolic number , they cannot be arbitrary functions but must satisfy certain conditions. There are several equivalent ways to obtain these conditions. A function is a function of the hyperbolic variable , if its derivative is independent of the direction (in the hyperbolic plane) with respect to which the incremental ratio is taken. This requirement leads to two partial differential equations, so called generalized Cauchy-Riemann (GCR) conditions, which relate and .

To obtain the GCR conditions we consider the expression only as a function of , but not of , i.e. the derivative with respect to shall be zero. First we perform the bijective substitution

(28)

based on . For computing the derivative with the help of the chain rule we need the derivatives of and of (28)

(29)

Using the chain rule we obtain

(30)

Requiring that both the real and the -part of (30) vanish we obtain the GCR conditions

(31)

Functions of a hyperbolic variable that fulfill the GCR conditions are functions of and , but they are only functions of , not of . Such functions are called (hyperbolic) holomorphic functions.

It follows from (31), that and fulfill the wave equation

(32)

and similarly

(33)

The wave equation is an important second-order linear partial differential equation for the description of waves – as they occur in physics – such as sound waves, light waves and water waves. It arises in fields like acoustics, electromagnetics, and fluid dynamics. The wave equation is the prototype of a hyperbolic partial differential equation [7].

Let us compute the partial derivatives , for the exponential function of (26):

(34)

We clearly see that the partial derivatives (34) fulfill the GCR conditions (31) for the exponential function , as expected by its definition (26). The exponential function is therefore a manifestly holomorphic hyperpolic function, but it is not bounded.

In the case of holomorphic hyperbolic functions the GCR conditions do not imply a Liouville type theorem like for holomorphic complex functions. This can most easily be demonstrated with a counter example

(35)

The function is pictured in Fig. 2.

Let us verify that the function of (35) fulfills the GCR conditions

(36)

where we repeatedly applied the chain rule for differentiation. Similarly we obtain

(37)

The GCR conditions (31) are therefore clearly fulfilled, which means that the hyperbolic function of (35) is holomorphic. Since the exponential function has a range of , the product also has values in the range of . Therefore the function has values in , and the components of the function of (35) have values

(38)

We especially have

(39)

and

(40)
Figure 2: Function . Horizontal axis , from left corner into paper plane . Vertical axis . (Figure produced with [8].)

The function (35) is representative for how to turn any real neural node activation function into holomorphic hyperbolic activation function via

(41)

We note that in [3, 4] another holomorphic hyperbolic activation function was studied, namely

(42)

but compare the quote from [4], p. 114, given in the introduction. The split activation function used in [2]

(43)

is clearly not holomorphic, because the real part depends only on and not on , and the -part depends only on and not on , thus the GCR conditions (31) can not be fulfilled.

5 Geometric interpretation of multiplication of hyperbolic numbers

In order to geometrically interpret the product of two complex numbers, it proves useful to introduce polar coordinates in the complex plane. Similarly, for the geometric interpretation of the product of two hyperbolic numbers, we first introduce hyperbolic polar coordinates for with radial coordinate

(44)

The hyperbolic polar coordinate transformation [5] is then given as

  1. , :

    i.e. the quadrant in the hyperbolic plane of Fig. 1 limitted by the diagonal idempotent lines, and including the positive -axis (to the right).

  2. , :

    i.e. the quadrant in Fig. 1 including the negative -axis (to the left).

  3. , :

    i.e. the quadrant in Fig. 1 including the positive -axis (top).

  4. , :

    i.e. the quadrant in Fig. 1 including the negative -axis (bottom).

The product of a constant hyperbolic number (assuming )

(45)

with a hyperbolic number (assuming , ) in hyperbolic polar coordinates is

(46)

The geometric interpretation is a scaling of the modulus and a hyperbolic rotation (movement along a hyperbola) .

In the physics of Einstein’s special relativistic space-time [11, 12], the hyperbolic rotation corresponds to a Lorentz transformation from one inertial frame with constant velocity to another inertial frame with constant velocity . Neural networks based on hyperbolic numbers (dimensionally extended to four-dimensional space-time) should therefore be ideal to compute with electromagnetic signals, including satellite transmission.

6 Conclusion

We have compared complex numbers and hyperbolic numbers, as well as complex functions and hyperbolic functions. We saw that according to Liouville’s theorem bounded complex holomorphic functions are necessarily constant, but non-constant bounded hyperbolic holomorphic functions exist. One such function has already beeng studied in [3, 4]. We have studied a promising example of a hyperbolic holomorphic function

(47)

in some detail. The distinct notions of idempotents and divisors of zero, special to hyperbolic numbers, were introduced. After further introducing hyperbolic polar coordinates, a geometric interpretation of the hyperbolic number multiplication was given.

Hyerbolic neural networks offer, compared to complex neural networks, therefore the advantage of suitable bounded non-constant hyperbolic holomorphic activation functions. It would certainly be of interest to study convergence, accuracy and decision boundaries of hyperbolic neural networks with the activation function (35), similar to [3, 4].

Acknowledgment

I want to acknowledge God [13]:

In the beginning was the Word222Greek term: logos. Note: A Greek philosopher named Heraclitus first used the term Logos around 600 B.C. to designate the divine reason or plan which coordinates a changing universe. [14], and the Word was with God, and the Word was God. He was with God in the beginning. Through him all things were made; without him nothing was made that has been made. In him was life, and that life was the light of all mankind.

I want to thank my dear family, as well as T. Nitta and Y. Kuroe.

References

  • [1] K. Guerlebeck et al, Holomorphic Functions in the Plane and n-dimensional Space, Birkhauser, 2008, chp. 7.3.3.
  • [2] S. Buchholz, G. Sommer,

    A hyperbolic multilayer perceptron

    , Proceedings of the International Joint Conference on Neural Networks, Como, Italy, vol. 2, 129/133 (2000).
  • [3] T. Nitta, S. Buchholz,

    On the Decision Boundaries of Hyperbolic Neurons

    , Proceedings of the International Joint Conference on Neural Networks, IJCNN’08-HongKong, June 1-6, 2973/2979(2008).
  • [4] S. Buchholz, PhD Thesis, A Theory of Neural Computation with Clifford Algebras, University of Kiel, 2005.
  • [5] F. Catoni et al, The Mathematics of Minkowski Space-Time, Birkhauser, 2008.
  • [6] Laplace’s equation, Wikipedia, accessed 24 August 2011, http://en.wikipedia.org/wiki/Laplace’s_equation
  • [7] Wave equation, Wikipedia, accessed 24 August 2011, http://en.wikipedia.org/wiki/Wave_equation
  • [8] Online 3D function grapher, http://www.livephysics.com/ptools/online-3d-function-grapher.php?
  • [9] Split-complex number, Wikipedia, accessed 29 August 2011, http://en.wikipedia.org/wiki/Split-complex_number
  • [10] Notes of collaboration with H. Ishi, Feb. 2011, p. 15.
  • [11] C. Doran and A. Lasenby, Geometric Algebra for Physicists, Cambridge University Press, Cambridge (UK), 2003.
  • [12] E. Hitzer, Relativistic Physics as Application of Geometric Algebra, in K. Adhav (ed.), Proceedings of the International Conference on Relativity 2005 (ICR2005), University of Amravati, India, January 2005, 71/90(2005).
  • [13] The Bible, New International Version (NIV), The Gospel according to John, chapter 1, verses 1-4, http://www.biblegateway.com/
  • [14]

    Strong’s Bible lexicon entry G3056 for

    logos, available online at Blue Letter Bible. http://www.blueletterbible.org/lang/lexicon/lexicon.cfm?Strongs=G3056&t=KJV