1 Reallike Algebras
Let
be nonnegative integer. We say that a vector space
over the real number field is the direct sum of its nonzero subspaces , , …, and we write if each in A can be represented uniquely in the form for for . The subspaces , , …, are called the homogeneous subspaces of . The elements of are said to be homogeneous of degree for . After expressing an element in as a sum of nonzero homogeneous elements of distinct degrees, these nonzero homogeneous elements are called the homogeneous components of and the homogeneous components of of least degree is called the initial component of .We now define reallike algebras which are the kind of real associative algebras we need in the study of automatic differentiation.
Definition 1.1
A commutative associative algebra over the real number field is called a reallike algebra if is the direct sum of its nonzero subspaces , , …, satisfying and for , where for .
The reallike algebras we will used in this paper is the reallike algebra , where
is the quotient associative algebra of the polynomial ring with respect to the ideal generated by the subset of . Clearly, is a reallike algebra. In fact, if then we have
where . The reallike algebra has appeared in automatic differentiation for a long time (see Section 13.2 in [4]). To the best of our knowledge, although is a normed algebra by [4], a normed algebraic structure has not been introduced on the reallike algebra . Since
norm is generally preferred in neural networks and more computational efficient than
norm, it is advantageous to have a normed algebraic structure on the algebras appearing in the study of automatic differentiation. At the end of this section, we give many ways of introducing a norm on the reallike algebra . For convenience, we use numbers to name the elements of the reallike algebra . Clearly, number are the dual numbers introduced by C. L. Clifford in [1]. Based on our research about the applications of the dual numbers, we strongly feel that if there exists a class of new numbers which can be used to extend the known mathematics based on real numbers in a satisfyingly way, then numbers should be the best candidate for this class of new numbers.The following proposition gives the basic properties of reallike algebras.
Proposition 1.1
Let be a reallike algebra and let be an element of with for .
 (i)

is a zerodivisor if and only if .
 (ii)

is invertible if and only if , in which case, the inverse of is given by , where is the matrix obtained by replaying the th column of the matrix
with the matrix and is the determinant of the matrix .
Proof (i) If is a zerodivisor, then for some . Let be the degree of the initial component of . Then we have , where for and . Assume that . By the fact that is in , the inverse of exists. It follows that
(1) 
Since the degree of the initial component of is at least , we have to have by (1), which is impossible. This proves that has to be .
Conversely, if , then . After choosing , we get . This proves that is a zerodivisor.
(ii) is invertible if and only if there exists with for such that , which is equivalent to
or
It follows from that for . Thus (ii) holds.
To study the norm algebraic structure of a reallike algebra, we introduce the concept of a homogeneous norm algebra in the following
Definition 1.2
We say that a reallike algebra has a homogeneous norm if is a function on the set of homogeneous elements of such that for , , and :
 (i)

, and if and only if ,
 (ii)

,
 (iii)

,
 (iv)

.
Let be a reallike algebra which has a homogeneous norm . Mimicking the definitions of the ordinary norm and norm on , we have the following natural extensions and of the function :
(2) 
where with for . The following proposition gives the basic properties of the two realvalued functions and .
Proposition 1.2
Let be a reallike algebra which has a homogeneous norm , and let and be the realvalued functions defined by (2).
 (i)

is a normed algebra with respect to the norm .
 (ii)

If for the identity of the algebra , then can not be made into a normed algebra via the realvalued function .
 (iii)

is a norm on .
Proof Recall that a function is called a norm on if for and , , we have
(3) 
and
(4) 
Also, is called a normed algebra if there is a norm on such that
(5) 
The proof of Proposition 1.2 follows from direct computations. We now proof (ii) to explain the way of doing the computation. Let . Then
(6) 
It follows from (2) and (6) that
or
which proves that (5) fails for .
Obviously, the map defined by
is a homogeneous norm on the reallike algebra which satisfies the assumption in Proposition 1.2 (ii), where is the absolute value of the real number . Hence, the natural idea of extending the ordinary way of defining a norm on can not give a normed algebraic structure on the reallike algebra by Proposition 1.2 (ii). This is possibly why we have not seen the way of making the reallike algebra into a normed algebra in automatic differentiation community even it has a normed algebraic structure. We now give many ways of introducing a normed algebraic structure on the reallike algebra .
Proposition 1.3
Let be a positive constant real numbers. If is the nonnegative real valued function defined by
(7) 
for with for , then makes the reallike algebra into a normed algebra.
Proof For convenience, we set for . Let be a map defined by
(8) 
For , , and , we clearly have
, and if and only if ,  (9) 
(10) 
and
(11) 
We now prove that
(12) 
By (9), (10), (11) and (12), is a homogeneous norm on the reallike algebra . By (7) and (8), we have
(15) 
It follows from (15) and Proposition 1.2 (iii) that is a norm on the reallike algebra .
In order to prove that the reallike algebra is a normed algebra via the norm , we need only to prove
(16) 
For , we define
(17) 
Then the map defined by (17) is an injective algebra homomorphism. Using this algebra homomorphism and the matrix norm which makes into a normed algebra, we get (16).
2 Automatic Differentiation induced by
The following definition is our way of conceptualizing automatic differentiation mathematically.
Definition 2.1
Let be a reallike algebra. A tuple consisting of an algebra homomorphism , a map and a family maps is called the graded automatic differentiation induced by on or the graded automatic differentiation if the following three conditions are satisfied:
 (i)

extends each function in , i.e., for all ;
 (ii)

preserves the invertible real numbers, i.e., is an invertible element of for each nonzero real number ;
 (iii)

preserves the composition of two differentiable functions, i.e.,
(18) and the map for , which is called the th derivative map, has the following property:
(19) where , and
Like the firstorder automatic differentiation which depends on one parameter, which is denoted by in the section 3.1.1 of [5], the higherorder automatic differentiation depends on many parameters. Different choices of these parameters give different ways of doing higherorder automatic differentiation.
We now explain how to get the graded automatic differentiation induced by on .
Let , and be three real constants. For , we define the maps and by
(20) 
and
(21) 
where , , , .
The following theorem, which is the main theorem of this paper, presents the new technique of automatic differentiation to compute the first, the second and the third derivatives exactly and simultaneously.
Proposition 2.1
Proof First, let and be the identity of the algebra and the algebra , respectively. By (30), we have
(23) 
Let , . Clearly, we have
(24) 
Note that
(25) 
and
(26) 
Comments
There are no comments yet.