Image classification and retrieval for color images are two hotspots in pattern recognition. How to extract effective features, which are robust to color variations caused by the changes in the outdoor environment and geometric deformations caused by viewpoint changes, is the key issue. The classical approach is to construct invariant features for color images. Moment invariants are widely used invariant features.
Moment invariants were first proposed by Hu in 1962. He defined geometric moments and constructed 7 geometric moment invariants which were invariant under the similarity transform(rotation, scale and translation). Researchers applied Hu moments to many fields of pattern recognition and achieved good results[2, 3]. Nearly 30 years later, Flusser et al. constructed the affine moment invariants (AMIs) which are invariant under the affine transform. The geometric deformations of an object, which are caused by the viewpoint changes, can be represented by the projective transforms. However, general projective transforms are complex nonlinear transformations. So, it’s difficult to construct projective moment invariants. When the distance between the camera and the object is much larger than the size of the object itself, the geometric deformations can be approximated by the affine transform. AMIs have been used in many practical applications, such as character recognition and expression recognition. In order to obtain more AMIs, researchers designed all kinds of methods. Suk et al. proposed graph method which can be used to construct AMIs of arbitrary orders and degrees. Xu et al.
proposed the concept of geometric primitives, including distance, area and volume. AMIs can be constructed by using various geometric primitives. This method made the construction of moment invariants have intuitive geometric meaning.
The above-mentioned moment invariants are all designed for gray images. With the popularity of color images, the moment invariants for color images began to appear gradually. Researchers wanted to construct moment invariants which are not only invariant under the geometric deformations but also invariant under the changes of color space. Geusebroek et al. proved that the affine transform model was the best linear model to simulate changes in color resulting from changes in the outdoor environment. Mindru et al.
proposed moment invariants which were invariant under the shape affine transform and the color diagonal-offset transform. The invariants were constructed by using the related concepts of Lie group. Some complex partial differential equations had to be solved. Thus, the number of them was limited and difficult to be generalized. Also, Suk et al. put forward affine moment invariants for color images by combining all color channels. But this approach was not intuitive and did not work well for the color affine transform. To solve these problems, Gong et al.[12, 13, 14] constructed the color primitive by using the concept of geometric primitive proposed in . Combining the color primitive with some shape primitives, moment invariants that are invariant under the shape affine and color affine transforms can be constructed easily, which were named shape-color affine moment invariants(SCAMIs). In , they obtained 25 SCAMIs which satisfied the independency of the functions. However, we find that a large number of SCAMIs with simple structures and good properties are missed in .
In this paper, we propose the general construction formula of shape-color primitives by using partial differentials of each color channel. Then, we use two kinds of shape-color primitives to construct shape-color differential moment invariants(SCDMIs), which are invariant under the shape affine and color affine transforms. We find that the construction formula of SCAMIs proposed in  is a special case of our method. Finally, commonly used image descriptors and SCDMIs are used for image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs proposed in this paper get better results.
2 Related Work
In order to construct image features which are robust to color variations and geometric deformations, researchers have made various attempts. Among them, SCAMIs proposed in  are worthy of special attention. SCAMIs are invariant under the shape affine and color affine transforms. Two kinds of affine transforms are defined by
where SA and CA are nonsingular matrices.
For the color image , let be three arbitrary points in the domain of . The shape primitive and the color primitive are defined by
where represents the mean value of , .
Then, using Eq.(3) and (4), the shape core can be defined by
where n and m represent that the sCore is the product of m shape primitives which are constructed by N points . , , . represents the number of point in all shape primitives, .
Similarly, the color core can be defined by
Where N and M represent that the cCore is the product of M color primitives which are constructed by N points . , , . represents the number of point in all color primitives, .
Suppose the color image is transformed into the image by two transformations defined by Eq.(1) and (2), in are the corresponding points of in . Gong et al. have proved
Further results can be concluded
Therefore, the SCAMIs are constructed by
Then there is a relation
It must be said that , and are named the degree, the shape order and the color order of SCAMIs, respectively. In fact, Eq.(11) can be expressed as polynomial of shape-color moment. This moment was first proposed in  and defined by
Gong et al. proposed that they constructed all SCAMIs of which degrees , shape orders and color orders . They obtained 24 SCAMIs which are functional independencies using the method proposed by Brown . However, we will point out in the Section 3 that they omitted many simple and well-behaved SCAMIs.
3 The Construction Framework of SCDMIs
In this section, we introduce the general definitions of shape-color differential moment and shape-color primitive, firstly. Then, using the shape-color primitive, the shape-color core can be constructed. Finally, according to Eq.(11) and the shape-color core, we obtain the general construction formula of SCDMIs. Also, 50 instances of SCDMIs are given for experiments in the Section 4.
3.1 The Definition of The General Shape-Color Moment
Suppose the color image have the k-order partial derivatives . The general shape-color differential moment is defined by
where , represent the k-order partial derivatives of . represent the mean values of . is the impact function which is defined by
We can find that Eq.(14) and Eq.(13) are identical, when . Therefore, the shape-color moment is a special case of the general shape-color differential moment.
3.2 The Construction of The General Shape-Color Primitive
Suppose the color image have the k-order partial derivatives . are three arbitrary points in the domain of . The general shape-color primitive is defined by
We can find that defined by Eq.(4) is a special case of , when .
3.3 The Construction of The General Shape-Color Core
Using Definition 2, the general shape-color core is defined by
Where , N and M represent that the is the product of M shape-color primitives constructed by N points . , , . represents the number of point in all shape-color primitives, .
Obviously, defined by Eq.(6) is a special case of , when .
3.4 The Construction of SCDMIs
Let the color image be transformed into the image by Eq.(1) and Eq.(2), in are corresponding points of in , respectively. Suppose that have the k-order partial derivatives . Then there is a relation
Further, the following relation can be obtained
By using Maple2015, the proof of Theorem 1 is obvious. We can find that Eq.(10) is a special case of Eq.(20), when . So, when we replace in Eq.(11) with , Eq.(12) is still tenable. Now, we can define SCDMIs.
Then there is a relation
Eq.(23) can be expressed as polynomial of . Eq.(11) is a special case of Eq.(23) when . The proof of Eq.(24) is exactly the same as that of Eq.(12) proposed in .
3.5 The Instances of SCDMIs
We can use Eq.(24) to construct instances of by setting different k values. However, color images are discrete data, the partial derivatives of each order can’t be accurately calculated. With the elevation of order, the error will be more and more, which will greatly affect the stability of SCDMIs. So, we set in this paper.
When , are equivalent to SCAMIs. We construct of which degrees , shape orders and color orders . Gong et al.  thought that in order to obtain the of which degrees , shape orders and color orders , must be . This judgment is wrong. In fact, can be , , and . Thus, lots of were missed in . By correcting this shortcoming, we get 25 that satisfy the independence of the function by using the method proposed by .
At the same time, when , is defined by
By replacing , , and with , , and , 25
can be obtained. Therefore, we can construct the feature vector SCDMI50, which is defined by
The construction methods of 50 instances are shown in Table 1.