Optimal translational-rotational invariant dictionaries for images

We provide the construction of a set of square matrices whose translates and rotates provide a Parseval frame that is optimal for approximating a given dataset of images. Our approach is based on abstract harmonic analysis techniques. Optimality is considered with respect to the quadratic error of approximation of the images in the dataset with their projection onto a linear subspace that is invariant under translations and rotations. In addition, we provide an elementary and fully self-contained proof of optimality, and the numerical results from datasets of natural images.

Authors

• 4 publications
• 1 publication
• 1 publication
• 1 publication
• A note on the elementary HDX construction of Kaufman-Oppenheim

In this note, we give a self-contained and elementary proof of the eleme...
12/24/2019 ∙ by Prahladh Harsha, et al. ∙ 0

• A complete characterization of optimal dictionaries for least squares representation

Dictionaries are collections of vectors used for representations of elem...
10/18/2017 ∙ by Mohammed Rayyan Sheriff, et al. ∙ 0

• Certainty Equivalent Control of LQR is Efficient

We study the performance of the certainty equivalent controller on the L...
02/21/2019 ∙ by Horia Mania, et al. ∙ 8

• Optimal dictionary for least squares representation

Dictionaries are collections of vectors used for representations of rand...
03/07/2016 ∙ by Mohammed Rayyan Sheriff, et al. ∙ 0

• Similarity of particle systems using an invariant root mean square deviation measure

Determining whether two particle systems are similar is a common problem...
06/17/2021 ∙ by Johannes Bulin, et al. ∙ 0

• A Liouville principle for the random conductance model under degenerate conditions

We consider a random conductance model on the d-dimensional lattice, d∈[...
08/28/2019 ∙ by Tuan-Anh Nguyen, et al. ∙ 0

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The purpose of this work is to present the theoretical solution, and outline the numerical implementation, for an optimal approximation problem on digital images. The problem is the following: given a dataset of square images, we want to find the optimal generators that provide, by translations and 90 degrees rotations, the best approximation of the dataset with respect to a quadratic error.

We are not considering the full set of all possible translations, which would give rise to a convolutional problem, but rather we consider translations on a lattice. Together with rotations, they will define a nonabelian semidirect product group of discrete Euclidean rigid movements of images.

This work is an adaptation of a result obtained for more general groups in [BCHM2019], which nevertheless can not be directly applied to this setting. Our approach to invariant approximation borrows several ideas from the theory of approximation by shift-invariant spaces developed in [ACHM2007], see also [AT2011, CMP2017]. Noncommutative problems of harmonic analysis related to group actions have a long tradition in signal processing, and recent works with relevant interactions with the present one are [BHP2015, BHP2018, GHP2018, GHI2018].

The presence of invariances in natural images has been long studied and exploited in vision (see [CS2006, B2015, PA2016

] and references therein), and it plays a central role in several approaches to machine learning [

Bekkers2018, AERP2019]. In particular, the solution presented in this work makes use of a special form of a data augmentation, a classical technique now of common use for networks training (see e.g. [DM2001, google2019] and references therein). Our approach differs in a fundamental way from patch-based ones such as [OF1996, AEB2006], because we do not extract patches from images, to be then used by translations, but rather consider entire images, and find the optimal generators for a fixed set of translations and rotations with methods of Fourier analysis.

The structure of this paper is the following. In Section 2 we describe the group invariance, focusing on invariant subspaces, and provide a formal statement of the approximation problem. In Section 3 we introduce an isometric isomophism that allows us to treat the group symmetries with Fourier analysis, and study how invariant subspaces behave under such a map. In Section 4 we provide a formal statement of the proposed solution, and outline the algorithm that allows us to compute it. In Section 5 we finally show the numerical results on a well-known dataset of natural images.

Most of the theoretical results presented in this paper could be deduced, without major difficulties, from the ones obtained in [BCHM2019]. The only obstruction from applying them directly to the present setting is due to the fact that the hypotheses for Proposition 4.1 of [BCHM2019] are not met here due to the presence of certain nontrivial stabilizers for the group action. In Section 3 we overcome this issue by defining an isometry that is slightly different from the one introduced in Section 4.2 of [BCHM2019]. The present setting represents a great simplification of the general case, mainly due to the finiteness of the problem. This gives us the possibility to present a fully self-contained approach to the solution. Indeed, although the arguments used to solve this problem refer to much more general principles, and could be proved with more abstract techniques, in the present case it is possible to provide full proofs of all the results needed to construct the desired approximation with only elementary techniques. We have chosen to do so with the intention of making this work accessible to the non specialist reader, which may be interested in applying this technique.

2 The invariant approximation problem

2.1 Group Invariance

We will consider grayscale digital images of

pixels, and we will treat here the case of an odd number

. It is convenient, for the purposes of this work, to consider a digital image as a function on the square lattice in centered coordinates

i.e. . This space is , indexed by , and endowed with the Euclidean norm, that we denote by , associated to the inner product

 ⟨f,f′⟩d=d−12∑n1=−d−12d−12∑n2=−d−12f(n1,n2)¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯f′(n1,n2),f,f′∈ℓ2(Zd×Zd). (2.1)

In particular, is the Frobenius norm of viewed as a matrix.

Note that, for simplicity, we allow ourselves the slight abuse of keeping the same notation commonly used for . For any we will also keep the additive notation, and denote always by the periodic sum

 n+n′=((n1+n′1+d−12)modd −d−12,(n2+n′2+d−12)modd −d−12). (2.2)

With this operation, is an abelian group, and 90 degrees rotations, defined by the linear action on of

 r=(0−110), (2.3)

are automorphisms. This can be easily checked because and, using (2.2), . For we denote by the -th power of the matrix , so corresponds to a 180 degrees rotation, etc. Note that is the identity: is a cyclic group of order 4.

Moreover, whenever is not a prime number, admits nontrivial proper subgroups for which 90 degrees rotations are also automorphisms of these subgroups. We give a precise statement in the next lemma.

Lemma 2.1

Let be odd, and let .

• is a subgroup, isomorphic to .

• is invariant for , i.e. , and for we have .

On images, and on every , the symmetries of translations and rotations are formally described as follows. Let be a subgroup. For , the translation of by , denoted by , is

 T(λ)f(n)=f(n−λ)n∈Zd×Zd

where the operation is intended as in (2.2). The 90 degrees rotation of , denoted by , is

 Rf(n1,n2)=f(r(n1,n2))=f(−n2,n1)

where is the 90 degrees rotation of given by (2.3). For we denote by the -th power (iteration) of the operator . The set of operators define a unitary representation of the nonabelian group of discrete Euclidean rigid movements, i.e. -translations and 90 degrees rotations on . The composition law of can be written as

 (λ,g)⋅(λ,g′)=(λ+r−g(λ′),g+g′),

for and . Indeed,

 T(λ)RgT(λ′)Rg′f(n)=f(rg′(rg(n−λ)−λ′))=T(λ+r−g(λ′))Rg+g′f(n),n∈Zd×Zd, (2.4)

and, for all , and all we have . Note that we are always using the composition (2.2) for variables, while periodic composition is used for the rotation variables, i.e. is considered mod 4.

By general arguments, see [BHP2018, Lemma 11], a subspace is invariant under the action of the group , i.e. it is such that for all , if and only if it is linearly generated by the

-orbit of a set of vectors of

. This is the object of the next definition.

Definition 2.2

For a set of generators, we denote by

 S(Ψ)=span{T(λ)Rgψj:λ∈Λ,g∈{0,1,2,3},j∈{1,…,κ}}

the -invariant linear subspace of generated by the action of on .

2.2 Best Approximation

The best approximation problem solved in this paper is the following. Suppose we are given a dataset of digital images. For , we want to find a family such that is a Parseval frame of , i.e. such that the orthogonal projection of onto can be written as

 PS(Φ)f=∑λ∈Λ3∑g=0κ∑j=1⟨f,T(λ)Rgϕj⟩dT(λ)Rgϕj,f∈ℓ2(Zd×Zd).

Moreover, we want this projection to minimize the quadratic error resulting from the projection onto a -invariant space with generators, i.e.

 m∑i=1∥fi−PS(Φ)fi∥2d=min{m∑i=1∥fi−PS(Ψ)fi∥2d:Ψ={ψj}κj=1⊂ℓ2(Zd×Zd)}, (2.5)

where is the orthogonal projection of onto .

The solution to this problem will be provided in Section 4 together with the construction that allows us to compute an optimal set of Parseval frame generators, given in Table 1.

Observe that the dimension of a -invariant space with generators is at most If and is as in Lemma 2.1, then . Thus, we will always consider families of generators with cardinality Indeed, this implies that the dimension of is certainly smaller than , that is the dimension of , and the ratio approximately quantifies the dimensionality reduction that one obtains when replacing with .

3 Fourier analysis of invariance

Fourier duality on is provided by the DFT:

 ˆf(k1,k2)=d−12∑n1=−d−12d−12∑n2=−d−12f(n1,n2)e−2πin1k1+n2k2d,k1,k2∈{−d−12,…,d−12}.

We will write, for short,

 ˆf(k)=∑n∈Zd×Zdf(n)e−2πin.kd,k∈Zd×Zd

where . The DFT is a multiple of an invertible isometry: it satisfies

 ∥ˆf∥d=d∥f∥d,

 f(n)=1d2∑k∈Zd×Zdˆf(k)e2πik.nd ,n∈Zd×Zd.

It is easy to see by direct computation that the DCT intertwines translations with a phase factor, i.e.

 ˆT(λ)f(k)=e−2πiλ.kdˆf(k),

while it commutes with rotations, i.e.

 ˆRf(k1,k2)=ˆf(−k2,k1)=Rˆf(k1,k2). (3.1)

For a subgroup of , its annihilator is defined by

 L={ℓ∈Zd×Zd:e2πiλ.ℓd=1  ∀ λ∈Λ}⊂Zd×Zd.

is a subgroup of that plays a special role in Fourier analysis. The next lemma, whose proof is elementary, defines its structure and how it relates with rotations.

Lemma 3.1

Let be odd, let , and let be as in Lemma 2.1.

• The annihilator of is .

• The set is a fundamental set for , in the sense that

• for all ;

• .

• is invariant for , i.e. , and for we have .

• Let . The set satisfies

• for all ;

• .

The following definition introduces a transform, denoted by , that is well adapted to perform Fourier analysis on in the presence of the action of the group . This transform is a variation of the one introduced in Section 4.2 of [BCHM2019].

Definition 3.2

Let , with odd, and let and be as in Lemma 3.1. Let be the linear map on defined by

 N(f)(n)=1d2⎛⎝12∑ℓ∈Le2πiℓ.ndˆf(ℓ)+∑k∈Zd×Zd∖Le2πik.ndˆf(k)⎞⎠ ,n∈Zd×Zd

or, equivalently, . For any we define

 TΓ[f](ω)={1dˆRgN(f)(ω+ℓ):g∈{0,1,2,3},ℓ∈L} (3.2)

which belongs to . For each we denote the corresponding element of by

 TΓ[f](ω)g={TΓ[f](ω)gℓ:ℓ∈L}

where is the component of :

 TΓ[f](ω)gℓ=1dˆRgN(f)(ω+ℓ)=1d⎧⎪⎨⎪⎩12 ˆRgf(ℓ) ,ω=0ˆRgf(ω+ℓ) ,ω≠0.

We provide now a proof of the following result, which also clarifies the role of the map .

Theorem 3.3

The map (3.2) is an invertible isometry . For , its inverse is

 T−1Γ[F](n)1d3∑g=0⎛⎝12∑ℓ∈Le2πirg(ℓ).ndF(0)gℓ+∑ℓ∈Le2πirg(ℓ).nd∑ω∈Ω0e2πirg(ω).ndF(ω)gℓ⎞⎠. (3.3)

Proof. Let us first prove that is an isometry. Using (3.1) and Lemma 3.1

 ∑ω∈Ω0∪{0} ∥TΓ[f](ω)∥2ℓ2({0,1,2,3}×L)=1d2∑ω∈Ω0∪{0}∑ℓ∈L∑g∈Γ|ˆRgN(f)(ω+ℓ)|2 =1d2∑g∈Γ∑ℓ∈L|ˆRgN(f)(ℓ)|2+1d2∑g∈Γ∑ω∈Ω0∑ℓ∈L|ˆRgN(f)(ω+ℓ)|2 =1d2∑g∈Γ∑ℓ∈L|ˆN(f)(ℓ)|2+1d2∑g∈Γ∑ω∈Ω0∑ℓ∈L|ˆN(f)(rgω+ℓ)|2 =1d2⎛⎝4∑ℓ∈L|ˆN(f)(ℓ)|2+∑ω∈Ω∖{0}∑ℓ∈L|ˆN(f)(ω+ℓ)|2⎞⎠=1d2∑ω∈Ω∑ℓ∈L|ˆf(ω+ℓ)|2 =1d2∥ˆf∥2d=∥f∥2d.

To prove the inversion formula, again using Lemma 3.1

 f(n) =1d2∑k∈Zd×Zde2πik.ndˆf(k)=1d2∑ℓ∈Le2πiℓ.nd∑ω∈Ωe2πiω.ndˆf(ω+ℓ) =1d2∑ℓ∈Le2πiℓ.ndˆf(ℓ)+1d2∑ℓ∈Le2πiℓ.nd3∑g=0∑ω∈Ω0e2πirg(ω).ndˆf(rg(ω)+ℓ) =1d23∑g=0⎛⎝14∑ℓ∈Le2πirg(ℓ).ndˆf(rg(ℓ))+∑ℓ∈Le2πirg(ℓ).nd∑ω∈Ω0e2πirg(ω).ndˆf(rg(ω+ℓ))⎞⎠ =1d23∑g=0⎛⎝12∑ℓ∈Le2πirg(ℓ).ndˆN(f)(rg(ℓ))+∑ℓ∈Le2πirg(ℓ).nd∑ω∈Ω0e2πirg(ω).ndˆN(f)(rg(ω+ℓ))⎞⎠ =1d3∑g=0⎛⎝12∑ℓ∈Le2πirg(ℓ).ndTΓ[f](0)gℓ+∑ℓ∈Le2πirg(ℓ).nd∑ω∈Ω0e2πirg(ω).ndTΓ[f](ω)gℓ⎞⎠

where the last identity is due to (3.1).

The next lemma shows how the isometry intertwines the action of .

Lemma 3.4

Let be as in Lemma 2.1, and let and be as in Lemma 3.1. For all , all , all and all it holds

 TΓ[T(λ)f](ω)g=e−2πiλ.rg(ω)dTΓ[f](ω)g

and

 TΓ[Rg′f](ω)g=TΓ[f](ω)g+g′.

Proof. Observe first that, since is only a normalization of the Fourier coefficients on the subgroup , it commutes with translations: . Thus, for all

 TΓ[T(λ)f](ω)gℓ =1d(RgN(T(λ)f))∧(ω+ℓ)=1d(N(T(λ)f))∧(rg(ω+ℓ)) =1d(T(λ)N(f))∧(rg(ω+ℓ))=1de−2πiλ.rg(ω)dˆN(f)(rg(ω+ℓ)) =e−2πiλ.rg(ω)dTΓ[f](ω)gℓ,

where the second to last identity makes use of the fact that is the annihilator of and of the invariance of under rotations. This last fact also implies that commutes with rotations: . Thus

 TΓ[Rg′f](ω)gℓ=1d(RgN(Rg′f))∧(ω+ℓ)=TΓ[f](ω)g+g′ℓ.□

3.2 Group invariant spaces

Using the previous results, we can deduce how -invariant spaces are transformed under the action of the map . The next theorem shows that, for any fixed , the element obtained by the transform of a linear combination of translates and rotates of a family is a linear combination of the transform of the rotates of .

Theorem 3.5

Let be as in Lemma 2.1, and let and be as in Lemma 3.1. Let , let be as in Definition 2.2, and let be the map defined in (3.2). For and , let us denote by

 TΓ[S(Ψ)](ω)g={v=TΓ[f](ω)g:f∈S(Ψ)}

and let us denote by

 SΨ(ω)=span{TΓ[ψj](ω)g′:j∈{1,…,κ},g′∈{0,1,2,3}}. (3.4)

Then these two subspaces of coincide:

 TΓ[S(Ψ)](ω)g=SΨ(ω). (3.5)

Proof. Observe first that (3.5) implies in particular that for any , any , and any , the elements and belong to the same subspace of . This, however, can already be deduced by Lemma 3.4, because , and is invariant under rotations.

Let us now prove that for every and every , noting that, by the previous argument, it suffices to prove this only for .

Let . Then, using Lemma 3.4

 TΓ[f](ω)0 =3∑g′=0κ∑j=1Cj(ω,g′)TΓ[ψj](ω)g′∈SΨ(ω) (3.6)

where we have set

 Cj(ω,g′)=∑λ∈Λcj(λ,g′)e−2πiλ.ωd. (3.7)

This proves that for every and every .

To prove the opposite inclusion, fix and let . We want to prove that for each there exists such that . Again, since is rotation invariant, by Lemma 3.4 it suffices to consider only . Now, observe that (3.7) is a DFT for . Indeed since for , equation (3.7) can be written as . Hence, it is inverted by

 cj(λ,g′)=1p2∑ω∈ΩCj(ω,g′)e2πiλ.ωd.

Thus, for we get . Now, by (3.2), the element of given by

 fv=1p2∑λ∈Λ3∑g=0κ∑j=1aj(g′)e2πiλ.ω0dT(λ)Rg′ψj

satisfies . This concludes the proof.

Remark 3.6

The function is a nonabelian variant of an object known in the literature on shift-invariant spaces as range function (see e.g. [Bownik2000, CP2010] and references therein), and, up to a minor change, it corresponds to the map introduced in Definition 4.5 of [BCHM2019]. We note here also that, for the same used in the previous proof, we can easily compute the components of . Using Lemma 3.4 and (3.2)

 TΓ[f](ω)g=3∑g′=0κ∑j=1Cj(rg(ω),g′−g)TΓ[ψj]