# Spatial and color hallucinations in a mathematical model of primary visual cortex

We study a simplified model of the representation of colors in the primate primary cortical visual area V1. The model is described by an initial value problem related to a Hammerstein equation. The solutions to this problem represent the variation of the activity of populations of neurons in V1 as a function of space and color. The two space variables describe the spatial extent of the cortex while the two color variables describe the hue and the saturation represented at every location in the cortex. We prove the well-posedness of the initial value problem. We focus on its stationary, i.e. independent of time, and periodic in space solutions. We show that the model equation is equivariant with respect to the direct product G of the group of the Euclidean transformations of the planar lattice determined by the spatial periodicity and the group of color transformations, isomorphic to O(2), and study the equivariant bifurcations of its stationary solutions when some parameters in the model vary. Their variations may be caused by the consumption of drugs and the bifurcated solutions may represent visual hallucinations in space and color. Some of the bifurcated solutions can be determined by applying the Equivariant Branching Lemma (EBL) by determining the axial subgroups of G . These define bifurcated solutions which are invariant under the action of the corresponding axial subgroup. We compute analytically these solutions and illustrate them as color images. Using advanced methods of numerical bifurcation analysis we then explore the persistence and stability of these solutions when varying some parameters in the model. We conjecture that we can rely on the EBL to predict the existence of patterns that survive in large parameter domains but not to predict their stability. On our way we discover the existence of spatially localized stable patterns through the phenomenon of "snaking".

## Authors

• 1 publication
• 1 publication
• 2 publications
12/06/2020

### Dynamics of solutions in the generalized Benjamin-Ono equation: a numerical study

We consider the generalized Benjamin-Ono (gBO) equation on the real line...
10/11/2017

### Solutions of Quadratic First-Order ODEs applied to Computer Vision Problems

The article proves the existence of a maximum of two possible solutions ...
12/30/2020

### Numerical study of soliton stability, resolution and interactions in the 3D Zakharov-Kuznetsov equation

We present a detailed numerical study of solutions to the Zakharov-Kuzne...
11/15/2012

### Color scales that are effective in both color and grayscale

We consider the problem of finding a color scale which performs well whe...
12/06/2002

### Optimized Color Gamuts for Tiled Displays

We consider the problem of finding a large color space that can be gener...
12/28/2021

### Inhomogeneous spatial patterns in diffusive predator-prey system with spatial memory and predator-taxis

In this paper, we consider a diffusive predator-prey system with spatial...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Neural Fields are a useful mathematical formalism for representing the dynamics of cortical areas at a macroscopic level, see the reviews [24, 9, 19]. This formalism has been broadly used to account for the observed activity of the cortical visual area V1. V1 receives inputs from the retina through the LGN. There is a massive feedback from V1 to the LGN. V1 also sends inputs to higher level cortical visual areas such as V2 and V4 and receives feedback signals from them. For a very accessible introduction to the various visual areas, the reader is referred to the book of David Hubel [35], and to [38] for a more recent presentation.

Neural Fields models of V1 are sets of integro-differential equations whose solutions are meant to describe its spatio-temporal activity. The well-posedness of these equations has been studied in depth by various authors [3, 27, 65, 49] with a special attention to the stationary solutions, i.e. those which do not depend upon time. These solutions, also called persistent states, are interesting because they appear to provide good models of memory holding tasks on the time scale of the second [18, 28, 42]. Moreover, they appear to resonate with the fascinating phenomenon of visual hallucinations and their relation with the functional architecture of the visual cortex [25, 10] .

With no exception to our knowledge, all previous work in neural fields theory has not taken into account the chromatic aspects of visual perception. In [57], we introduced a neural field model for color perception to explore how the synergy of two antagonistic phenomena, simultaneous contrast and chromatic assimilation, could lead to a "color sensation".

In the present article, we address the questions of how this model can predict visual hallucinations and what are their spatial and chromatic structures. We are guided in this venture by our previous work [64] and make good use of the theory of equivariant bifurcations.

It is structured as follows. In Section 2 we recall the neural field model described in [57] and prove its well-posedness, Section 3 introduces the notion of stationary solutions and their bifurcations. Section 4 is dedicated to the computation of the spectrum of the linear operator in the neural field model. Section 5 describes the symmetries of the model and uses the equivariant branching lemma to predict the type of bifurcations at the primary bifurcation points and the shape of the bifurcated solutions, or planforms. Section 6 shows examples of such planforms. Section 7 goes away from this local analysis and explores numerically, thanks to the development of innovative software, a much larger volume of the set of stationary solutions to the neural field equations. We conclude in Section 8.

## 2 The model

In this work we think of V1 as the closure of a regular domain, noted , stands for space, of , in effect the open square , where is a positive number which, for simplicity and without loss of generality, we take equal to 1, except for some of the numerical experiments presented in Section 7.

The visual cortex is organized into hypercolumns, groups of neurons sharing the same receptive field in the retina and coding for specific physical quantities such as edge orientation, spatial frequency, temporal frequency. These signals are mapped from the retina to V1 following an approximately log polar retinotopic transformation (see Remark 7.1). Unlike in the case of orientation, for which the existence of such hypercolumns in V1 is now well established [58], the anatomical and physiological bases for a functional architecture encoding color are still debated. These bases are most likely connected to the presence of blobs [34, 35]. Hence, in light of the promising findings made by [70, 13] it is reasonable to assume in our work a hypercolumnar organisation of cells tuned to a continuum of colors. Our work also supposes the presence of long-range lateral connections between hypercolumns, in agreement with observations of [41] where horizontal connections tend to link blobs to blobs. Note that this visual information is stored in the cortex in three dimensions, i.e. the cortex has some thickness. We neglect in our work this thickness and consider only its spatial extent.

We now briefly recall the model described in [57]. It is based on an opponent representation of colors such as Hering’s opponent space [32]. In this setting, a color is a pair of real numbers which encode the chromaticity of the color111We do not consider the achromatic component of a color, except for display, see Section 7.1, and restrict ourselves to the chromatic components.. Details are provided in Appendix A. The reader can think of as encoding the yellow-blue colors and as encoding the red-green colors in Hering’s theory. What is important for us is that the set of chromaticities is symmetric w.r.t the origin, i.e. if is a chromaticity, then is also a chromaticity, called the opponent chromaticity or color of . The set of chromaticities is therefore a bounded regular domain of which is symmetric w.r.t the origin, in effect the open disk centered at the origin and of radius , where is a positive number which we take without loss of generality and for convenience equal to 1. We implicitly assume that the topology of the chromaticity space is that of the Euclidean plane. This is only a coarse approximation. Note that the problem of defining a metric in color space is still open [69, 40, 50, 6, 61]. For technical reasons, we consider only the open disk minus a radius, i.e. . We use polar coordinates to parametrize . A chromaticity is therefore represented by a pair with and , keeping in mind that the actual polar coordinates are . The elimination of the semi open radius from

is practically not important since the functions that we will manipulate, in particular the eigenfunctions of the operator

, see below, will be smooth and therefore can be continuously extended to the closed disk . The perceptual interpretation of is the saturation of the color while is its hue. The opponent chromaticity of is therefore . The set of chromaticities is therefore the open square .

We define

 Ω=Ωs×Ωc

to be the bounded domain, in effect the open rectangle of , encapsulating the spatial and chromatic coordinates that will be of interest in the sequel.

### 2.1 Connectivity kernel

Putting all this together, at each point of , we consider a neural mass222A neural mass is an aggregate of neurons in which the spatial dimension is ignored, see e.g. [67]. whose average membrane potential is noted . It is a function defined on , being an interval of containing 0. In [57], we assumed that the function was the solution to an initial value problem333More precisely we used instead of the "activity" variable , see the definition (2

) of the sigmoid function below. The two formulations are equivalent in the case

considered here. related to a Hammerstein equation444In mathematical neuroscience this integro-differential equation is often called a Wilson-Cowan equation [68] which writes

 τ∂V∂t(r,c,t)=−V(r,c,t)+w⋆Υ(V(r,c,t))+Iext(r,c,t), (1)

together with the initial condition .

This equation describes the time variation of the scalar function defined on starting from the initial condition . At each time , belongs to some functional space, in effect a Hilbert space , that we describe in the next section.

We now discuss the various elements that appear in this equation.

is a time constant that defines the speed of the exponential decay toward the initial condition. Without loss of generality we can assume .

is a sigmoidal function mapping to the open interval

. It is called the activation function, relating the values of the membrane potential

to the neuronal activity (0 meaning quiet, 1 meaning highly active). It writes

 Υ(x)=Sig(γx−ε) where Sig(x)=11+e−x. (2)

is a parameter that allows us to shift the origin, controls the slope of the sigmoid at the (shifted) origin, it is often called the nonlinear gain.

is a function representing the input to the neural mass from different brain areas. In the remaining of this paper we take , i.e. we consider that area is isolated from the rest of the brain. This is clearly an approximation but allows us to do some mathematics and is a first step toward the analysis of the general case.

is the connectivity kernel555It is only qualitatively the same as the one in [57].. It models the influence of neighboring neural masses at on the neural mass at as a (separable in space and color) linear superposition operation

 w⋆s(V(r,ρ,φ,t))=∫Ωs×Ωcw(r,ρ,φ,r′,ρ′,φ′)Υ(V(r′,ρ′,φ′,t))ρ′dρ′dφ′dr′, (3)

with

 w(r,c,r′,c′)=ws(r−r′)wc(ρ,ρ′,φ−φ′),

where the index stands for “space” and the index stands for “chromaticity”.

If is positive (respectively negative) the neural mass at excites (respectively inhibits) the one at . The product of and is intended to model the antagonistic effects of color assimilation and contrast which are parts of the class of perceptual phenomena called chromatic interactions [66].

is a “classical” two-dimensional “Mexican hat” function, see [9] and Figure 7-Left, which we write as the difference of two circularly symmetric Gaussians centered at 0:

 ws(r)=μse−∥r∥22/2α2s−νse−∥r∥22/2β2s, (4)

where is the usual Euclidean norm in . Biology dictates that , are very small w.r.t , the size of . This indicates that our model only takes into account the neural connections which are local to the visual area V1 and take place within the gray matter while it is known that different visual areas communicate through the fiber bundles (in the biological sense) forming part of the white matter. This would be part of the term in (1) which we have taken to be equal to 0.

If , i.e. if , the neural masses at such that is small enough excite the neural mass at , and if those sufficiently far away inhibit it.

We furthermore assume

 ∫Ωsws(r)dr=0, (5)

i.e. that the spatial excitation and inhibition are balanced. This is both compatible with some biological evidence (balanced networks [20]) and mathematically convenient.

The function is separable in polar coordinates,

 wc(ρ,ρ′,φ)=wc,m(ρ,ρ′)wc,a(φ), (6)

with

 wc,m(ρ,ρ′)=e−ξc|ρ2−ρ′2|, (7)

and

 wc,a(φ)=μce−2παc|(φ−12mod1)−12|−νce−2πβc|(φmod1)−12| (8)

The parameters , , , and are positive and such that is positive if is close to and negative if is close to , i.e. in terms of polar coordinates if is close to and either is close to or is close to .

Note that the function is 1-periodic and even. For technical convenience we perform the change of variable so that simplifies to

 wc,m(ϱ,ϱ′)=e−ξc|ϱ−ϱ′|, (9)

and the measure on is equal to . A chromaticity is therefore represented by a pair , .

The product of and behaves as a function of as shown in Table 1 adapted from [57].

As pointed out in this paper, this is in qualitative agreement with the nonlinear behaviour of color shifts found in [44, 43].

Note that by definition of and the function is symmetric w.r.t. and , and and , respectively.

### 2.2 Choice of the appropriate functional space F and well-posedness of (1)

Our choice of is guided by three criteria:

1. The well-posedness of the problem,

2. Its biological relevance,

3. Its suitability for numerical computations.

The choice of a Hilbert space is appealing and a natural choice is . As argued in [65], this unfortunately allows the membrane potential to be singular since for example the function . It is desirable that the average membrane potential stays bounded on the cortex and a way to achieve this is to allow for more spatial and chromatic regularity by assuming that is differentiable almost everywhere.

The choice of the Sobolev space , is convenient for two reasons. First it is a Hilbert space endowed with the usual inner product:

 ⟨V1,V2⟩F=m∑|α|=0⟨DαV1,DαV2⟩L2(Ω)∀V1,V2∈F (10)

where the multi-index is a sequence of 4 integers and , and the symbol represents a partial derivative, e.g.,

 DαV=∂α1+⋯+α4∂rα11∂rα22∂ϱα3∂φα4V

The second reason is that, because the boundary of is sufficiently regular (it satisfies the cone-property [1, 2]), is a commutative Banach algebra for pointwise multiplication [1][Chapter V, Theorem 5.23].

This is necessary in the upcoming bifurcation analysis in order to apply the Equivariant Branching Lemma, see Proposition 5, for which we require some smoothness of

. The reader probably wonders what is the value of

. In order to have the Banach algebra property, we need to have , being the dimension of , i.e. 4. Hence the smallest possible value of is 3. But in Section 7 we assume that is one-dimensional making equal to 3 and the smallest possible value of is equal to 2 in this case.

We next define the operator as acting on as follows. Let , we define

 W⋅U(r,c):=w⋆U(r,c)=∫Ωws(r−r′)wc(c,c′)U(r′,c′)dr′dc′

Remember that the measure is in effect equal to . It is clear that this is well defined and we have the following proposition. The operator maps to and hence to . The proof follows from Proposition 2.3 in [65].

We also have the following Proposition about the solutions to (1). For each there exists a unique solution in to the following Cauchy problem

 {dVdt=−V+W⋅Υ(V)V(0)=V0, (11)

where the function is defined by (2). The proof is a an immediate consequence of Proposition 2.5 in [65].

## 3 Stationary solutions and bifurcations thereof

In this paper, we focus on the stationary (independent of time) solutions to (11), the steady-states. They are important because they are thought to be good models of the memory holding tasks on the timescale of the second as demonstrated experimentally on primates [18, 28, 42]. These solutions may change drastically when some of the parameters such as and in (2), , , , in (4), in (7), and , , , in (8) vary. In effect, we will concentrate on the first parameter, the nonlinear gain , which is important in determining the relation between neuronal activity (a number between 0 and 1) and the associated membrane potential . It is known that the ingestion of drugs such as LSD and marijuana a) can change this relation and b) can trigger hallucinatory patterns [47, 54]. It is therefore very much worth our efforts to investigate if, when varying the parameter , stationary solutions to (11) do bifurcate and if the bifurcated solutions resemble some of the known visual illusions.

To summarize, we are going to study the bifurcations when varies of the solutions to the equation

 F(V,γ)=0, (12)

where the operator is defined by

 Fdef.={Wm,2(Ω)×R⟶L2(Ω)(V,γ)⟶−V+W⋅Υ(V). (13)

In order to achieve such a task, we need to determine the spectrum of the operator and the symmetry properties of the operator with respect to some groups of transformations of . We describe the spectrum of in Section 4 and the symmetry properties of in Section 5.

It is natural at this point to introduce the two operators and acting on associated to the functions of (4) and of (6) as follows

 Ws⋅U(r,c)=∫Ωsws(r−r′)U(r′,c)dr′andWc⋅U(r,ϱ,φ)=12∫Ωcwc(ϱ,φ,ϱ′,φ′)U(r,ϱ′,φ′)dϱ′dφ′.

The separability of the function in the space and color variables is reflected in the notation . Fubini’s Theorem dictates that is also equal to .

Similarly the separability of the function as the product of in (7) for the magnitude and in (8) for the angle implies that is itself separable, :

 Wc,m⋅U(r,ϱ,φ)=12∫10wc,m(ϱ,ϱ′)U(r,ϱ′,φ)dϱ′% andWc,a⋅U(r,ϱ,φ)=∫10wc,a(φ−φ′)U(r,ϱ,φ′)dφ′ (14)

where the convolution in the right hand side of the definition of is a periodic convolution.

## 4 The spectrum of W in L2(Ω)

The reader can verify that, given the symmetry properties of the functions and , the operator is symmetric in , i.e. satisfies

 ⟨U1,W⋅U2⟩L2(Ω)=⟨W⋅U1,U2⟩L2(Ω).

Another important property of is that it is compact, i.e. given any bounded sequence of functions in the sequence contains a converging subsequence for the norm of . This is a direct consequence of Lemma 2.4 in [65].

A consequence of the fact that

is compact is that its spectrum is compact and at most countable. Moreover each point in the spectrum, except perhaps 0, is isolated. All the non zero elements of the spectrum are eigenvalues and all eigenvalues are real since

is symmetric.

Since is separable in space and color, the spectrum of is defined by the spectra of and : the eigenvalues are the product of those of each operator and the eigenfunctions are separable in space and color, being the products of those of and .

### 4.1 The spectrum of Ws in L2(Ωs)

Because we are studying the symmetry-breaking steady-state bifurcations of the solutions to our model, we know from previous work on heat-conduction in fluids [14] or on visual hallucinations [10] that this leads to the formation of spatially periodic patterns.

This brings into play the lattice

generated by the two vectors

and where is the canonical basis of and commands that we quotient by thus obtaining the 2-torus . The dual lattice is also generated by the two vectors and .

We thus make the following assumption about the solutions to equations (1) which is inspired by biology with the advantage that it simplifies the mathematics:

###### Hypothesis 1

The biologically relevant solutions to (1) are -periodic i.e. satisfy

 V(r+l,c,t)=V(r,c,t)∀l∈L

We now work on the Hilbert space of -periodic functions with the same inner product as before. We note the operator restricted to this space i.e. defined by

 ~Ws⋅V(r)=∫Ωsws(r−r′modL)V(r′)dr′∀r∈Ωs,

Note that the spatial convolution in (3) is now a periodic convolution. is clearly a symmetric compact operator on .

It is easy to characterize the spectrum of : The eigenvalues , , of are the (real) Fourier coefficients of the -periodic even function :

 λsm,n=∫Ωsws(r)cos2π⟨km,n,r⟩dr,

where

 km,ndef.=mk1+nk2.

Because is even, so is the sequence of eigenvalues: , . For

the eigenspaces of

are of even dimension, larger than or equal to 2. If , because of (5), and the dimension of the kernel of is 1. Given an eigenvalue the corresponding eigenspace is generated by the functions:

 cos2π⟨kp,q,r⟩andsin2π⟨kp,q,r⟩,

for all such that . This follows for example from the fact that

is diagonalized by the Fourier Transform.

### 4.2 Spectrum of Wc in L2(Ωc)

Define the operator acting on the -periodic functions of period as

 τc⋅u(ϱ,φ)=u(ϱ,−φmod1) (15)

We have the following Proposition. The linear operator on is symmetric and compact, it commutes with the operator defined by (15). Obvious from the definitions. From this we have The spectrum of is real and at most countable. The eigenfunctions are separable w.r.t. , . The first assertion follows from the symmetry and compactness of . The second follows from the definitions (6)-(8). It remains to characterize the eigenfunctions and eigenvalues. The eigenspaces of are generated by the functions and , . Hence they are one-dimensional for , two-dimensional otherwise. The corresponding eigenvalues are the (real) Fourier coefficients of the -periodic even function given by (8) of index . This follows immediatly from the fact that the operator is a -periodic convolution and the -periodic function is even. Regarding , we have the following Proposition. The eigenspaces of corresponding to the non zero eigenvalues are one-dimensional. Its kernel is reduced to the null function. The eigenvalues , , are obtained from the countable solutions to the transcendental equation

 tanx=2ξcxx2−ξ2cx>0 (16)

from the relation

 λcn=ξcξ2c+x2n (17)

The eigenfunction corresponding to the eigenvalue is given by

 ec,mλn(ϱ)=ξcsinxnϱ+xncosxnϱ (18)

We have, according to (9) and (14)

 Wc,m⋅u(ϱ)=12∫10e−ξc|ϱ−ϱ′|u(ϱ′)dϱ′

Deriving twice w.r.t.

 (Wc,m⋅u)(2)(ϱ)=ξ2cWc,m⋅u(ϱ)−ξcu(ϱ),

so that if is an eigenfunction corresponding to the eigenvalue we obtain

 λcu(2)(ϱ)=ξc(ξcλc−1)u(ϱ),

which shows that if , . If we define . The eigenfunctions are solutions to

 u(2)(ϱ)=Ku(ϱ) (19)

which are of the form for some constants and , real or complex. , and are determined by writing that the function thus defined is indeed an eigenfunction of corresponding to the eigenvalue .

It can be verified that if , i.e. if , there are no solutions. Hence we must assume and hence . The solutions to (19) write

 u(ϱ)=asin√−Kϱ+bcos√−Kϱ (20)

A symbolic computation system shows that

 Wc,m⋅u(ϱ)=λcu(ϱ)+λc2ξc(eξcϱe−ξc(−a(√−Kcos√−K+ξcsin√−K)+b(√−Ksin√−K−ξccos√−K))+e−ξcϱ(a√−K−bξc))

The two conditions

 −a(√−Kcos√−K+ξcsin√−K)+b(√−Ksin√−K−ξccos√−K)=0,

and

 a√−K−bξc=0 (21)

are necessary and sufficient to guarantee that for all . Since and are not both equal to 0 (otherwise and hence ) we must have

 2ξc√−Kcos√−K+(ξ2c+K)sin√−K=0,

which is equivalent to

 tan√−K=−2ξc√−Kξ2c+K

Letting , the eigenvalues are found by solving for the equation

 tanx=2ξcxx2−ξ2c.

For each value of , the corresponding value of is found to be

 λc=ξcξ2c+x2

The function decreases monotonically from for to 0 when . The corresponding curve therfore intersects the curve representing the -periodic function at a countable number of points , , yielding the eigenvalues of .

According to (20) and (21) the corresponding eigenfunction is

 ec,mλcn(ϱ)=ξcsinxnϱ+xncosxnϱ

## 5 Symmetries of the model and equivariant bifurcations of the solutions

It follows from (5) that is a solution to (12) and (13) for all values of . We are interested in the problem of determining how this solution bifurcates when increases, while allowing us to change somewhat the value of the threshold in (2).

Because of (12), (13) and (2) we have

 dF(0,γ)=−Id+γSig(−ε)W,

Let be the largest positive eigenvalue of . When one increases from 0 to the solution bifurcates to another solution.

We investigate the properties of the operator defined by (13) under the action of the Euclidean group restricted to the lattice defined at the beginning of Section 4.1 for the spatial part, and the group for the color part. This group arises from the rotations acting on the angle by and the reflection . An element of this group is of the form

 ψ=Rcφcτmc,m∈{0,1}. (22)

The action of on the space of -periodic functions is best understood by considering separately the translations and the rotations. Since the translations of leave the set of -periodic functions invariant and translations in fix all -periodic functions, the effective action of the group of translations of is as the 2-torus which is compact. For the rotations, recall that the holohedry of the lattice is the largest subgroup of that leaves invariant. In the case of a square lattice is the dihedral group of the symmetries of the square. It follows that the largest subgroup of that leaves invariant is the compact semi-direct sum . Therefore, we are interested in the action on of the compact group . We have the following Proposition. The operator defined in (13) is equivariant w.r.t. the action of the group . The action of the element on , with given by (22), is defined as follows

 g⋅F(V,γ)(r,ϱ,φ)=F(V,γ)(ϖ−1r,ϱ,(−1)m(φ−φc)), m∈{0,1},

and, because of Proposition 4.2, the reader will verify that is equivariant w.r.t. the action of , i.e. that we have

 g⋅F(V,γ)(r,c)=F(g⋅V,γ)(r,c) ∀g∈G. (23)

At this point, branches of planforms are usually obtained by applying the equivariant branching lemma [60, 17, 30, 14] as follows. The kernel of is -invariant. We fix an isotropy subgroup of (i.e. for which there exists such that for all ) and compute the dimension of the fixed point subspace of associated with where

 FixV(Σ)def.={v∈V,σ⋅v=v, ∀σ∈Σ}.

The equivariant branching lemma states that if is an axial subgroup of , i.e. such that

 dim FixV(Σ)=1, (24)

then generically, there is a branch of steady-state solutions to (1) with symmetries . The genericity condition requires that the eigenvalue that goes through 0 does so with nonzero speed and that some components of the Taylor expansion of are non zero.

We characterize the isotropy subgroups of and the corresponding fixed point subspaces in the following Proposition. The isotropy subgroups of fall in two classes: those which do not contain the color reflection are those of , and those that do contain . In the first case the corresponding fixed subspaces are those of as a subgroup of . In the second case the fixed subspaces are those of projected on along , the eigenspaces of corresponding to the eigenvalues 1 and -1, respectively. If we restrict the color group to , i.e. if we only consider the action of the color rotations, our symmetry group is the same as for the equivariant Hopf bifurcation with symmetry [55, 23]. We note that the fixed point subspaces have even dimension since commutes with and all real finite dimensional non trivial representation of are of even dimension. All fixed point subspaces of for the action of are therefore of even dimension.

If we now extend to , i.e. to , given an isotropy subroup of , either it is an isotropy subgroup of and we are done, or it is not and the color reflection operator is not reduced to the identity on . But is a projector. It has therefore two eigenspaces corresponding to its eigenvalues and is the direct sum of the corresponding eigenspaces. Let . We write where . We have

 τc⋅v=τc⋅v1+τc⋅v−1=v1−v−1, (25)

so that the projection of on along is included in .

Conversely, let . The condition and (25) impose , i.e. , i.e. . so that and .

In fact, writing , with the (closed) subgroups of fall in three classes [36]

I

Closed subgroups of .

II

Closed subgroups containing : they are of the form , a subgroup of .

III

Closed subgroups of which are not a subgroup of and do not contain .

In Proposition 5 we considered only the subgroups of of class I and II. As shown in, e.g., [30][Chapter 13, page 131], those subgroups are determined by pairs of subgroups of such that has index 2 in , see also [46]. If , , is the projection, then is isomorphic to and for any . It is not difficult to list all such pairs using the results in [45]. None of them produces a subgroup of with a nonzero fixed subspace, basically because the color rotations act on by multiplications with a magnitude 1 complex exponential.

Since is -invariant and is compact, we can write as a direct sum of -irreducible subspaces (subspaces such that their only -invariant subspaces are 0 or the whole subspace), see e.g. [31][Theorem 1.22]:

 V=V1⊕⋯⊕Vρ.

It follows easily that

 FixV(Σ)=FixV1(Σ)⊕⋯⊕FixVρ(Σ).

Hence if then for some

so that the first step in classifying the planforms associated with a fixed lattice

is to enumerate each -irreducible subspace of . Moreover is of the form where is an eigenspace of irreducible under the action of and is an eigenspace of irreducible under the action of .

This has been worked out in the case of by several authors including [21]. Dropping the upper index for simplicity, the irreducible representations of must be of the form

 Vs=VsK1⊕⋯⊕VsKρ,

where , . are elements of and the set is an orbit in of the holohedry . As proved in [21], there exists only one such representation of dimension 4 and a countable number of them of dimension 8 if we add the further constraint that they have to be translation free. A representation is translation free if there are no (nontrivial) translations in that act trivially on (28). This requirement ensures that we have found the finest lattice, , that supports the neutral modes (28) [21].

The first one corresponds to

 {K1=k1K2=k2, (26)

the second to

 ⎧⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪⎩K1=uk1+vk2K2=−vk1+uk2K3=vk1+uk2K4=−uk1+vk2, (27)

where and are relatively prime strictly positive integers such that

is odd.

With our notations, we have

 V={u(r,ϱ,ϕ)=k∑j=1zje2πi⟨Kj,r⟩e2πinφeh,m(ϱ)+wje−2πi⟨Kj,r⟩e2πinφeh,m(ϱ)+c.c,zj,wj∈C},k=2,4, (28)

which is isomorphic to by .

If we now consider an isotropy subgroup of , such that , Proposition 7.2 in Chapter XVI in [30] asserts that is a twisted group, i.e. there is a pair of subgroups of and a unique homomorphism such that is the kernel of : . In [23][Table 16 on page 157 and Table 21 on page 160] the authors compute all such subgroups and their fixed point subspaces. Using Proposition 5 we can obtain the corresponding subgroups of and their corresponding fixed point subspaces.

In detail is generated by the rotation, noted and the reflection, noted through the -axis. The action of on induces an action on . The reader will verify that this action is given by

 Rsπ/2⋅(z,w)={(z2,w1,w2,z1)k=2(z2,w1,w2,z1,z4,w3