On the properties of some low-parameter models for color reproduction in terms of spectrum transformations and coverage of a color triangle

10/21/2021
by   Alexey Kroshnin, et al.
0

One of the classical approaches to solving color reproduction problems, such as color adaptation or color space transform, is the use of low-parameter spectral models. The strength of this approach is the ability to choose a set of properties that the model should have, be it a large coverage area of a color triangle, an accurate description of the addition or multiplication of spectra, knowing only the tristimulus corresponding to them. The disadvantage is that some of the properties of the mentioned spectral models are confirmed only experimentally. This work is devoted to the theoretical substantiation of various properties of spectral models. In particular, we prove that the banded model is the only model that simultaneously possesses the properties of closure under addition and multiplication. We also show that the Gaussian model is the limiting case of the von Mises model and prove that the set of protomers of the von Mises model unambiguously covers the color triangle in both the case of convex and non-convex spectral locus.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/13/2016

Learning to Generate Compositional Color Descriptions

The production of color language is essential for grounded language gene...
12/17/2014

Color Image Enhancement In the Framework of Logarithmic Models

In this paper, we propose a mathematical model for color image processin...
02/08/2019

Flexibility of triangle-free planar graphs

Let G be a planar graph with a list assignment L. Suppose a preferred co...
07/13/2017

Maximal Area Triangles in a Convex Polygon

The widely known linear time algorithm for computing the maximum area tr...
09/17/2019

Grounding learning of modifier dynamics: An application to color naming

Grounding is crucial for natural language understanding. An important su...
03/02/2017

Enhancement of human color vision by breaking the binocular redundancy

To see color, the human visual system combines the responses of three ty...
02/26/2019

Chromatic Adaptation Transform by Spectral Reconstruction

A color appearance model (CAM) is an advanced colorimetric tool used to ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

When analyzing images, information about the color of a scene element is usually available only from a three-channel sensor point of view. Due to the mismatch between the dimensions of the spectrum space and the sensor space, the measured tristimulus can be generated by different metameric spectra, which complicates the solution of the color reproduction problem [horn1984exact]. As a result, lighting compensation is always possible only with some approximation [brill1978device, west1979, qiu2018image]. The situation is similar with color space transform [Finlayson1994] and spectral reconstruction [otsu2018].

To overcome this difficulty, one can introduce restrictions on the set of possible spectra [logvinenko2009object, logvinenko2013metamer], which allows us to construct correct color transformations. A particular case of this approach relies on the so-called spectral models that define a set of protomeric spectra [Stiles1962, weinberg1976geometry, nikolayev1985model, Griffin2019]. By the term ‘‘spectral model’’ we will mean bijective mapping of the tristimulus space into the space of model parameters. In this case, each spectrum from a certain subspace of all spectra is represented in a unique way using the model parameters [nikolaev2007spectral].

In the paper [nikolaev2008spectral], the authors note that in order to achieve high-quality color reproduction, the spectral model must have certain properties that correspond to the linear color formation model [nikolaev2004linear]. Particularly, it is important that the set of spectral approximations is closed under addition, multiplication by a number and multiplication by itself. Closure under addition allows us to describe the summation effect of several lighting sources correctly, and closure under multiplication allows us to describe multiple reflections of light from surfaces [nikolaev2006efficiency, gusamutdinova2017verification] and to approximate high-saturation spectra higher accuracy [nikolaev2008spectral]. Also, an important property is the color set coverage [nikolaev2008spectral, mizokami2012, mirzaei2014object]. As a rule, it is not always possible to satisfy all the properties simultaneously, so the question arises: ‘‘How many (and which) properties can a single model have?’’.

In the paper we discuss linear models [brill1978device, Stiles1962, yilmaz1962theory, land1971lightness, nyberg1971II, Sallstrom1973, buchsbaum1980spacial, cohen1982r_matrix, Maloney86constancy, maloney1986evaluation, Marimont1992, Lee1995]. They are of interest because, on the one hand, they are computationally efficient, and, on the other hand, they are closed under addition. However, all of them have a significant drawback – they poorly approximate high saturation spectra [maloney1986evaluation, brill1986chromatic] and, as a result, have a low level of coverage of the color triangle (using three primaries, it is impossible to cover strictly convex color triangle) [nikolaev2007spectral].

It is important to note a special case of linear model – the banded spectral model, in which the spectra are assumed piecewise constant [Stiles1962, land1971lightness, nyberg1971II, maximov1984transformation]. Previously, it was hypothesized that this is the only model that simultaneously has the properties of closure under addition and multiplication [nikolaev2007spectral]. In this paper, we provide a proof of this statement in Section 3.1. In this regard, it makes no sense to look for other models closed under addition and multiplication, but there is still the question of covering a set of colors.

Later on, exponent-based spectral models were proposed, in which the parameters are arguments of an exponential function (in the future we will be interested in a narrower class of them, see Section 3). The use of such models makes it possible to approximate high saturation spectra and increase the coverage area. First of them was the Gaussian model [weinberg1976geometry, nikolayev1985model] and its variants [mizokami2012, Brill2002, MacLeod2003, logvinenko2013object]. Like all exponential models, the Gaussian model has the property of closure under multiplication. In addition, its parameters intuitively correspond to the color appearance characteristics [nikolaev2007spectral, nikolaev2008spectral, mizokami2012]

. The peak of the Gaussian roughly corresponds to hue, the standard deviation corresponds to saturation, and the amplitude corresponds to brightness or lightness. The Gaussian model can be used to explain the Abney effect, which might be an indication that the human eye code the colors in a similar way

[Mizokami2006]. Experiments [nikolaev2006efficiency, nikolaev2005comparative] has confirmed that the quality of color constancy problem description is significantly higher for the Gaussian models compared to the linear [Lee1995] ones. The disadvantage of the Gaussian models is that they do not allow achieving uniform coverage of chromaticity values on the color triangle [mizokami2012, mirzaei2014object].

In an attempt to solve this problem, the authors of [nikolaev2007spectral] has introduced the von Mises model

. Authors experimentally proved that this model provides complete coverage for the standard observer color triangle. At the moment, however, it is not theoretically proven whether the von Mises model allows to completely cover the color triangle of an arbitrary sensor (including ones with non-convex spectral locus). In the Section 

3, the von Mises model is formally defined, in the Section 3.2 its connection with the Gaussian model is studied, and the following Section 4 is devoted to the answers to the questions about coverage.

2 Mathematical model and assumptions

By , we denote a wavelength of light. To describe a spectral power distribution (SPD) of incident radiance on a sensor, we use a finite Borel measure on : . Note that we use general measures, not just absolutely continuous (i.e. having density) w.r.t. the Lebesgue measure, because contains discrete measures, which allows us to describe a laser, a gas-discharge lamp, etc. Moreover, it immediately indicates possible operations on SPDs: one can integrate a (response) function w.r.t. an SPD, multiply it by a function (e.g., a reflectance), or sum up SPDs, but cannot multiply them. These properties are sufficient for linear model description [nikolaev2004linear]. This also highlights the different nature of illuminants, represented by SPDs, and spectral reflectances. We discuss the structure and properties of various SPDs families in more detail in Sections 3 and 4.

Now let us describe a model of color perception we use. Following [weinberg1976geometry], we characterize a sensor (of a camera or an observer) by its response function . Then, a color observed by the sensor under an incident light with the SPD is given by

(2.1)

Note that one can consider this as an equivalence class of indistinguishable SPDs for a given sensor. Any SPD corresponding to some color is called a metamer of [weinberg1976geometry]. We assume

, i.e. it is a continuous bounded vector function. Also, suppose

with depending on and on .

Figure 1: Color coordinates, spectral locus, and color triangle for the standard observer CIE 1931 and Nikon D90 (provided by [Jiang2013]). In the case of a camera, a non-convexity of the locus can be observed.

Now we define the normalized response function

(2.2)

where is the vector of ones, is the standard -dimensional simplex, and stands for the dot product. We assume that can be continuously extended to , i.e. there exist

The curve is called the spectral locus, see Fig. 1. Then we define a reweighted SPD as

(2.3)

Thus,

However, we would like to point out that is concentrated on , and not all measures from can be represented in this way (e.g., the ones having an atom at or ).

It is easy to see that the set of all colors for a given sensor is a pointed convex cone called the color cone. Take an affine subspace ; then the color triangle is a base of , i.e.

We will usually consider

as a subset of the hyperplane

endowed with the corresponding topology. We always suppose is non-degenerate, i.e.  (hence ).

Finally, let us briefly discuss a relation between the spectral locus and the color triangle.

Lemma 2.1.

The following relations hold between the color triangle and the spectral locus:

Proof.

Since is continuous, for any compact set it holds that is compact, thus its convex hull is compact as well due to Carathéodory’s theorem. By the definition of the color cone and the color triangle we have

For any with one can find a sequence of discrete measures such that weakly converge to and .

Since

, and is closed, we conclude that

On the other hand, taking again discrete measures one can obtain any color from . Hence

Remark 1.

If or are not physically reachable, then ‘‘purple’’ colors can be outside the color triangle, nevertheless, the color triangle coincides with the spectral locus convex hull up to the boundary.

3 Spectral models: properties and uniqueness

We define a spectral model as a parametric family of illuminants and spectral reflectances that are Borel functions from to [nikolaev2007spectral, nikolaev2008spectral]. Note that from a physical point of view, we should consider only nonnegative measures and reflectances making values only from ; however, sometimes a general setting is considered, e.g. in linear models described below. In the simplest case, a radiance on the sensor is given by multiplication of an illuminant by a reflectance and some positive constant depending on the geometry of the scene and the viewing conditions: . The above model is by definition hybrid in terms of [nikolaev2008spectral], i.e. the set of illuminants differs from the set of reflectances. However, it is possible to consider a non-hybrid model as well: fix a reference SPD and consider illuminants of the form

Probably the most popular type of a (hybrid) spectral model is the linear one, where and are finite-dimensional linear spaces of signed measures and functions, respectively [brill1978device, yilmaz1962theory, maloney1986evaluation]. Another model that we are interested in is the von Mises one (called the Besselian model in [nikolaev2007spectral]) with width : it is a non-hybrid model with spectral densities and reflectances of the form

(3.1)

Obviously, if , these functions are periodic on . We also define the generalized von Mises family generated by a function as follows:

(3.2)

In particular, taking , we obtain the well-known Gaussian model (if we allow negative , then it also includes reciprocal Gaussians) [weinberg1976geometry].

Now, let us list some important properties that a spectral model may satisfy (cf. [nikolaev2008spectral, Section 3]).

  1. is closed under multiplication by a positive constant.

  2. is closed under multiplication by a constant from or .

  3. is closed under pointwise multiplication.

  4. is closed under multiplication by reflectances.

  5. Additivity: or is closed under addition.

  6. Completeness: map from to colors is surjective.

  7. Injectivity: map from to colors is one-to-one.

  8. Periodicity: and are closed under cyclical shifts on some interval .

It is quite natural to always assume properties 1 and 2, since a source SPD and a reflectance define an incident radiance only up to a multiplicative constant depending on the geometry of the scene and the viewing conditions. Property 3 allows incorporating multiple reflections. Property 4

is related to conjugate priors in Bayesian statistics and can be important to non-hybrid models. The role of property 

8 will be explained later in Section 4.

Finally, properties 6 and 7 allow us to parametrize colors via a spectral model. This, in turn, is essential for the solution of a color constancy or an illumination discounting problems in the following way (an inversion model in terms of [brill1986chromatic]): assume we simultaneously obtain colors corresponding to a source and a reflected light

; then we can estimate an illuminant

from the first color and, based on this estimate, a reflectance from the second one. Given the reflectance estimate, one can compute a corresponding color under a reference illumination, e.g. an equi-energy spectrum or daylight (D50, D65) spectra. In particular, for a linear model, it boils down to solving two systems of linear equations: a system for is fixed, and the other one for depends on its solution. Another possible way is to use a group of protomers [weinberg1976geometry]: consider a non-hybrid model where , and is a Lie group w.r.t. pointwise multiplication; we call them protomers if the map is injective, i.e. for any color there exists at most one metamer from this family. As shown in [weinberg1976geometry], has a form

(3.3)

Obviously, one can choose any basis that spans the same linear space. By definition, this model satisfies properties 3, 4, and 7. Moreover, if contains a constant function, it also satisfies properties 1 and 2. An important example of protomers is given by [weinberg1976geometry] is the Gaussian family. Section 4 shows that under additional assumptions on a response function (satisfied by the standard observer’s one), the von Mises model with is protomeric and complete. Note that the Gaussian family is not complete in the case of the standard observer [nikolaev2007spectral, mizokami2012, mirzaei2014object].

Further, in particular, it will be shown that, from the point of view of certain properties, the banded spectral model (see Section 1) and the von Mises model are unique.

3.1 Banded spectral model

The following proposition shows that the banded model is the only model closed under both addition and multiplication.

Proposition 3.1.

Let be a -dimensional convex cone of functions from to , closed under pointwise multiplication, i.e. for any it holds that . Then there exist unique nonempty disjoint sets such that any function can be represented as

Proof.

For the -dimensional convex cone , its linear span is

and . Thus, any function from can be represented as

and all the basis functions belong to the cone .

Step 1.

We show that an arbitrary function from can take no more than non-zero values. Assume the opposite, namely, that there is a function taking a non-zero value. Consider from , such that with different values . Consider the determinant of the matrix of powers:

The column of the matrix of this determinant consists of the values of the function at the points . Since the values are distinct, the determinant turns out to be non zero. However, the dimension of the space is , which means that the rank of the matrix does not exceed . Thus, the determinant must be zero, and we get a contradiction with the fact that functions can take more than values.

Step 2.

Now we show that there is a function from that takes exactly non-zero values. Suppose the opposite, let there be a function that takes non-zero values , where , and there are no function from taking more values. It follows that there exist non-empty disjoint sets , such that , .

Take a function . Obviously, any their linear combination also takes no more than non-zero values, and hence on any set these functions take constant values. This means that any function from can be represented as a linear combination of only basis functions of the form , which contradicts the fact that the dimension of the space is .

Therefore, any function from and, consequently, from the cone is a simple function:

where , and sets are such that .

Step 3.

Now we show uniqueness of the sets . Suppose that there is another similar family of non-empty disjoint sets . Let a function take non-zero different values. If the sets and do not match, there exist a set , , and , such that . Then , which means that on and on the function takes the same values, which contradicts the fact that the function takes values. ∎

According to [Shepelev2020], for any banded model there exists a linear mapping from the model parameters space to the color space, represented as a matrix with columns from the color cone. The converse is also true, i.e. that any such matrix approximately corresponds to some banded model.

Proposition 3.2.

Let be atomless and , and denotes the closure of . Then for any matrix with columns and there exist disjoint closed sets such that

i.e.  is almost a matrix of transition from parameters of a banded model to the color space.

Proof.

For any there is and such that , , and (see Section 4). Take and divide into disjoint segments , , with . Assign disjoint sets of such segments to each , so that the segments numbered , , correspond to . Each such set of segments will correspond to a set : .

For each , select segments , , such that

Note that it is possible once

since is atomless. Define sets . It follows from the construction of that for any .

Let us introduce the following piecewise constant function:

Since the function is uniformly continuous, converges to uniformly as . Using this approximation, we obtain for any that

which completes the proof. ∎

Remark 2.

Note that if some column of does not belong to , then for any set and

i.e.  cannot be approximated by a matrix of a banded model.

3.2 Von Mises model

The next lemma shows that the von Mises model given by (3.1) is in some sense the only family of form (3.3) satisfying properties 2, 8 (thus it necessarily is a generalized von Mises family (3.2)), and able to approximate spectral colors.

Proposition 3.3.

Let be a generalized von Mises family with continuous periodic function having only one maximum point on . Assume is closed under pointwise multiplication. Then it is the von Mises family, i.e. one can take .

Proof.

W.l.o.g. assume . Consider the Fourier series for :

where . Closedness under pointwise multiplication implies that for any , there are , , such that

Thus their Fourier series coincide:

i.e.  and

Note that the equation has at most two solutions , and they are conjugated. Then taking such that is irrational, we conclude that there is at most one such that . Thus

Since has one maximum point on , we get . The claim follows. ∎

The next proposition links the von Mises model to the Gaussian one.

Proposition 3.4.

Fix , , and . Denote by the following subfamily of the von Mises model with the width restricted to

In the limit it converges to (reciprocal) Gaussians in a sense that for any there are such that

Proof.

For simplicity consider , . Fix and denote . Using the Taylor theorem at the point we obtain that

Therefore, on we have

where , , . Finally, note that , and once , thus the claim follows. ∎

4 On parametrization of colors and coverage of a color triangle

In this section we consider only the case of , what corresponds to a standard observer and most cameras. For the sake of simplicity we assume , .

Having found out that only the banded model can simultaneously satisfy the properties of closure under multiplication and addition it makes no sense to look for any model with the same two properties and with a wider coverage of chromaticity values on a color triangle (including strictly convex case). Getting rid of closure under multiplication seems to be not promising — in this way it is impossible to reproduce saturated colors, which are characterized by sharpened spectra. Other way is to get rid of the closure under addition, which is characteristic of linear models, and consider models based on the exponential functions. Since the Gaussian model is the limiting case of the von Mises model, the main purpose of this section will be to show that the von Mises model completely covers the color triangle in both the case of convex and non-convex spectral locus.

Here we study the question of colors parametrization (modulo intensity). Namely, given a response function and a reference SPD we want to describe conditions under which for some (parametric) family of nonnegative spectral densities the map

(4.1)

is onto, one-to-one, or a bijection to the color triangle (thus the same holds for multiples of and the color cone ). In terms of spectral models it means completeness and injectivity (properties 6 and 7) of the set of illuminants . Note that in the general case a linear model with and non-negative basis functions , , is not complete: indeed, it covers only a polygonal subset of the color triangle. In this section we consider some families of densities including generalized von Mises families which cover the color triangle under suitable assumptions on the map .

Remark 3.

If and , then .

Proof.

It immediately follows from the proof of Lemma 2.1. ∎

Let and , i.e.  for any . It will be useful to consider a -dimensional torus as with identified points and . We also define a closed cyclic interval as follows:

Note that it is a subset of , not of . In a similar way we can define an open cyclic interval , and half-open intervals , .

Since is -dimensional, we can define an angle from to fixing an arbitrary direction and orientation of .

Figure 2: Coverage using the von Mises model of the color triangle of a standard observer CIE 1931 and Nikon D90.
Definition 1.

We say a spectral locus is convex if

(4.2)

and for any (a continuous version of) the angle from to is a monotone function on with .

If, in addition, for any , then the spectral locus is strictly convex. Here denotes the segment of the line connecting and .

It is important to note that the spectral locus for the standard observer CIE 1931 (Fig. 2) is considered to be convex [logvinenko2009object]. Now we discuss some important properties of a convex or strictly convex spectral locus.

Lemma 4.1.

If the spectral locus is convex, then for any closed half-plane there is a (closed) cyclic interval such that

Moreover, .

Proof.

Fix . W.l.o.g. assume that , . Since is convex and bounded there exists a continuous bijection (here denotes the interval with identified endpoints) such that for any . Note that the inverse of is also continuous.

It is easy to see from a geometrical consideration that due to convexity of for any closed half-plane the set is a closed cyclic interval . Then intersection is also a closed cyclic interval in , and we get from the monotonicity of that

is a closed cyclic interval in .

Now note that , otherwise , and hence , is contained in a convex circular segment with a center at thus . Suppose and consider the segment . Since we obtain that , and thus

(4.3)

Let be the closed half-plane containing and with passing through , . Assume that . Then there is such that . But then either and

or and

This contradicts to the fact that . Therefore, is a supporting line of , and thus . Finally, (4.3) yields that . The claim follows. ∎

Lemma 4.2.

If the spectral locus is convex, then for any

Proof.

Fix and assume there is

Take a half-plane such that , and a corresponding cyclic interval from Lemma 4.1. Since we have either or , thus

Therefore, , and we got a contradiction. ∎

Let us also mention a simple sufficient condition for convexity of a spectral locus. Let . Denote by the directional angle of . If the function is increasing and

then the spectral locus is convex.

We say that a -measurable function on changes sign twice on , if , , and there exists a cyclic interval such that both and have nonempty interior, on , and outside . The next lemma is useful to show that on some families is one-to-one.

Lemma 4.3.

Let the spectral locus be strictly convex. Take -integrable functions such that and changes sign twice on . Then .

Proof.

Note that

Then is equivalent to

Define . Now take an interval and such that