1 Introduction
1.1 Motivations and context
One of the central problems in statistics and stochastic modelling is to capture and encode the dependence of a random response on predictors in a flexible manner. Estimating some (conditional) response distributions given values of predictors
is sometimes referred to as density regression and has received attention in many scientific application areas. However, this problem becomes particularly challenging when this dependence does not only concern the mean and/or the variance of the distribution, but other features can evolve, including for instance their shape, their unimodal versus multimodal nature, etc. An example of temperature distribution field is represented in Fig.
1.Among the most notable approaches typically used in a frequentist framework to address this challenge, one can cite finite mixture models [rojas2005conditional]
[fan1996estimation, hall1999methods]. Kernel approaches usually require estimating the bandwidth which is done with crossvalidation [fan2004crossvalidation], bootstrap [hall1999methods] or other methods. Generalized lambda distributions have recently been used in [zhu2020emulation]for flexible semiparametric modelling of unimodal distributions depending on covariables.
Within a Bayesian context, it is natural to put a prior on density functions and derive a posterior distribution accounting for observed data. The most common class of models are infinite mixture models. Its popularity is partly due to the wide literature on algorithms for posterior estimation within a Markov Chain Monte Carlo framework
[jain2004split, walker2007sampling, papaspiliopoulos2008retrospective] or fast approximation [minka2001family].Nonparametric approaches includes generalizing stick breaking processes [dunson2008kernel, dunson2007bayesian, chung2009nonparametric, griffin2006order]
, multivariate transformation of a Beta distribution
[trippa2011multivariate] or transforming a Gaussian Process [jara2011class, donner2018efficient, Tokdar2010, gautier2021SLGPoptim]. The latter model is itself a spatial generalization of the Logistic Gaussian Process (LGP) model [Lenk1988, Lenk1991]. It is a basic brick of the class of models that we are considering in this paper.Choosing a particular model or prior is generally motivated by its theoretical properties as well as its tractability. One of the most essential theoretical properties is the posterior consistency, which ensures that it is possible to asymptotically recover the true data generating process. In the recent years, the literature on Bayesian asymptotics has widely flourished with many fundamental breakthroughs. The most notable achievements concern general results for contraction rates [Ghosal2000, vandervaart2008Rates, ghosal2007convergence, rivoirard2012posterior]. We also note that for conditional density estimation using Gaussian Processes, a consistency result was stated by [Pati2013].
It is also important to study the adequacy of the smoothness of densities delivered by an approach with the one expected based on prior knowledge. In geostatistics and spatial statistics [Matheron1963, stein2012interpolation, Cressie1993], quantifying the spatial regularity of a scalar valued process has been well studied and a wide literature is available. These approaches have been extended to the setting of function valued processes [ramsay2004functional, henao2009geostatistical, nerini2010cokriging] but the main contributions in this field are generally limited to stationary functional stochastic processes valued in . Extensions to the distributional setting have also been proposed through embedding into an infinitedimensional Hilbert Space using Aitchison geometry [Aitchison1982, pawlowsky2011compositional] and classical results on stationary functional processes. Here, we will propose an approach adapted to density valued field that does not require Hilbert Space embedding.
Our work is inspired by results from [Tokdar2010] and following up on [gautier2021SLGPoptim]. We focus on spatial logistic Gaussian Process models (SLGPs), that extend indeed LGP models to random fields of probability measures by exponentiating and marginally normalizing a Gaussian Random Field (colloquially referred here to as a Gaussian Process or GP) defined on a product space between responses and spatial coordinates (predictors). One of our contributions is to revisit both LGP and SLGP models in terms of random measures and random measure fields, investigating in turn different notions of equivalence/indistinguishably between random measure fields and related them to classical related notions of coincidence for the inducing scalarvalued random fields.
We furthermore investigate various notions of spatial regularity for random measure fields induced by SLGPs, with a particular focus on random probability density fields in the case of SLGPs induced by continuous GPs. Sufficient conditions of meanpower continuity (with respect to TV, Hellinger and KL) of SLGPs are established in terms of the covariance kernel of a suitable field of logincrement underlying the considered SLGPs. Also, almost sure results are obtained, that build upon general results on Gaussian measures in Banach spaces.
From the computational side, we introduce a Markov Chain Monte Carlo algorithmic approach for conditional density estimation relying on Random Fourier Features approximation [rahimi2008random, rahimi2009weighted], and apply it to a meteorological data set.
1.2 Structure of the paper
This document is structured in the following way: in Section 2, we present the Logistic Gaussian Process (LGP) and explore its links to Random Measures. We present characterisations of this model under different set of assumptions. We then introduce an extension of this model, called Spatial Logistic Gaussian Process and explore its properties. We build upon our study of the LGP to suggest characterisations for the random fields of dependant measures considered. Throughout Section 3, we study the spatial regularity of the SLGP by relying on notions in spatial statistics and basic yet powerful results from Gaussian measure theory. Some results over analytical test functions and over a meteorological data set are presented in Section 4. We also included definitions and properties of the notion of consistency as well as some short results of posterior consistency in Appendix B, some proofs in Appendix C, and details on the implementation of the density field estimation in Appendix D.
1.3 Notations
Throughout the document, we consider an index space and a response space , . We suppose that both and are compact and convex.
We denote by the Lebesgue measure on .
We use the notation to indicate that is a Gaussian Process (GP) with mean function and covariance function . The definition of a GP is available in Appendix A alongside other basic definitions.
For a set (here or ), we denote by , ,
the sets of continuous real functions, Probability Density Functions (PDFs) and almost everywhere positive PDFs on
, respectively. Finally, we denote by the set of fields of PDFs on indexed by , and by its counterpart featuring almost everywhere positive PDFs.2 Logistic Gaussian random measures and measure fields
Generative approaches to samplebased density estimation build upon generative probabilistic models for the unknown densities. A convenient option to devise such probabilistic models over the set consists in renormalizing nonnegative random functions that are almost surely integrable.
When the random density is obtained by exponentiation and normalization of a Gaussian Random Process, the resulting process is called Logistic Gaussian Process (LGP). This provides a flexible prior over positive density functions, and the smoothness of the generated densities is directly inherited from the GP’s smoothness. We first review here basic facts and literature regarding this approach and establish in turn links with the framework of random measures. We then extend the latter to a spatial extension of LGP models, namely the Spatial Logistic Gaussian Process (SLGP) models.
2.1 From the LGP for density estimation to random measures
The LGP for density estimation was established and studied in [Lenk1988, Lenk1991, Leonard1978]. Its definition relies on the socalled logistic density transformation:
Definition 2.1 (Logistic density transformation).
The logistic density transformation is defined over the set of measurable such that , by
(1) 
hence being a mapping between exponentially integrable measurable real functions and .
Let us note that the term transformation is to be understood in a colloquial sense as is by no means bijective: translating any exponentially integrable measurable real function by a constant (confounded here with the constant function ) preserves the image density as .
As we will develop below, can be leveraged to induce a probability distribution over probability measures and fields thereof, and the invariance property above will be of importance when investigating LGPs and SLGPs in the following. Let us first review different theoretical settings in which the LGP has been introduced in the statistical literature.
2.1.1 A brief literature review on the LGP
The LGP is commonly introduced as a random probability density function obtained by applying to a sufficiently wellbehaved GP , resulting in
(2) 
In the literature, various assumptions and theoretical settings have been proposed that (often, implicitly) specify what wellbehaved refers to and in what sense the colloquial definition above is meant. We present a concise review of a few papers among the ones we deem to be most representative on the topic. The list is ordered by publication date, we summarize some measurability and other assumptions underlying the respective LGP constructions, as well as the main contribution of the papers. In particular, we focus on whether the stochastic process the authors define has sample paths that are probability density functions, and if so, whether the papers consider some (response) measurable space suitable to accommodate such objects.

When first introduced in [Leonard1978], the LGP was studied in a unidimensional setting, with being a compact interval. In this seminal paper, the LGP was obtained by transforming a GP possessing an exponential covariance kernel . Assuming that the mean function of is continuous, this particular choice of kernel ensures that admits a version that is continuous almost surely on .

In [Lenk1988], a more general approach to the LGP was introduced. Starting off from an underlying probability space , the LGP was defined as a mapping from to with sample paths integrating to . The space of functions from to was equipped with , coined Borel field of , and with its restriction to functions integrating to . With this construction, the GP was a measurable map from to and therefore, the LGP a random PDF. Also, a generalized logistic Gaussian processes (gLGP) was constructed and elegant formulation of the posterior distribution of the gLGP conditioned on observations were derived. Numerical approaches for calculating the Bayes estimate were proposed, constituting the starting point of the followup paper [Lenk1991]. Although measurability assumptions were tackled and a particular focus was set on ensuring that one transforms only measurable GP, some points remained unclear. Since the author did not specify a topology on , it is unclear how to define a Borel field on this space. However, we note that is possible to work with the product field on , which would guarantee the joint measurability of the LGP.

In [TokdarConsist2007], the LGP was introduced from a hierarchical Bayesian modelling perspective, allowing in turn to handle the estimation of GP hyperparameters. This paper considered a separable GP that is exponentially integrable almost surely, stating that the LGP thus takes values in . The main result in the paper is that the considered hierarchical model achieves weak and strong consistency for density estimation at functions that are piecewise continuous. It is completed by another paper, [TokdarTowards2007], where the authors propose a tractable implementation of the density estimation with such a model. Let us note that the GP’s separability alleviates some technicalities regarding the measurability assumptions to consider, and having a.s. allows us to state that LGP realizations are PDFs almost surely.

In [vandervaart2008Rates], the GP was chosen with bounded sample paths, which allowed it to be viewed as a Borel measurable map in the space of bounded functions of equipped with the supnorm. This paper derived concentration rates for the posterior of the LGP. With these assumptions, the LGP can be considered as a Borel measurable map in the same space as and is guaranteed to have sample paths that are bounded probability density functions.
This short review emphasizes the lack of consensus regarding the LGP’s definition including underlying structures and assumptions. It is interesting to note that in [vandervaart2008Rates], the authors require to be bounded surely, whereas the authors of the three other papers worked with almost sure properties of (mostly, the almost surely continuity of the process).
We will see in the next subsection that working with sure properties rather than almostsure ones allows us to draw links between the LGP and the fertile framework of random measures. We will revisit the definition of LGP in order to build up our subsequent analyses and generalizations on transparent mathematical foundations.
2.1.2 Exploring the links between LGP models and random measures
Throughout the rest of the article, we denote by the ambient probability space, and consider only Gaussian processes that are measurable. We call a random process exponentially integrable when for any , we have .
Since this section aims at exploring the relationships between Random Measures (RM) and LGP, we briefly recall some basic properties and definitions of (locally finite) random measures. We rely on the definitions from [kallenberg2017random]. In the terminology of [kallenberg2017random], our sample space of interest is here the space , equipped with the Euclidean metric (and hence Polish by the compactness assumption) and the corresponding Borel algebra of .
Definition 2.2 (Considered sigmafield on probability measures on ).
We denote the collection of all probability measures on , and take the field on to be the smallest field that makes all maps from to measurable for .
Definition 2.3 (Random Measures).
A random measure is a random element from to such that for any , where is a null set, we have:
(3) 
Note that here the term bounded is between parentheses as is assumed compact and so all elements of are bounded. Among the motivations listed for the choice of this structure, we retain that the field is identical to the Borel field for the weak topology of . This structure ensures that the random elements considered are regular conditional distributions on :

For any , the mapping is a measure on .

For any , is  measurable.
Remark 1 (RMs seen as random fields).
We can see a RM as a particular instance of a random field indexed by , namely . Therefore it is natural to revisit the notions of equality in distribution and of indistinguishability for RMs. In particular, we will call two random measures and indistinguishable from one another if and only if:
(4) 
With this in mind, let us establish a connection between LGPs and RMs.
Proposition 2.4 (RM induced by a GP).
For an exponentially integrable GP,
(5) 
defines a random measure that we call random measure induced by .
Proof of proposition 2.4.
Since is a measurable GP, and its integrals are measurable as well. Therefore, for any . the mapping is measurable from to . Furthermore, for any , is a probability measure on , so a fortiori locally finite. ∎
Remark 2.
We consider the condition of sure exponential integrability made in Definition 2.4 not to be overly restrictive. Indeed, let us consider a GP that is a.s. exponentially integrable (meaning that is integrable for all except some null set noted ). Then, being compact, we can always construct a surely exponentially integrable GP indistinguishable from via
(6) 
Remark 3.
While it is tempting to characterise a LGP by some underlying GP, it is hopeless. In fact, let us consider an exponentially integrable GP
and a random variable
defined on the same probability space. Then, and induce the exact same random measure , since:(7) 
Due to the normalisation constant in Equation 5, there is no onetoone correspondence between GPs and associated random measures. In particular, needs not to be measurable and needs not to be a GP.
The arising questions that we will try to address through the rest of this section is: how to characterise the random measures that can be obtained through Equation 5, and can we give sufficient conditions on measurable and exponentially integrable GPs for them to yield the same random measure?
Proposition 2.5 (Characterisation of a RM induced by a GP).
Let be a random measure such that on , being a null set, and that there exists a (nonnecessarily Gaussian) random field with:

exponentially integrable,

The increment field is Gaussian.

On , almost everywhere.
Then, is indistinguishable from the random measure induced by , an exponentially integrable GP, with its field of increments indistinguishable from .
Conversely, (1)(3) is satisfied for all the RMs induced by exponentially measurable GPs that admit a RadonNikodym with respect to almost surely.
Proof of proposition 2.5.
Let and satisfy the three conditions above. Let us show that there exists an exponentially integrable GP with increments indistinguishable from those of such that is induced by .
Let be an arbitrary point in and set
(for and on ).
Then, is a GP and on we have:
(8) 
and . Moreover, for , we have almost everywhere, which implies that:
(9) 
In particular: .It follows that for any :
(10) 
Therefore, coincides with the random measure induced by on .
Conversely, let us prove that if is the random measure induced by an exponentially integrable GP , there exists satisfying the three conditions above. We set . Then, the increment process is a GP and, as for all , we have , then for all , we have:
(11) 
Finally, , and by definition, for all in . Therefore almost everywhere. ∎
Remark 4.
In practice, GPs are often defined up to a version, by specifying their mean and covariance kernel (and therefore their finitedimensional distributions). However, since the definition of the RM induced by a GP involves the sample path of over all , having two exponentially integrable GPs and with:
(12) 
is not sufficient to ensure that the RM and they respectively induce satisfy:
(13) 
One wellknown exception to this remark arises when is a.s. continuous and is a version of . Then, both GPs are separable and indistinguishable, and so are the RMs they induce. This property is stated in [azais2009] Ch. 1, Sec. 4, Prop. 1.9 in dimension 1, and generalized to higher dimensions in [scheuerer2009] Ch. 5 Sec. 2 Lemma 5.2.8.
There are several families of results yielding sufficient conditions for stochastic process to be continuous almost surely, or to admit a version that is continuous almost surely. Some of these results are presented in Section 3. Almost surely continuity proves to be a reasonable and practical assumption, and motivates us to work with it for the rest of this section.
Proposition 2.6 (LGP induced by an a.s. continuous GP).
Let be an exponentially integrable GP and a null set such that is continuous for any . Then, the LGP associated to , given by:
(14) 
is almost surely the continuous representer of , where is the random measure induced by .
Considering an almost surely continuous GP allows us to work with functions that are almost surely random densities, rather than considering random measures. It allows for a perhaps more instrumental characterisation of the LGPs than the one stated in Proposition 2.5.
Proposition 2.7 (Characterisation of the almost surely continuous LGP).
A stochastic process that is almost surely a continuous element of is indistinguishable from a LGP if and only if its process of log increments is indistinguishable from a continuous GP.
Proof of proposition 2.7.
Let us consider such a process. Then, there exists a nullset and a GP such that for any , is a continuous element of and for all . Let us prove that there exists an almost surely continuous GP such that on . For be an arbitrary anchor point in , set . is a GP and is continuous on . Moreover, for any , is a positive PDF, and we have:
(15)  
(16)  
(17) 
Therefore, is indistinguishable from the LGP associated to .
Conversely, if is the LGP associated to an almost surely continuous GP , then there exist a nullset such that is continuous and is a positive continuous PDF on and:
(18)  
(19) 
Therefore, the process of log increments of is indistinguishable from the process of increments of , which is an almost surely continuous GP on ∎
Here, we are working in the a.s. continuous case. Since is compact, equality up to a version and indistinguishability of processes coincide. Therefore, we can characterise the exponentially integrable, a.s. continuous GPs that yield indistinguishable LGPs through their increment kernels and means. Indeed, it follows from Equation 19 two GPs with respective mean and covariance kernel yielding two LGPs indistinguishable from one another, is the same as having:
(20)  
(21) 
Remark 5.
These results and characterisations provide us with a basis on which we can build up to introduce a spatial extension of the Logistic Gaussian Process.
2.2 On Spatial LGP models and associated random measure fields
In this section, we build upon the work of [Pati2013] and present the considered spatial extension of the Logistic Gaussian Process in a setting similar to that of Subsection 2.1. We point that, rather than focusing on the posterior consistency of the model as the authors of the aforementioned paper did, we will study its spatial regularity.
In this Section, we will call a measurable GP exponentially measurable alongside if and only if for any .
We start by generalizing the notion of logistic density transformation:
Definition 2.8 (Spatial logistic density transformation).
The spatial logistic density transformation is defined over the set of measurable such that , by:
(22) 
hence being a mapping between measurable functions that are exponentially integrable alongside and .
Informally speaking, we introduced the LGP as the random process obtained after exponentiating and rescaling a GP indexed by . For the Spatial Logistic Gaussian Process (SLGP), we will do the same with a wellbehaved GP and study the stochastic process obtained from applying the spatial logistic density transformation to :
(23) 
As for our study of the LGP, we will start working with few assumptions on the measurability of the GP. We generalize Proposition 2.4 and consider fields of dependant random measures (i.e. collections of random measures on the same probability space).
Proposition 2.9 (RMF induced by a GP).
For , an exponentially integrable GP,
(24) 
defines a Random Measure Field (RMF) that we call random measure field induced by .
The similarities between LGP and SLGP immediately induce that Remark 1 admits a straightforward extension in this spatial setting. We briefly introduce the notions of indistinguishability for RMF. Later, in remark 8 we will discuss the benefits of the indistinguishability, compared to other notions of equality for RMF.
Remark 6.
Let be a RMF, can be seen as a collection of random variables on the same probability space, indexed by . Therefore it is a random field, and we will call two RMF and indistinguishable from one another if and only if:
(25) 
As the SLGP can be seen as an indexed version of the LGP, it also presents the same caveats. We can extend remark 3 and see that it is possible for two random fields (not necessarily Gaussian) to yield the same RMF through spatial logistic density transformation.
Remark 7.
Let us consider an exponentially integrable alongside GP and a random field defined on the same probability space. Then, and induce the exact same RMF, as once again:
(26) 
It is possible to easily adapt the characterisations presented in Propositions 2.5 and 2.7 to the spatial context and obtain characterisations of the SLGP whenever it admits RadonNikodym at all almost surely, or whenever it is continuous.
Proposition 2.10 (Characterisation of a RMF induced by a GP).
Let be a RMF, such that on for all , N being a Pnull set independent of , and that there exists a random field with:

is exponentially integrable alongside .

The increment field is Gaussian.

On , almost everywhere for all .
Then, is indistinguishable from the RMF induced by , an exponentially integrable alongside GP, with its field increments alongside : indistinguishable from .
Conversely, (1)(3) is satisfied for all the RMFs induced by exponentially measurable GPs that admit a RadonNikodym with respect to for all almost surely.
Proof of proposition 2.10.
Let and satisfy the three conditions above. Let us show that there exists an exponentially integrable GP with increments alongside indistinguishable from those of such that is induced by .
Let be an arbitrary point in and set .
Then, is a GP and on we have for any :
(27) 
and for all .
Moreover, for any , we have almost everywhere, which implies:
(28) 
In particular: .
It follows that for any :
(29) 
Therefore, coincides with the field of random measures induced by on , meaning that the two random processes are indistinguishable.
Conversely, let us prove that if is the field of random measures induced by a GP exponentially integrable alongside , there exists satisfying the three conditions above.
We set .
Then, is a GP, and as for all , we have , then:
(30) 
Finally, , and by definition, for all in . Therefore almost everywhere for any . ∎
Remark 8 (Indistinguishability compared to others notions of equality between RMF).
In Proposition 2.10, we worked with the indistinguishability of random measure fields, as defined in Equation 25.
Although one could consider other types in equality between RMF, such as the equality in distribution:
(31) 
Or a notion of equality that is between those of Equations 25 and 31:
(32) 
we found out while investigating generalizations of the characterisation 2.5 in characterisation 2.10 that the indistinguishability (Equation 25) seems to be the best fit, as it naturally relates indistinguishability of SLGPs to that of underlying (Gaussian) fields of increment and log increment. The same also holds for the generalization of 2.7 that we will see later in this section.
Similarly to characterisation 2.5, the characterisation 2.10 only applies to random measure fields such that on for any , to ensure that the RadonNikodym exists almost surely. This condition can be satisfied if is induced by a GP that is continuous a.s.. In such setting, another simpler characterisation is possible.
Proposition 2.11 (SLGP induced by an a.s. continuous GP).
Let be an exponentially integrable alongside GP and a null set such that is continuous for any . Then, the SLGP associated to , given by:
(33) 
is almost surely the continuous representer of , where is the RMF induced by .
Considering an almost surely continuous GP allows us to work with fields of functions that are almost surely random densities, rather than considering RMFs. Once again, in this setting we can use the increments of the log of a SLGP to characterise it.
Proposition 2.12 (Characterisation of the a.s. continuous SLGP).
A stochastic process that is almost surely in continuous and in is indistinguishable from a SLGP if and only if its process of log increments alongside : is indistinguishable from a continuous GP on .
Proof of proposition 2.12.
Let us consider such a process.
Then, there exists a nullset and a GP such that for any , is a continuous element of and for all .
Let us prove that there exists an almost surely continuous GP such that on .
We let be an arbitrary anchor point in and set . is a GP and is continuous on .
Moreover, for any , is an element of , and we have for any :
(34)  
(35)  
(36) 
Therefore, is indistinguishable from the LGP associated to .
Conversely, if is the SLGP associated to an almost surely continuous GP
Comments
There are no comments yet.