Adaptive optimal kernel density estimation for directional data

1 Introduction

Directional data arise in many fields such as wind direction for the circular case, astrophysics, paleomagnetism, geology for the spherical case. Many efforts have been put to devise statistical methods to tackle the density estimation problem. We refer to [Mardia and Jupp, 2000] and more recently to [Ley and Verdebout, 2017] for a comprehensive view. Nonparametric procedures have been well developed. In this article we focus on kernel estimation but we may cite a series of works using projection methods on localized bases adapted to the sphere ([Baldi et al., 2009], [Kerkyacharian et al., 2011]). Classical references for kernel estimation with directional data include the seminal papers of [Hall et al., 1987] and [Bai et al., 1988]. It is well-known that the choice of the bandwidth is a key and intricate issue when using kernel methods. Various techniques for selecting the bandwidth have been suggested since the popular cross-validation rule by [Hall et al., 1987]. We shall cite plug-in and refined cross-validatory methods in [Taylor, 2008] and [Oliveira et al., 2012] for the circular case, [Di Marzio et al., 2011] on the torus. More recently, [García-Portugués, 2013] devised an equivalent of the rule-of-thumb of [Silverman, 1986] for directional data, whereas [Amiri et al., 2017] explored computational problems with recursive kernel estimators based on the cross-validation procedure of [Hall et al., 1987]. But to the best of our knowledge, all the rules proposed so far for selecting the bandwidth are empirical. Although they prove efficient in practice, only little attention has been put on their theoretical properties. [Klemelä, 2000] studied convergence rates for $L_{2}$ error over some regularity classes for the kernel estimator for the estimation of the density and its derivatives but the asymptotically optimal bandwidth depends on the density and its smoothness degree which is unfeasible for applications. In the present paper, we try to fill the gap between theory and practice. Our goal is two-fold. We aim at devising an automatic and fully data driven choice of the kernel bandwidth so that the resulted estimator achieves optimal rates in the minimax sense for the $L_{2}$ risk over some regularity classes. We emphasize that the estimator is adaptive to the smoothness degree of the underlying density. It means that the method does not require the specification of the regularity of the density. Our work is inspired by very recent techniques developped for the multivariate case. In the problem of multivariate kernel density estimation, adaptive minimax approaches have been tackled in a remarkable series of papers by [Lepskiĭ, 1990], [Lepskiĭ, 1991], [Goldenshluger and Lepski, 2008] and very recently by [Lacour et al., 2017]. Our methodology is inspired from the PCO (Penalized Comparison of Overfiting) procedure of [Lacour et al., 2017]. It is based on concentration inequalities for $U -$ statistics. Last but not least, our procedure is simple to be implemented and in examples based on simulations, it shows quite good performances in a reasonable computation time.

This paper is organized as follows. In section 2, we present our estimation procedure. In section 3 we provide an oracle inequality and rates of convergences of our estimator for the MISE. Section 4 gives some numerical illustrations. Section 5 gives the proofs of theorems. Finally the Appendix gathers some technical lemmas.

Notations For two integers $a$ , $b$ , we denote $a \land b := min (a, b)$ and $a \lor b := max (a, b)$ . And $⌊ y ⌋$ denotes the largest integer smaller than $y$ such that $⌊ y ⌋ \leq y < ⌊ y ⌋ + 1$ .

Depending on the context, $∥ \cdot ∥$ denotes the classical $L_{2}$ -norm on $R$ or $S^{d - 1}$ , $⟨ \cdot, \cdot ⟩$

the associated scalar product. For a vector

x \in R^{d}

∥ x ∥

stands for the Euclidian norm on

R^{d}

. And

∥ \cdot ∥_{\infty}

is the usual

L_{\infty}

-norm on

S^{d - 1}

The scalar product of two vectors $x$ and $y$ , is denoted by $x^{T} y$ , where $^{T}$ is the transpose operator.

2 Estimation procedure

We observe $n$ i.i.d observations $X_{1}, \dots, X_{n}$ on $S^{d - 1}$ the unit sphere of $R^{d}$ , $d \geq 3$ . The $X_{i}$ ’s are absolutely continuous with respect to the Lebesgue measure $ω_{d}$ on $S^{d - 1}$ with common density $f$ . Therefore, a directional density $f$ satisfies

\int_{S^{d - 1}} f (x) ω_{d} (d x) = 1.

We aim at constructing an adaptive kernel estimator of the density $f$ with a fully data-driven choice of the bandwidth.

2.1 Directional approximation kernel

We present some technical conditions that are required for the kernel.

Assumption 1 The kernel $K : [0, \infty) \to [0, \infty)$ is a bounded and Riemann integrable function such that

0 < \int_{0}^{\infty} x^{(d - 3) / 2} K (x) d x < \infty .

Assumption 1 is classical in kernel density estimation with directional data, see for instance Assumptions D1-D3 in [García-Portugués et al., 2013] and Assumption A1 in [Amiri et al., 2017]. An example of kernel which satisfies Assumption 1 is the popular von-Mises kernel $K (x) = e^{- x}$ .

Now following [Klemelä, 2000] we shall define what is called a kernel of class $s$ . Let

α_{i} (K) = \int_{0}^{\infty} x^{(i + d - 3) / 2} K (x) d x, i \in N .

Assumption 2 Let $s \geq 0$ be even. The kernel $K$ is of class $s$ i.e it is a measurable function $K : [0, + \infty [\to R$ which satisfies :

$α_{i} (K) < + \infty$ for $i = 0, \dots, s$
$α_{0} (K) \neq 0$ ,
$\int_{0}^{h^{- 2}} x^{(2 i + d - 3) / 2} K (x) d x = o (h^{s - 2 i})$ for $i = 1, \dots, s / 2 - 1$ , when $h$ tends to $0$ .

2.2 Family of directional kernel estimators

We consider the following classical directional kernel density estimator $(K_{h} (x, y) := K (\frac{1 - x^{T} y}{h}))$

{^f}_{h} (x) = \frac{c_{0} (h)}{n} n \sum i = 1 K (\frac{1 - x^{T} X_{i}}{h^{2}}) = \frac{c_{0} (h)}{n} n \sum i = 1 K_{h^{2}} (x, X_{i}), x \in S^{d - 1},

where $K$ is a kernel satisfying Assumption 1 and $c_{0} (h)$ a normalizing constant such that ${^f}_{h} (x)$ integrates to unity:

c_{0}^{- 1} (h) := \int_{S^{d - 1}} K_{h^{2}} (x, y) ω_{d} (d y) .

(2.1)

It remains to select a convenient value for $h$ .

2.3 Bandwidth selection

In kernel density estimation, a delicate step consists in selecting the proper bandwith $h$ for ${^f}_{h}$ . We suggest the following data-driven choice of bandwith $^h$ inspired from [Lacour et al., 2017]. We name our procedure SPCO (Spherical Penalized Comparison to Overfitting). We denote $E ({^f}_{h}) := f_{h} .$ Our selection rule is the following:

^h:=argminh∈H{∥^fh−^fhmin∥2+penλ(h)}λ∈R,

(2.2)

where $h_{min} = min H$

{pen}_{λ} (h) := \frac{λ c_{0}^{2} (h) c_{2} (h)}{n} - \frac{1}{n} \int_{S^{d - 1}} {(c_{0} (h_{min}) K_{h_{min}^{2}} (x, y) - c_{0} (h) K_{h^{2}} (x, y))}_{0}^{2} ω_{d} (d y),

(2.3)

c_{2} (h) := \int_{S^{d - 1}} K_{h^{2}}^{2} (x, y) ω_{d} (d y),

and $H$ a set of bandwiths defined by

H = ⎧ ⎪ ⎨ ⎪ ⎩ h : {(\frac{∥ K ∥_{\infty}}{n} \frac{1}{R_{0} (K)})}^{\frac{1}{d - 1}} \leq h \leq 1, and h^{- 1} is an integer ⎫ ⎪ ⎬ ⎪ ⎭,

(2.4)

with $R_{0} (K) := 2^{(d - 3) / 2} σ_{d - 2} α_{0} (K)$ and $σ_{d - 1} = ω_{d} (S^{d - 1})$ denotes the area of $S^{d - 1}$ . We recall that $σ_{d - 1} = \frac{2 π^{d / 2}}{Γ (d / 2)},$ with $Γ$ the Gamma function.

The estimator of $f$ is ${^f}_{^h}$ .

The procedure SPCO involves a real parameter $λ$ . In section 3 we study how to choose the optimal value of $λ$ leading to a fully data driven procedure.

Remark 1.

Note that $c_{0} (h)$ , $c_{2} (h)$ and ${pen}_{λ} (h)$ do not depend on $x$ . Indeed its is known (see [Hall et al., 1987]) that if $y$ is a vector and $x$ a fixed element of $S^{d - 1}$ , then denoting $t = x^{T} y$ their scalar product, we may always write

y = t x + (1 - t^{2})^{1 / 2} ξ,

where $ξ$ is a unit vector orthogonal to $x$ . Further, the area element on $S^{d - 1}$ can be written as

ω_{d} (d x) = (1 - t^{2})^{\frac{d - 3}{2}} d t ω_{d - 1} (d ξ) .

Thus, using these conventions, one obtains

$c_{0}^{- 1} (h)$	$=$	$\int_{S^{d - 1}} K (\frac{1 - x^{T} y}{h^{2}}) ω_{d} (d y)$
	$=$	$\int_{S^{d - 2}} \int_{- 1}^{1} K (\frac{1 - x^{T} (t x + (1 - t^{2})^{1 / 2} ξ)}{h^{2}}) (1 - t^{2})^{\frac{d - 3}{2}} d t ω_{d - 1} (d ξ)$
	$=$	$\int_{S^{d - 2}} ω_{d - 1} (d ξ) \int_{- 1}^{1} K (\frac{1 - t x^{T} x}{h^{2}}) (1 - t^{2})^{\frac{d - 3}{2}} d t$
	$=$	$σ_{d - 2} \int_{- 1}^{1} K (\frac{1 - t}{h^{2}}) (1 - t^{2})^{\frac{d - 3}{2}} d t .$

Using similar computations, one gets that

c_{2} (h) = σ_{d - 2} \int_{- 1}^{1} K^{2} (\frac{1 - t}{h^{2}}) (1 - t^{2})^{(d - 3) / 2} d t,

and

{pen}_{λ} (h) = \frac{λ c_{0}^{2} (h) c_{2} (h)}{n} - \frac{σ_{d - 2}}{n} \int_{- 1}^{1} {(c_{0} (h_{min}) K (\frac{1 - t}{h_{min}^{2}}) - c_{0} (h) K (\frac{1 - t}{h^{2}}))}_{0}^{2} (1 - t^{2})^{(d - 3) / 2} d t .

3 Rates of convergence

3.1 Oracle inequality

First, we state an oracle type inequality which highlights the bias-variance decomposition of the

L_{2}

risk.

| H |

denotes the cardinality of

H

. We recall that we denote

f_{h} := E ({^f}_{h})

Theorem 1.

Assume that kernel $K$ satisfies Assumption 1 and $∥ f ∥_{\infty} < \infty$ . Let $x \geq 1$ and $ε \in (0, 1)$ . Then there exists $n_{0}$ independent of $f$ , such that for $n \geq n_{0}$

with probability larger than

1 - C_{1} | H | e^{- x}

\begin{matrix} ∥ {^f}_{^h} - f ∥^{2} & \leq C_{0} (ε, λ) min h \in H ∥ {^f}_{h} - f ∥^{2} + C_{2} (ε, λ) ∥ f_{h_{min}} - f ∥^{2} + C_{3} (ε, K, λ) (\frac{∥ f ∥_{\infty} x^{2}}{n} + \frac{c_{0} (h_{min}) x^{3}}{n^{2}}), \end{matrix}

(3.1)

where $C_{1}$ is an absolute constant and $C_{0} (ε, λ) = λ + ε$ if $λ \geq 1$ , $C_{0} (ε) = \frac{1}{λ} + ε$ if $0 < λ < 1$ . The constant $C_{2} (ε, λ)$ only depends on $ε$ and $λ$ and $C_{3} (ε, K, λ)$ only depends on $ε$ , $K$ and $λ$ .

This oracle inequality bounds the quadratic risk of SPCO estimator by the infimum over $H$ of the tradeoff between the approximation term $∥ f_{h_{min}} - f ∥^{2}$ and the variance term $∥ {^f}_{h} - f ∥^{2}$ . The terms $C_{3} (ε, K, λ) (\frac{∥ f ∥_{\infty} x^{2}}{n} + \frac{c_{0} (h_{min}) x^{3}}{n^{2}})$ are remaining terms. Hence, this oracle inequality justifies our selection rule. For further details about oracle inequalities and model selection see [Massart, 2007].

The next theorem shows that we cannot choose $λ$ too small ( $λ < 0$ ) at the risk of selecting a bandwidth close to $h_{min}$ with high probability. This would lead to an overfitting estimator. To this purpose, we suppose

∥ f - f_{h_{min}} ∥^{2} \frac{n}{c_{0}^{2} (h_{min}) c_{2} (h_{min})} = o (1) .

(3.2)

This assumption is quite mild. Indeed the variance of ${^f}_{h}$ is of order $\frac{c_{0}^{2} (h) c_{2} (h)}{n}$ , thus this assumption means that the smallest bias is negligible with respect to the corresponding integrated variance. Last but not least, because $c_{0}^{2} (h_{min}) c_{2} (h_{min})$ is of order $h_{min}^{1 - d}$ (see Lemme 3), this assumption amounts to

∥ f - f_{h_{min}} ∥^{2} = o (\frac{h_{min}^{1 - d}}{n}) .

Theorem 2.

Assume that kernel $K$ satisfies Assumption 1 and $∥ f ∥_{\infty} < \infty$ . Assume also (3.2) and

\frac{∥ K ∥_{\infty}}{n} \frac{1}{R_{0} (K)} \leq h_{min}^{d - 1} \leq \frac{(log n)^{β}}{n}, β > 0.

Then if we consider ${pen}_{λ} (h)$ defined in (2.3) with $λ < 0$ , then we have for $n$ large enough, with probability larger than $1 - C_{1} | H | e^{- (n / l o g n)^{1 / 3}}$ :

^h \leq C (λ) h_{min} \leq C (λ) {(\frac{(log n)^{β}}{n})}^{\frac{1}{d - 1}},

where $C_{1}$ is an absolute constant and $C (λ) = {(1.23 (2.1 - \frac{1}{λ}))}^{\frac{1}{d - 1}}$ .

Remark 2.

Theorem 2 invites us to discard $λ < 0$ . Now considering oracle inequality (3.6), $λ = 1$ yields the minimal value of the leading constant $C_{0} (ε, λ) = λ + ε$ . Thus, the theory urges us to take the optimal value $λ = 1$ in the SPCO procedure. Actually, we will see in the numerical section that the choice $λ = 1$ is quite efficient.

3.2 Tuning-free estimator and rates of convergence

Results of Section 3.1 about the optimality of $λ = 1$ enable us to devise our tuning-free estimator ${^f}_{ˇ h}$ with bandwidth $ˇ h$ defined as follows

ˇh:=argminh∈H{∥^fh−^fhmin∥2+pen(h)}

(3.3)

and

pen (h) := \frac{c_{0}^{2} (h) c_{2} (h)}{n} - \frac{1}{n} \int_{S^{d - 1}} {(c_{0} (h_{min}) K_{h_{min}^{2}} (x, y) - c_{0} (h) K_{h^{2}} (x, y))}_{0}^{2} ω_{d} (d y),

(3.4)

and $h_{min} = min H$ where $H$ is a set of bandwiths defined by

H = ⎧ ⎪ ⎨ ⎪ ⎩ h : {(\frac{∥ K ∥_{\infty}}{n} \frac{1}{R_{0} (K)})}^{\frac{1}{d - 1}} \leq h \leq 1, and h^{- 1} is an integer ⎫ ⎪ ⎬ ⎪ ⎭ .

(3.5)

The following corollary of Theorem 1 states an oracle inequality satisfied by ${^f}_{ˇ h}$ . It will be central to compute rates of convergence.

Corollary 1.

Assume that kernel $K$ satisfies Assumption 1 and $∥ f ∥_{\infty} < \infty$ . Let $x \geq 1$ and $ε \in (0, 1)$ . Then there exists $n_{0}$ independent of $f$ , such that for $n \geq n_{0}$ with probability larger than $1 - C_{1} | H | e^{- x}$ ,

\begin{matrix} ∥ {^f}_{ˇ h} - f ∥^{2} & \leq C_{0} (ε) min h \in H ∥ {^f}_{h} - f ∥^{2} + C_{2} (ε) ∥ f_{h_{min}} - f ∥^{2} + C_{3} (ε, K) (\frac{∥ f ∥_{\infty} x^{2}}{n} + \frac{c_{0} (h_{min}) x^{3}}{n^{2}}), \end{matrix}

(3.6)

where $C_{1}$ is an absolute constant and $C_{0} (ε) = 1 + ε$ . The constant $C_{2} (ε)$ only depends on $ε$ and $C_{3} (ε, K)$ only depends on $ε$ and $K$ .

We now compute rates of convergence for the MISE (Mean Integrated Square Error) of our estimator ${^f}_{ˇ h}$ over some smoothness classes. [Klemelä, 2000] defined suitable smoothness classes for the study of the MISE. In particular, theses regularity classes involve a concept of an ”average” of directional derivatives which was first defined in [Hall et al., 1987]. Let us recall the definition of these smoothness classes.

Let $η \in S^{d - 1}$ , $d \geq 3$ and $T_{η} = {ξ \in S^{d - 1} | ξ ⊥ η}$ . Let $ϕ_{η} : S^{d - 1} ∖ {η, - η} \to T_{η} \times] 0, π [$ be a parameterization of $S^{d - 1}$ defined by

ϕ_{η}^{- 1} (ξ, θ) = η cos (θ) + ξ sin (θ) .

When $g : R^{d} \to R$ and $x, ξ \in R^{d}$ , define the derivative of $g$ at $x$ in the direction of $ξ$ to be $D_{ξ} g (x) = {lim}_{h \to 0} h^{- 1} [g (x + h ξ) - g (x)]$ and $D_{ξ}^{s} g = D_{ξ} D_{ξ}^{s - 1} g,$ for $s \geq 2$ an integer.

We now shall define the derivative of order $s$ .

Definition 1.

Let $f : S^{d - 1} \to R$ define $¯ f : R^{d} \to R$ by $¯ f (x) = f (x / ∥ x ∥)$ . The derivative of order $s$ is $D^{s} f : S^{d - 1} \to R$ defined by

D^{s} f (x) = \frac{1}{σ_{d - 1}} \int_{T_{x}} D_{ξ}^{s} ¯ f (x) ω_{d} (d ξ),

where $d \geq 3$ , $T_{x} = {ξ \in S^{d - 1} : ξ ⊥ x}$ .

Definition 2.

When $f : S^{d - 1} \to R$ define ${~ D}^{s} f : S^{d - 1} \times R \to R$ by

{~ D}^{s} f (x, θ) = \frac{1}{σ_{d - 1}} \int_{T_{x}} D_{ϕ_{x}^{- 1} (ξ, θ + \frac{π}{2})}^{s} ¯ f (ϕ_{x}^{- 1} (ξ, θ)) ω_{d} (d ξ) .

We are now able to define the smoothness class $F_{2} (s)$ (see [Klemelä, 2000]).

Definition 3.

Let $s \geq 2$ be even and $1 \leq p \leq \infty$ . Let $F_{2} (s)$ be the set of such functions $f : S^{d - 1} \to R$ that $∥ D^{i} f ∥ < \infty$ for $i = 0, \dots, s$ for all $x \in S^{d - 1}$ and for all $ξ \in T_{x}$ , $\frac{\partial^{s}}{\partial θ^{s}} f (ϕ_{x}^{- 1} (ξ, θ))$ is continuous as a function of $θ \in R$ , $∥ {~ D}^{s} f (\cdot, θ) ∥$ is bounded for $θ \in [0, π]$ and ${lim}_{θ \to 0} ∥ {~ D}^{s} f (\cdot, θ) - D^{s} f ∥ = 0$ .

An application of the oracle inequality in Corollary 1 allows us to derive rates of convergence for the MISE of ${^f}_{ˇ h}$ .

Theorem 3.

Consider a kernel $K$ satisfying Assumption 1 and Assumption 2. For $B > 0$ , let us denote ${~ F}_{2} (s, B)$ the set of densities bounded by $B$ and belonging to $F_{2} (s)$ . Then we have

sup f \in ~ F (s, B) E [∥ {^f}_{ˇ h} - f ∥^{2}] \leq C (s, K, d, B) n^{\frac{- 2 s}{2 s + (d - 1)}},

with $C (s, K, d, B)$ a constant depending on $s$ , $K$ , $d$ and $B$ .

Theorem 3 shows that the estimator ${^f}_{ˇ h}$ achieves the optimal rate of convergence in dimension $d - 1$ (minimax rates for multivariate density estimation are studied in [Ibragimov and Khasminski, 1980], [Ibragimov and Khasminski, 1981], [Hasminskii and Ibragimov, 1990]). Furthermore, our statistical procedure is adaptive to the smoothness $s$ . It means that it does not require the specification of $s$ .

4 Numerical results

We investigate the numerical performances of our fully data-driven estimator ${^f}_{ˇ h}$ defined in section 3.2 with simulations. We compare ${^f}_{ˇ h}$ to the widely used cross-validation estimator and to the ”oracle” (that will be defined later on). We focus on the unit sphere $S^{2}$ (i.e the case $d = 3$ ).

We aim at estimating the von-Mises Fisher density $f_{1, v M}$ (see Figure 1):

f_{1, v M} = \frac{κ}{2 π (e^{κ} - e^{- κ})} e^{κ x^{T} μ},

with $κ = 2$ and $μ = (1, 0, 0)^{T}$ . We recall that $κ$ is the concentration parameter and $μ$ the directional mean. We also estimate the mixture of two von-Mises Fisher densities, $f_{2, v M}$ :

f_{2, v M} = \frac{4}{5} \times \frac{κ}{2 π (e^{κ} - e^{- κ})} e^{κ x^{T} μ} + \frac{1}{5} \times \frac{κ^{'}}{2 π (e^{κ^{'}} - e^{- κ^{'}})} e^{κ^{'} x^{T} μ^{'}},

with $κ^{'} = 0.7$ and $μ^{'} = (- 1, 0, 0)^{T}$ . Note that the smaller the concentration parameter is, the closer to the uniform density the von-Mises Fisher density is.

Figure 1: The density $f_{1, v M}$ in spherical coordinates

Now let us define what the ”oracle” ${^f}_{h_{o r a c l e}}$ is. The bandwidth $h_{o r a c l e}$ is defined as

h_{o r a c l e} = {a r g m i n}_{h \in H} ∥ {^f}_{h} - f ∥^{2} .

$h_{o r a c l e}$ can be viewed as the ”ideal” bandwidth since it uses the specification of the density of interest $f$ which is here either $f_{1, v M}$ or $f_{2, v M}$ . Hence, the performances of ${^f}_{h_{o r a c l e}}$ are used as a benchmark.

In the sequel we present detailed results for $f_{1, v M}$ , namely risk curves and graphic reconstructions and we compute MISE both for $f_{1, v M}$ and $f_{2, v M}$ . We use the von-Mises kernel $K (x) = e^{- x}$ .

Before presenting the performances of the various procedures, we shall remind that theoretical results of section 3.1 have shown that setting $λ = 1$ in the SPCO algorithm was optimal. We would like to show how simulations actually support this conclusion. Indeed, Figure 2 displays the $L_{2}$ -risk of ${^f}_{^h}$ in function of parameter $λ$ . Figure 2 a/ shows a ”dimension jump” and that the minimal risk is reached in a stable zone around $λ = 1$ : negative values of $λ$ lead to an overfitting estimator ( $h_{min}$ is chosen) with an explosion of the risk, whereas large values of $λ$ make the risk increase again (see a zoom on Figure 2 b/). Next, we will realize that $λ = 1$ yields quite good results.

Figure 2: a/ $L_{2}$ risk of ${^f}_{ˇ h}$ to estimate $f_{1, v M}$ in function of parameter $λ$ b/ A zoom

In Lemma 4 of the Appendix, we clarify the expression (3.3) to be minimized to implement our estimator ${^f}_{ˇ h}$ . We now recall the cross-validation criterion of [Hall et al., 1987]. Let

{^f}_{h, i} (x) = \frac{c_{0} (h)}{n - 1} n \sum j \neq i e^{- (1 -^{t} x X_{j}) / h^{2}}

then

C V_{2} (h) = ∥ {^f}_{h} ∥^{2} - \frac{2}{n} n \sum i = 1 {^f}_{h, i} (x) .

$C V_{2}$

is an unbiased estimate of the MISE of

{^f}_{h}

. The cross-validation procedure to select the bandwidth

h

consists in minimizing

C V_{2}

with respect to

h

. We call this selected value

h_{C V_{2}}

Figure 3: Risks curves in function of $h$ for $f_{1, v M}$ , $n = 500$ : a/ $R_{o r a c l e}$ b/ $R_{S P C O}$ c/ $C V_{2}$

Figure 4: Reconstruction of $f_{1, v M}$ , $n = 500$ : a/ ${^f}_{h_{o r a c l e}}$ , $h_{o r a c l e} = 0.33$ b/ SPCO ${^f}_{^h}$ , $^h = 0.25$

In the rest of this section, $S P C O$ will denote the estimation procedure related to ${^f}_{ˇ h}$ . In Figure 3, for $n = 500$ we plot as a function of $h$ : $R_{o r a c l e} := ∥ {^f}_{h} - f_{1, v M} ∥^{2} - ∥ f_{1, v M} ∥^{2}$ for the oracle, $R_{S P C O} := ∥ {^f}_{h} - {^f}_{h_{min}} ∥^{2} + pen (h)$ for SPCO and $C V_{2} (h)$ for cross-validation. We point out on each graphic the value of $h$ that minimizes each quantity. In Figure 4, we plot in spherical coordinates, for $n = 500$ , the density $f_{1, v M}$ and density reconstructions for the oracle, SPCO and cross-validation. Eventually, in Tables 1 and 2, we compute MISE to estimate $f_{1, v M}$ and $f_{2, v M}$ for the oracle, SPCO and cross-validation for $n = 100$ and $n = 500$ , over $100$ Monte-Carlo runs.

	$n = 100$	$n = 500$
Oracle	$0.0064$	$0.0027$
SPCO	0.0091	0.0048
Cross -Validation	0.0099	0.0053

Table 1: MISE over 100 Monte-Carlo repetitions to estimate

f_{1, v M}

	$n = 100$	$n = 500$
Oracle	$0.0051$	$0.0027$
SPCO	0.0083	0.0043
Cross -Validation	0.0096	0.0047

Table 2: MISE over 100 Monte-Carlo repetitions to estimate

f_{2, v M}

When analyzing the results, SPCO shows quite satisfying performances. Indeed, SPCO is close to the oracle and is slightly better than cross-validation when looking at the MISE computations for both densities.

5 Proofs

Before proving Theorem 1, we need several intermediate results. The first one is the following proposition which is the counterpart of Proposition 4.1 of [Lerasle et al., 2016] for $S^{d - 1}$ .

Let

R_{1} (K) := 2^{(d - 3) / 2} σ_{d - 2} \int_{0}^{\infty} x^{(d - 3) / 2} K^{2} (x) d x .

(5.1)

Proposition 4.

Assume that kernel $K$ satisfies Assumption 1. Let $Υ \geq (1 + 2 ∥ f ∥_{\infty}) \lor \frac{8 π ∥ K ∥_{\infty} R_{0} (K)}{R_{1} (K)}$ . There exists $n_{0}$ , such that for $n \geq n_{0}$ ( $n_{0}$ not depending on $f$ ), all $x \geq 1$ and for all $η \in (0, 1)$ with probability larger than $1 - □ | H | e^{- x}$ , for all $h \in H$ each of the following inequalities holds

	$∥ f - {^f}_{h} ∥$	$\leq$	$(1 + η) (∥ f - f_{h} ∥^{2} + \frac{c_{0}^{2} (h) c_{2} (h)}{n}) + □ \frac{Υ x^{2}}{η^{3} n}$		(5.2)
	$∥ f - f_{h} ∥^{2} + \frac{c_{0}^{2} (h) c_{2} (h)}{n}$	$\leq$	$(1 + η) ∥ f - {^f}_{h} ∥^{2} + □ \frac{Υ x^{2}}{η^{3} n} .$		(5.3)

Proof of Proposition 4.

To prove Proposition 4, we need to verify Assumptions (11) - (16) of [Lerasle et al., 2016]. We remind that

f_{h} = E ({^f}_{h}) = c_{0} (h) \int_{y \in S^{d - 1}} f (y) K_{h^{2}} (x, y) ω_{d} (d y) .

Let us check Assumption (11) of [Lerasle et al., 2016]. This one amounts to prove that for some $Γ$ and $Υ$

Γ (1 + ∥ f ∥_{\infty}) \lor sup h \in H ∥ f_{h} ∥^{2} \leq Υ .

We have

$∥ f_{h} ∥^{2}$	$\leq$	$∥ f ∥_{\infty} \int_{x} (\int_{y} c_{0} (h) K_{h^{2}} (x, y) ω_{d} (d y))      = 1 (\int_{y} f (y) c_{0} (h) K_{h^{2}} (x, y) ω_{d} (d y)) ω_{d} (d x)$
	$\leq$	$∥ f ∥_{\infty} \int_{y} f (y) \int_{x} c_{0} (h) K_{h^{2}} (x, y) ω_{d} (d x) ω_{d} (d y)$
	$\leq$	$∥ f ∥_{\infty},$

hence Assumption (11) in [Lerasle et al., 2016] holds with $Γ = 1$ and $Υ \geq 1 + ∥ f ∥_{\infty}$ .

Let us check Assumption (12) of [Lerasle et al., 2016]. We have to prove that

\int c_{0}^{2} (h) K_{h^{2}}^{2} (x, x) ω_{d} (d x) \leq Υ n \int \int c_{0}^{2} (h) K_{h^{2}}^{2} (x, y) ω_{d} (d x) f (y) ω_{d} (d y) .

But since

\int \int c_{0}^{2} (h) K_{h^{2}}^{2} (x, y) ω_{d} (d x) f (y) ω_{d} (d y) = c_{0}^{2} (h) c_{2} (h) \int f (y) ω_{d} (d y) = c_{0}^{2} (h) c_{2} (h),

and

\int c_{0}^{2} (h) K_{h^{2}}^{2} (x, x) ω_{d} (d x) = 4 π c_{0}^{2} (h) K^{2} (0),

Assumption (12) amounts to check that

4 π c_{0}^{2} (h) K^{2} (0) \leq Υ n c_{0}^{2} (h) c_{2} (h) ⟺ Υ \geq \frac{4 π K^{2} (0)}{n c_{2} (h)} .

But using (6.2), we have

c_{2} (h) = R_{1} (K) h^{d - 1} + o (1),

when $h$ tends to $0$ uniformly in $h$ . Thus there exists $n_{1}$ , $n_{1}$ independent of $f$ , such that for $n \geq n_{1}$ , $c_{2} (h) \geq \frac{1}{2} R_{1} (K) h^{d - 1}$ . Now using that $h^{d - 1} \geq \frac{∥ K ∥_{\infty}}{R_{0} (K) n}$ and $K (0) \leq ∥ K ∥_{\infty}$ , it is sufficient to have $Υ \geq \frac{8 π ∥ K ∥_{\infty} R_{0} (K)}{R_{1} (K)}$ to ensure Assumption (12) in [Lerasle et al., 2016].

Assumption (13) in [Lerasle et al., 2016] consists to prove that

∥ f_{h} - f_{h^{'}} ∥_{\infty} \leq Υ \lor \sqrt{Υ n} ∥ f_{h} - f_{h^{'}} ∥_{2} .

For any $h \in H$ and any $x \in S^{d - 1}$ , we have

∥ f_{h} ∥_{\infty} \leq ∥ f ∥_{\infty},

therefore Assumption (13) in [Lerasle et al., 2016] holds for $Υ \geq 2 ∥ f ∥_{\infty}$ .

Assumptions (14) and (15) of [Lerasle et al., 2016] consist in proving respectively that

E {[c_{0}^{2} (h) \int K_{h^{2}} (X, z) K_{h^{2}} (z, Y) ω_{d} (d z)]}_{h^{2}}^{2} \leq Υ c_{0}^{2} (h) c_{2} (h)

and

sup x \in S^{d - 1} E {[c_{0}^{2} (h) \int K_{h^{2}} (X, z) K_{h^{2}} (z, x) ω_{p} (d z)]}_{h^{2}}^{2} \leq Υ n .

We have

c_{0}^{2} (h) \int K_{h^{2}} (x, z) K_{h^{2}} (z, y) ω_{d} (d z) \leq c_{0}^{2} (h) c_{2} (h) \land c_{0} (h) ∥ K ∥_{\infty} .

Indeed if $y = z$ , then $\int K_{h^{2}} (x, z) K_{h^{2}} (z, y) ω_{d} (d z) = \int K_{h^{2}}^{2} (x, z) ω_{d} (d z) = c_{2} (h)$ , otherwise

c_{0}^{2} (h) \int K_{h^{2}} (x, z) K_{h^{2}} (z, y) ω_{d} (d z) \leq c_{0} (h) ∥ K ∥_{\infty} c_{0} (h) \int K_{h^{2}} (x, z) ω_{d} (d z)      = 1 = c_{0} (h) ∥ K ∥_{\infty} .

Furthermore (6.1) entails that there exists $n_{2}$ independent of $f$ , such that for $n \geq n_{2}$ , $c_{0}^{- 1} (h) \geq \frac{1}{2} R_{0} (K) h^{d - 1}$ and consequently $c_{0} (h) \leq \frac{2 n}{∥ K ∥_{\infty}}$ , using (3.5). Thus for $n \geq n_{2}$

c_{0}^{2} (h) \int K_{h^{2}} (x, z) K_{h^{2}} (z, y) ω_{d} (d z) \leq c_{0}^{2} (h) c_{2} (h) \land 2 n .

(5.4)

We have

$E [c_{0}^{2} (h) \int_{z} K_{h^{2}} (X, z) K_{h^{2}} (z, x) ω_{d} (d z)]$	$=$	$c_{0}^{2} (h) \int_{z} (\int_{y} K_{h^{2}} (y, z) f (y) ω_{d} (d y)) K_{h^{2}} (z, x) ω_{d} (d z)$	(5.5)
	$\leq$	$∥ f ∥_{\infty} c_{0} (h) \int_{z} c_{0} (h) \int_{y} K_{h^{2}} (y, z) ω_{d} (d y)      = 1 K_{h^{2}} (z, x) ω_{d} (d z)$
	$\leq$	$∥ f ∥_{\infty} .$

Therefore for $n \geq n_{2},$

sup x \in S^{d - 1} E {[c_{0}^{2} (h)}

Adaptive optimal kernel density estimation for directional data

Adaptive nonparametric estimation of a component density in a two-class mixture model

Adaptive Density Estimation on Bounded Domains

Adaptive greedy algorithm for moderately large dimensions in kernel conditional density estimation

Obfuscation via Information Density Estimation

Robust Comparison of Kernel Densities on Spherical Domains

β-Divergence loss for the kernel density estimation with bias reduced

Fast Nonparametric Conditional Density Estimation

1 Introduction

2 Estimation procedure

2.1 Directional approximation kernel

2.2 Family of directional kernel estimators

2.3 Bandwidth selection

Remark 1.

3 Rates of convergence

3.1 Oracle inequality

Theorem 1.

Theorem 2.

Remark 2.

3.2 Tuning-free estimator and rates of convergence

Corollary 1.

Definition 1.

Definition 2.

Definition 3.

Theorem 3.

4 Numerical results

5 Proofs

Proposition 4.

Adaptive optimal kernel density estimation for directional data

Related Research

Adaptive nonparametric estimation of a component density in a two-class mixture model

Adaptive Density Estimation on Bounded Domains

Adaptive greedy algorithm for moderately large dimensions in kernel conditional density estimation

Obfuscation via Information Density Estimation

Robust Comparison of Kernel Densities on Spherical Domains

β-Divergence loss for the kernel density estimation with bias reduced

Fast Nonparametric Conditional Density Estimation

1 Introduction

2 Estimation procedure

2.1 Directional approximation kernel

2.2 Family of directional kernel estimators

2.3 Bandwidth selection

Remark 1.

3 Rates of convergence

3.1 Oracle inequality

Theorem 1.

Theorem 2.

Remark 2.

3.2 Tuning-free estimator and rates of convergence

Corollary 1.

Definition 1.

Definition 2.

Definition 3.

Theorem 3.

4 Numerical results

5 Proofs

Proposition 4.