A reversal phenomenon in estimation based on multiple samples from the Poisson--Dirichlet distribution

1 Introduction

Let us consider the statistical modeling of a random sampling that produces data containing $n s$ records by drawing $s$ sets of $n$ elements with replacement, where $n$ and $s$ are positive integers. Let $Z_{i j}$

be random variables corresponding

j (= 1, \dots, n)

-th element in

i (= 1, \dots, s)

-th sample. For example, when we are interested in the population mean

μ

and the normal distribution

N (μ, 1)

with unit variance is used as a population model, the sample mean

{¯ Z}_{\cdot \cdot}

n s

records, which is a complete sufficient statistic, coincides with the sample mean of

s

sample means

{¯ Z}_{1 \cdot}, \dots, {¯ Z}_{n \cdot}

, which are calculated from each sample set of size

n

. There exists a relation

n s {¯ Z}_{\cdot \cdot} = n s \sum i = 1 {¯ Z}_{i \cdot},

and both samples have the same information for $μ$ . Under the same setting, if we use the sample median ${~ Z}_{\cdot \cdot}$ of $n s$ records to estimate the population mean $μ$ (which is identical to the population median in the case of $N (μ, 1)$ ), which is not a sufficient statistic, have different information from one of which the sample mean ${¯ ~ Z}_{\cdot \cdot}$ of $s$ sample medians ${~ Z}_{1 \cdot}, \dots, {~ Z}_{n \cdot}$ , which are calculated from each sample set of size $n$ . This implies that when to pool affects the efficiency of inference. However, both of the asymptotic distributions of ${~ Z}_{\cdot \cdot}$ and ${¯ ~ Z}_{\cdot \cdot}$ as $n \to \infty$ are $N (μ, π / (2 n s))$ , which indicates that they asymptotically have the same Fisher information, and so the efficiencies of inference asymptotically coincide.

In this paper, we report a more curious phenomenon that the magnitude relation of the Fisher information reverses depending on the values of parameters. For the purpose of inferring the population diversity, the Poisson–Dirichlet distribution (Kingman, 1975) with parameter $θ$ is assumed as a statistical model of the descending order population frequency. If $θ$ is known, the homozygosity ( $1 / (1 + θ)$

), the probability of discovering a new species (

θ / (n + θ)

where

n

is the sample size), the expected number of singletons (population uniques) in a finite population (

n_{0} θ / (n_{0} + θ - 1)

where

n_{0}

is the population size), and so on can be determined. Moreover, the Poisson–Dirichlet distribution prior (or the Dirichlet process prior) plays an important role in the nonparametric Bayesian estimation when there exist possibly infinite labels. For example, see Crane (2016) and its discussions; Ewens (1972); Hoshino and Takemura (1998); Samuels (1998). Therefore, estimating

θ

efficiently is a problem and we will show that sampling procedure influence the magnitude relation of the Fisher information, so our result can be applied to considering sampling procedures for estimating

θ

efficiently.

Remark 1.

Professor Masaaki Sibuya pointed out that our result represents an aspects of differences between independently and identically distributed (iid) sequences and non-iid sequences (in this paper exchangeable sequences are considered).

2 Sampling from the Poisson–Dirichlet distribution

2.1 Sampling procedures

Assume that the descending order frequency of the infinite population follows the Poisson–Dirichlet distribution with parameter $θ$ . For details about the properties of the Poisson–Dirichlet distribution and related distributions, we refer the reader to Feng (2010). In this paper, the following two types of samples are considered:

Sample (i)
Drawing $s$ samples of $n$ elements with replacement. The numbers of distinct components in the respective sample partitions are denoted by $X_{1}, \dots, X_{s}$ .
Sample (ii)
Drawing a single sample of $n s$ elements. The number of distinct components in the sample partition is denoted by $Y$ .

If a sample of $m$ elements is drawn from a population that follows the Poisson–Dirichlet distribution with parameter $θ$ , then we have a sample frequency. When the population diversity is in interest, we convert the sample frequency to a sample partition, and the sample partition follows the Ewens sampling formula with parameter $(m, θ)$ . Then, the number of distinct components in the sample partition, which is the number of labels in the sample, follows the falling factorial distribution with parameter $(m, θ)$ (this distribution is described in the next subsection). Note that the number of distinct components in the sample is the complete sufficient statistic for $θ$ . Hence, $X_{1}, \dots, X_{s}$ independently and identically follow the falling factorial distribution with parameter $(n, θ)$ , and $Y$ follows the falling factorial distribution with parameter $(n s, θ)$ .

Even though the total numbers of elements are $n s$ in both sampling processes, Samples (i) and (ii) contain different information. In this paper, we calculate the Fisher information acquired from the samples obtained using these two sampling processes and demonstrate that the magnitude relation of the Fisher information can be reversed based on the values of the parameters $n$ , $s$ and $θ$ .

In the literature, there is another model called the multi-observer ESF (Barbour and Tavaré, 2010), where the 2-observer version is considered in Ewens et al. (2007). In the model, a sample of size $n_{0}$ consists of $S$ subsamples whose sizes are $n_{1}, \dots, n_{S}$ where $n_{1} + \dots + n_{S} = n_{0}$ . The motivation of Barbour and Tavaré (2010) is testing of the spatial homogeneity of tumors, which means the parameters $θ$ are common in the left side and right side of a tumor. In the tumor example, when a common underlying evolution process is considered, Sample (i) fits the situation because the population frequencies are considered to be different even in the same side (see the data in Table 2.1 of Barbour and Tavaré (2010), or its original methylation patterns data in Fig 2 of Siegmund et al. (2009)).

Remark 2.

As for the multi-observer ESF, the marginal distributions of subsamples are also the Ewens sampling formula. Even if $s$ subsamples in the whole $S$ subsamples are considered with $n_{1} + \dots + n_{s} << n_{0}$ , $s$ subsamples are not regarded as independent. However, they are exchangeable, and conditionally iid when the population frequency is fixed to one realization. On the other hand, our Sample (i) corresponds to the case where there are different $s$ population frequencies.

Remark 3.

In the context of microdata disclosure risk assessments, the Ewens sampling formula has been used as a model for the frequency of frequencies, which is sometimes called the size index. See, for example, Hoshino and Takemura (1998) and Samuels (1998). The problem considered here fits the situation where an investigation is conducted in $s$ areas and $n$ individuals are surveyed in each area with an assumption on the Poisson–Dirichlet hyperpopulation. Sample (i) corresponds to the case where there are different $s$ population frequencies for each area, and Sample (ii) corresponds to the case where there is only one population frequency. When collecting survey data, it is natural to consider that sample sizes are different area-by-area. However, to clearly observe the reversal phenomenon of the Fisher information, we assume that the sample sizes of all areas are the same.

Remark 4.

Consider there exists possibly infinite number of labels, and we have sample frequency with finite number of labels. In such situation, the Poisson-Dirichlet distribution can be used as a prior distribution, and in order to conduct empirical Bayes estimation its parameter should be estimated. Suppose there are $s$ samples of size $n$ . If the samples are drawn from different populations which independently follow the Poisson–Dirichlet distribution then Sample (i) is suitable; on the other hand if the samples are drawn from one population which follows the Poisson–Dirichlet distribution then Sample (ii) is suitable.

2.2 The falling factorial distribution

If the probability distribution of a positive-integer-valued random variable

X

is given by

p r (X = x) = f (x, θ) = \frac{¯ s (n, x) θ^{x}}{(θ)_{n}}, (x = 1, 2, \dots, n),

then the distribution of $X$ is sometimes called the falling factorial distribution with parameter $(n, θ)$ (Watterson, 1974), where $(θ)_{n} = θ (θ + 1) \dots (θ + n - 1)$ and $¯ s (n, x)$ is the coefficient of $θ^{x}$ in $(θ)_{n}$ . The falling factorial distribution is also called STR1F in Sibuya (1988). In a random partition following the Ewens sampling formula with parameter $(n, θ)$ , the number of distinct components follows the falling factorial distribution with parameter $(n, θ)$ (Ewens, 1972). Since the number of distinct components is the complete sufficient statistic for $θ$ , considering the falling factorial distribution is enough in order to estimate $θ$ . In this subsection, we describe the known properties of a random variable $X$ which follows the falling factorial distribution with parameter $(n, θ)$ .

The moment generating function of

X

is given as

E [e^{X t}] = n \sum i = 1 \frac{¯ s (n, i) (θ e^{t})^{i}}{(θ)_{n}} = \frac{(θ e^{t})_{n}}{(θ)_{n}} = \frac{Γ (θ e^{t} + n) Γ (θ)}{Γ (θ + n) Γ (θ e^{t})},

where $Γ (\cdot)$ is the gamma function. Letting

L_{n} (θ) = n \sum i = 1 \frac{1}{θ + i - 1}, ℓ_{n} (θ) = n \sum i = 1 \frac{i - 1}{(θ + i - 1)^{2}},

the mean and variance of $X$ can be written as

E[X]=θLn(θ),\rm var(X)=θℓn(θ).

Let

l_{θ} (x) = log (f (x, θ)) = log (\frac{¯ s (n, x) θ^{x}}{(θ)_{n}}) .

Then, the log-likelihood is $l_{θ} (X)$ and its derivative ${˙ l}_{θ} (X)$ with respect to $θ$ is

{˙ l}_{θ} (X) = \frac{\partial}{\partial θ} l_{θ} (X) = \frac{X}{θ} - n \sum i = 1 \frac{1}{θ + i - 1} .

Since $E [{˙ l}_{θ} (X)] = 0$ , the Fisher information of $θ$ is

E[(˙lθ(X))2]=1θ2% \rm var(X)=ℓn(θ)θ.

The maximum likelihood estimator $^θ$ of $θ$ is the root of

X - n \sum i = 1 \frac{^θ}{^θ + i - 1} = 0.

It is well known that if $θ$ is fixed then $^θ$ enjoys the asymptotic normality as $n \to \infty$ :

\sqrt{log n} (^θ - θ) \Rightarrow N (0, θ)

as $n \to \infty$ , where $\Rightarrow$ denotes the convergence in distribution. Moreover, it may be useful for some situation to consider the case where $θ$ grows with $n$ (Feng, 2007; Griffiths and Tavaré, 2017; Tsukuda, 2017). In particular, the asymptotic normality of $^θ$ has been extended to the asymptotic case where both $n$ and $θ$ tend to infinity (Tsukuda, 2017):

\sqrt{\frac{ℓ_{n} (θ)}{θ}} (^θ - θ) \Rightarrow N (0, 1)

as $n, θ \to \infty$ with $n^{2} / θ \to \infty$ , or as $n \to \infty$ leaving $θ > 0$ fixed. The asymptotic variance $θ / ℓ_{n} (θ)$ of $^θ$ generally does not become $θ / log n$ . Indeed if $n / θ \to \infty$ then $ℓ_{n} (θ) \sim log (n / θ)$ and if $n / θ \to 0$ then $ℓ_{n} (θ) \sim n^{2} / (2 θ^{2})$ , where $n$ and $θ$ grow simultaneously. Therefore, it holds that if $n / θ \to \infty$ then $θ / ℓ_{n} (θ) \sim θ / log (n / θ)$ ; further, if $n / θ \to 0$ then $θ / ℓ_{n} (θ) \sim 2 θ^{3} / n^{2}$ .

3 The Fisher information of two samples

In this section, the Fisher information of Samples (i) and (ii) is calculated, and the maximum likelihood estimators are presented.

3.1 Sample (i)

Since $X_{1}, \dots, X_{s}$ independently and identically follow the falling factorial distribution with a parameter $(n, θ)$ , the likelihood is given by

s \prod k = 1 f (X_{k}, θ) .

Thus, the log-likelihood $M_{θ}$ is

M_{θ} = s \sum k = 1 l_{θ} (X_{k}),

so its derivative with respect to $θ$ is

{˙ M}_{θ} = s \sum k = 1 {˙ l}_{θ} (X_{k}) = s \sum k = 1 (\frac{X_{k}}{θ} - n \sum i = 1 \frac{1}{θ + i - 1}) = s \sum k = 1 \frac{X_{k}}{θ} - n \sum i = 1 \frac{s}{θ + i - 1} .

It follows from $E [{˙ M}_{θ}] = 0$ and from the independence of ${{˙ l}_{θ} (X_{k})}_{k = 1}^{s}$ that the Fisher information $I_{1} (θ; n, s)$ with respect to $θ$ is

I1(θ;n,s)=E[(˙Mθ)2]=\rm var(˙Mθ)=s∑k=1\rm var(˙lθ(Xk))=sℓn(θ)θ.

The maximum likelihood estimator ${^θ}^{(1)}$ is the root of

¯ X - n \sum i = 1 \frac{{^θ}^{(1)}}{{^θ}^{(1)} + i - 1} = 0,

where $¯ X = s^{- 1} T$ denotes the sample mean of ${X_{k}}_{k = 1}^{s}$ where

T = s \sum i = 1 X_{i} .

(3.1)

Remark 5.

The moment generating function of $T$ is

E [e^{T t}] = {(\frac{(θ e^{t})_{n}}{(θ)_{n}})}^{s} = n \prod j = 1 {(\frac{θ}{θ + j - 1} e^{t} + \frac{j - 1}{θ + j - 1})}^{s} .

(3.2)

Therefore, when $s \neq 1$ , $T$ does not follow the falling factorial distribution. Moreover, when $n \to \infty$ , if and only if $s n^{2} / θ \to \infty$ ,

\frac{T - s θ L_{n} (θ)}{\sqrt{s θ ℓ_{n} (θ)}} \Rightarrow N (0, 1)

(see Proposition 5 in Appendix).

3.2 Sample (ii)

Since $Y$ follows the falling factorial distribution with parameter $(n s, θ)$ , the Fisher information $I_{2} (θ; n, s)$ with respect to $θ$ is given as

I_{2} (θ; n, s) = \frac{1}{θ} n s \sum i = 1 \frac{i - 1}{(θ + i - 1)^{2}} = \frac{ℓ_{n s} (θ)}{θ} .

The maximum likelihood estimator ${^θ}^{(2)}$ is the root of

Y - n s \sum i = 1 \frac{{^θ}^{(2)}}{{^θ}^{(2)} + i - 1} = 0.

4 Comparing the two samples

As is stated before, the Fisher information of Samples (i) and (ii) – $I_{1} (θ; n, s)$ and $I_{2} (θ; n, s)$ , respectively– are different even though the total numbers of elements in the two Samples are the same. Considering simple asymptotic settings, the asymptotic magnitude relation between $I_{1} (θ; n, s)$ and $I_{2} (θ; n, s)$ can be observed as follows: If $n \to \infty$ leaving $s (\geq 2)$ and $θ$ fixed, then

I_{1} (θ; n, s) \sim \frac{s log n}{θ}, I_{2} (θ; n, s) \sim \frac{log n}{θ},

so $I_{1} (θ; n, s) > I_{2} (θ; n, s)$ asymptotically; if $s \to \infty$ leaving $n (\geq 2)$ and $θ$ fixed, then

I_{1} (θ; n, s) = \frac{s ℓ_{n} (θ)}{θ}, I_{2} (θ; n, s) \sim \frac{log s}{θ},

so $I_{1} (θ; n, s) > I_{2} (θ; n, s)$ asymptotically. On the contrary, if $n = 1$ and $s \geq 2$ , $I_{1} (θ; 1, s) = 0 < I_{2} (θ; 1, s)$ , so $I_{1} (θ; n, s) < I_{2} (θ; n, s)$ . These observations indicate that the magnitude relation can be reversed based on the values of the parameters $n, s$ and $θ$ . To demonstrate this phenomenon more precisely, non-asymptotic sufficient conditions that guarantee the magnitude relation between $I_{1} (θ; n, s)$ and $I_{2} (θ; n, s)$ are provided in Subsection 4.1. Moreover, the asymptotic conditions that guarantee the asymptotic magnitude relation where a pair of parameters either $(n, θ)$ , $(s, θ)$ , or $(n, s)$ , grows simultaneously leaving the rest parameter fixed are provided in Subsection 4.2.

4.1 Non-asymptotic results

In this subsection, let us provide sufficient conditions under which $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ and a sufficient condition under which $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ . The results indicate that, roughly speaking, the former inequality holds when $θ$ is relatively small in comparison to $n$ and $s$ , and the latter inequality holds when $θ$ is relatively large in comparison to that relative to $n$ and $s$ .

Firstly, we provide a sufficient condition under which $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ .

Theorem 1.

(1) When $θ \geq 2$ , if integers $s$ and $n$ satisfy $s \geq 2$ and

n > 1 + \frac{⌊ θ ⌋ - 1 + \frac{1}{s}}{ℓ_{⌊ θ ⌋} (θ)},

(4.1)

then $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ .
(2) When $θ < 2$ , if integers $s$ and $n$ satisfy $s \geq 2$ and

n > 1 + (θ + 1)^{2} (\frac{1}{s} + 1),

(4.2)

then $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ .

Proof.

First, note that $x \mapsto x (θ + x)^{- 2}$ is decreasing as $x$ increases in the range $x > θ$ because its derivative is $x \mapsto (θ - x) (θ + x)^{- 3}$ . Moreover, either the hypothesis of (1) or (2) implies $n > θ$ , since

	$1 + \frac{⌊ θ ⌋ - 1 + \frac{1}{s}}{ℓ_{⌊ θ ⌋} (θ)} - θ$	$>$	$1 + \frac{⌊ θ ⌋ - 1 + \frac{1}{s}}{1 / 2} - θ = 2 ⌊ θ ⌋ - 1 + \frac{2}{s} - θ$
		$\geq$	$⌊ θ ⌋ - 2 + \frac{2}{s},$

where

ℓ_{⌊ θ ⌋} (θ) < \frac{⌊ θ ⌋ (⌊ θ ⌋ - 1)}{2 θ^{2}} < \frac{θ (θ - 1)}{2 θ^{2}} < \frac{1}{2}

are used in the first inequality and

1 + (θ + 1)^{2} (\frac{1}{s} + 1) > θ .

Therefore,

	$s ℓ_{n} (θ) - ℓ_{n s} (θ)$
$=$	$(s - 1) n \sum i = 2 \frac{i - 1}{(θ + i - 1)^{2}} - n s - s + 1 \sum i = n + 1 \frac{i - 1}{(θ + i - 1)^{2}} - n s \sum i = n s - s + 2 \frac{i - 1}{(θ + i - 1)^{2}}$
$>$	$(s - 1) n \sum i = 2 \frac{i - 1}{(θ + i - 1)^{2}} - (n s - n - s + 1) \frac{n}{(θ + n)^{2}}$
	$- (s - 1) \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}$
$=$	$(s - 1) [n \sum i = 2 {\frac{i - 1}{(θ + i - 1)^{2}} - \frac{n}{(θ + n)^{2}}} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}] .$	(4.4)

Hereafter, the assertions (1) and (2) are considered separately.

(1) When $θ \geq 2$ , the right-hand side of (4.4) is larger than

(s - 1) ⎡ ⎣ ⌊ θ ⌋ \sum i = 2 {\frac{i - 1}{(θ + i - 1)^{2}} - \frac{n}{(θ + n)^{2}}} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}} ⎤ ⎦ .

Further, if the hypothesis is true then this display is positive, because the terms enclosed in the brackets are

			$⌊ θ ⌋ \sum i = 2 {\frac{i - 1}{(θ + i - 1)^{2}} - \frac{n}{(θ + n)^{2}}} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}$
		$=$	$ℓ_{⌊ θ ⌋} (θ) - (⌊ θ ⌋ - 1) \frac{}{n} (θ + n)^{2} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}$
		$>$	$ℓ_{⌊ θ ⌋} (θ) - (⌊ θ ⌋ - 1) \frac{}{1} n - 1 - \frac{1}{s (n - 1)}$
		$=$	$ℓ_{⌊ θ ⌋} (θ) - (⌊ θ ⌋ - 1 + \frac{1}{s}) \frac{1}{n - 1} .$

(2) When $θ < 2$ , the right-hand side of (4.4) is larger than

(s - 1) {\frac{1}{(θ + 1)^{2}} - \frac{n}{(θ + n)^{2}} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}} .

Further, if the hypothesis is true then this display is positive, because the terms enclosed in the brackets are

			$\frac{1}{(θ + 1)^{2}} - \frac{n}{(θ + n)^{2}} - \frac{n s - s + 1}{(θ + n s - s + 1)^{2}}$
		$>$	$\frac{1}{(θ + 1)^{2}} - \frac{1}{n - 1} - \frac{1}{s (n - 1)}$
		$=$	$\frac{1}{(θ + 1)^{2}} - (1 + \frac{1}{s}) \frac{1}{n - 1} .$

This completes the proof. ∎

Remark 6.

To evaluate (4.1), some values of

1 + \frac{⌊ θ ⌋ - 1 + \frac{1}{s}}{ℓ_{⌊ θ ⌋} (θ)}

(4.5)

are shown in Table 1. The condition given in Theorem 1 is only a sufficient condition; when $(n, s, θ) = (7, 2, 3)$ , $s ℓ_{n} (θ) - ℓ_{n s} (θ) = 0.042 > 0$ .

	$2$	$3$	$4$	$5$	$10$	$100$
$s = 2$	13.50	17.54	22.32	27.30	52.83	518.53
$s = 3$	12.00	16.37	21.26	26.29	51.90	517.66
$s = 10$	9.90	14.74	19.77	24.87	50.61	516.44

Table 1: Values of (4.5)

If $θ \leq 1$ then (4.2) is satisfied for $n \geq 8$ and $s \geq 2$ . Therefore, we have the following corollary.

Corollary 2.

If $θ \leq 1$ , then $I_{1} (θ; n, s) > I_{2} (θ; n, s)$ for any integers $n \geq 8$ and $s \geq 2$ .

Next, we provide another sufficient condition under which $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ .

Theorem 3.

1 \leq θ \leq {(\frac{s}{log (1 + n s)})}^{1 / 2} - 1,

(4.6)

then $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ for any integers $n \geq 2$ and $s \geq 2$ .

Proof.

From

ℓ_{n} (θ) = L_{n} (θ) - θ n \sum i = 1 \frac{1}{(θ + i - 1)^{2}},

n \sum i = 1 \frac{1}{(θ + i - 1)^{2}} \geq \frac{n}{θ (n + θ)},

and

L_{n} (θ) < log (1 + \frac{n}{θ}) + \frac{n}{θ (θ + n)}

(see the proof of Proposition 1 in Tsukuda (2017) for derivations of these inequalities), it follows that

ℓ_{n} (θ) < log (1 + \frac{n}{θ}) + (\frac{1}{θ} - 1) \frac{n}{θ + n} .

When $θ \geq 1$ , this display implies that

	$ℓ_{n s} (θ)$	$<$	$log (1 + \frac{n s}{θ})$		(4.7)
		$\leq$	$log (1 + n s) .$		(4.8)

On the other hand, when $n \geq 2$ , it holds that

s ℓ_{n} (θ) \geq \frac{s}{(θ + 1)^{2}} .

(4.9)

Hence, (4.8) and (4.9) imply that

s ℓ_{n} (θ) - ℓ_{n s} (θ) > \frac{s}{(θ + 1)^{2}} - log (1 + n s) \geq 0,

where the last inequality follows from the assumption (4.6). This completes the proof. ∎

Remark 7.

By using (4.7) instead of (4.8), we can loose (4.6) to

1 \leq θ \leq {(\frac{s}{log (1 + n s / θ)})}^{1 / 2} - 1.

Remark 8.

By assuming more direct condition, we may have sharper sufficient conditions. For example, if

ℓ_{n} (θ) > \frac{log s}{s + 1} - \frac{n θ}{(n + 1 + θ) (n s + 1 + θ)}

(4.10)

then $s ℓ_{n} (θ) - ℓ_{n s} (θ) > 0$ . That is because

	$s ℓ_{n} (θ) - ℓ_{n s} (θ)$	$>$	$(s - 1) ℓ_{n} (θ) + log (\frac{n + θ}{n s + θ}) + \frac{(s - 1) n θ}{(n + 1 + θ) (n s + 1 + θ)}$
		$>$	$(s - 1) {ℓ_{n} (θ) + \frac{n θ}{(n + 1 + θ) (n s + 1 + θ)} - \frac{log s}{s + 1}},$

where we have used

	$b \sum i = a \frac{i}{(i + θ)^{2}}$	$=$	$b \sum i = a {\frac{1}{i + θ} - \frac{θ}{(i + θ)^{2}}}$
		$<$	$log (\frac{b + θ}{a - 1 + θ}) - \frac{θ (b - a + 1)}{} (a + θ) (b + 1 + θ)$

for positive integers $a, b (a \leq b)$ . However, interpreting (4.10) in the current form may be difficult.

Finally, we provide a sufficient condition under which $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ .

Theorem 4.

θ > \sqrt{(n - 1) (n s + n - 1)},

(4.11)

then $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ for any integers $n \geq 1$ and $s \geq 2$ .

Proof.

Define the function $g : s \mapsto g (s)$ as

g (s) = s ℓ_{n} (θ) - ℓ_{n s} (θ) .

First, when $n = 1$ , it holds that $g (s) = - ℓ_{s} (θ) < 0$ for $s = 2, 3, \dots$ . Next, consider $n \geq 2$ . Since $g (1) = 0$ , it is sufficient to show that $g (s + 1) < g (s)$ when (4.11) is satisfied. It holds that

			$g (s + 1) - g (s)$
		$=$	$ℓ_{n} (θ) - ℓ_{n (s + 1)} (θ) + ℓ_{n s} (θ)$
		$=$	$n \sum i = 1 \frac{i - 1}{(θ + i - 1)^{2}} - n s + n \sum i = n s + 1 \frac{i - 1}{(θ + i - 1)^{2}}$
		$=$	$n \sum i = 1 {\frac{i - 1}{(θ + i - 1)^{2}} - \frac{n s + i - 1}{(θ + n s + i - 1)^{2}}}$
		$=$	$n \sum i = 1 {\frac{(i - 1) (θ + n s + i - 1)^{2} - (n s + i - 1) (θ + i - 1)^{2}}{(θ + i - 1)^{2} (θ + n s + i - 1)^{2}}},$

where the numerator of the fraction contained in the brackets on the right-hand side is negative since

n s {- θ^{2} + (i - 1) (n s + i - 1)} \leq n s {- θ^{2} + (n - 1) (n s + n - 1)} < 0.

Therefore it holds that $g (s + 1) < g (s)$ , which implies $g (s) < g (1) = 0$ for $s = 2, 3, \dots$ . This completes the proof. ∎

4.2 Asymptotic results

In this subsection, we consider situations where two of $n, s,$ and $θ$ tend to infinity simultaneously leaving the rest one fixed. The results presented in this subsection are summarized in Table 2, where $I_{1} (θ; n, s)$ and $I_{2} (θ; n, s)$ are denoted by $I_{1}$ and $I_{2}$ , respectively.

asymptotic setting	$s ℓ_{n} (θ)$	$ℓ_{n s} (θ)$	magnitude relation
(I) $n / θ \to \infty$ , $s$ :fixed	$s log (n / θ)$	$log (n / θ)$	$I_{1} > I_{2}$
(II) $n / θ \to 0$ , $s$ :fixed	$n^{2} s / (2 θ^{2})$	$n^{2} s^{2} / (2 θ^{2})$	$I_{1} < I_{2}$
(III) $s / θ \to \infty$ , $n$ :fixed	$n^{2} s / (2 θ^{2})$	$log (s / θ)$	case-by-case
(IV) $s / θ \to c \neq 0$ , $n$ :fixed	$n^{2} s / (2 θ^{2})$	$ℓ_{n s} (θ)$	$I_{1} < I_{2}$
(V) $s / θ \to 0$ , $n$ :fixed	$n^{2} s / (2 θ^{2})$	$n^{2} s^{2} / (2 θ^{2})$	$I_{1} < I_{2}$
(VI) $n, s \to \infty$ , $θ$ :fixed	$s log n$	$log (n s)$	$I_{1} > I_{2}$

Table 2: Asymptotic values of

s ℓ_{n} (θ)

and

ℓ_{n s} (θ)

and the asymptotic magnitude relations of the corresponding Fisher information

Six different cases with various asymptotic settings are considered as follows:

(I) $n / θ \to \infty$ , leaving $s \geq 2$ fixed
Since

s ℓ_{n} (θ) \sim s log (\frac{n}{θ}), ℓ_{n s} (θ) \sim log (\frac{n}{θ}),

it follows that $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ asymptotically. This corresponds to Theorem 1 since $ℓ_{⌊ θ ⌋} (θ) ≍ 1$ as $θ \to \infty$ .

(II) $n / θ \to 0$ , leaving $s \geq 2$ fixed
Since

s ℓ_{n} (θ) \sim s \frac{n^{2}}{2 θ^{2}}, ℓ_{n s} (θ) \sim \frac{n^{2} s^{2}}{2 θ^{2}},

it follows that $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ asymptotically. This corresponds to Theorem 4.

(III) $s / θ \to \infty$ , leaving $n \geq 2$ fixed
Since

s ℓ_{n} (θ) \sim s \frac{n^{2}}{2 θ^{2}}, ℓ_{n s} (θ) \sim log (\frac{s}{θ}),

there are three possible cases that differ in terms of the asymptotic magnitude relation between $s$ and $θ^{2} log (s / θ)$ : (a) If $s / (θ^{2} log (s / θ)) \to \infty$ then $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ asymptotically, which corresponds to Theorem 3 (see Remark 7). (b) If $s / (θ^{2} log (s / θ)) \to 0$ then $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ asymptotically, which partly corresponds to Theorem 4. (c) If $s ≍ θ^{2} log (s / θ)$ then

s ℓ_{n} (θ) \sim \frac{n^{2} K}{2} log θ, ℓ_{n s} (θ) \sim log θ,

where $K = lim s / (θ^{2} log (s / θ))$ ; therefore, in this case, the asymptotic magnitude relation between $s ℓ_{n} (θ)$ and $ℓ_{n s} (θ)$ is determined by the magnitude relation between $K$ and $2 / n^{2}$ .

(IV) $s ≍ θ$ , leaving $n \geq 2$ fixed
Since

s ℓ_{n} (θ) \sim s \frac{n^{2}}{2 θ^{2}} \to 0

and

liminf s, θ ℓ_{n s} (θ) > liminf s, θ {\frac{n s (n s - 1)}{2 θ^{2}} \frac{1}{(1 + n s / θ)^{2}}} > 0,

which follows from

ℓ_{n s} (θ) > \frac{1}{(θ + n s)^{2}} n s \sum i = 1 (j - 1) = \frac{n s (n s - 1)}{2 θ^{2}} \frac{1}{(1 + n s / θ)^{2}},

it holds that $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ asymptotically. This corresponds to Theorem 4.

(V) $s / θ \to 0$ , leaving $n \geq 2$ fixed
Since

s ℓ_{n} (θ) \sim s \frac{n^{2}}{2 θ^{2}}, ℓ_{n s} (θ) \sim \frac{n^{2} s^{2}}{2 θ^{2}},

it follows that $s ℓ_{n} (θ) < ℓ_{n s} (θ)$ asymptotically. This corresponds to Theorem 4.

(VI) $n, s \to \infty$ , leaving $θ$ fixed
Since

s ℓ_{n} (θ) \sim s log n, ℓ_{n s} (θ) \sim log (n s),

it follows that $s ℓ_{n} (θ) > ℓ_{n s} (θ)$ asymptotically. This corresponds to Theorem 1.

5 Concluding remarks

Under the assumption of the Poisson–Dirichlet population, we have presented that two ways (i) and (ii) of sampling procedures have different information, that is to say, they lead totally different results. The reason of this phenomenon is that the Ewens sampling formula represents the law of samples from the Poisson–Dirichlet population, a typical random discrete distribution. By virtue of the de Finetti theorem, for an exchangeable sequence there exists a directing measure such that the sequence is conditionally iid. Our result indicates that when data analyses using random distributions are conducted, it is crucial to decide whether the data in interest is sample from an unobservable identical directing measure or not.

Appendix A Appendix: The asymptotic normality of $T$

In this Appendix, the following proposition, which was mentioned in Remark 5, is proven.

Proposition 5.

\frac{s n^{2}}{θ} \to \infty,

(A.1)

then

\frac{T - s θ L_{n} (θ)}{σ} \Rightarrow N (0, 1)

(A.2)

where $T$ is as defined in (3.1) and $σ = σ (n, s, θ) = (s θ ℓ_{n} (θ))^{1 / 2}$ . Moreover, when $n \to \infty$ , (A.1) is also necessary for (A.2).

Proof.

For $i = 1, 2, \dots, n s$ , let

p_{i} = \frac{θ}{θ + j}, (i - 1 \equiv j (mod n)) .

Consider a triangular sequence ${ξ_{i}}_{i = 1}^{n s}$ of independent Bernoulli variables, where $E [ξ_{i}] = p_{i}$ for all $i = 1, \dots, n s$ . Then, from (3.2), the distribution of $T$ is the same as the distribution of

n s \sum i = 1 ξ_{i} .

First, we prove that (A.1) implies

\frac{\sum_{i = 1}^{n s} ξ_{i} - s θ L_{n} (θ)}{σ} \Rightarrow N (0, 1) .

(A.3)

From the central limit theorem for bounded random variables, it is sufficient to show that

σ \to \infty

. When

n / θ \to \infty

n / θ \to c > 0

, it follows that

σ^{2} = s θ ℓ_{n} (θ) \to \infty

. When

n / θ \to 0

, it follows from (A.1) that

σ^{2} = s θ ℓ_{n} (θ) \sim \frac{s n^{2}}{2 θ} \to \infty .

Next, we prove that when $n \to \infty$ (A.3) implies (A.1). Assume that (A.1) does not hold. Consider $n, s, θ$ such that $n s < θ$ . Since $σ^{2} \sim s n^{2} / (2 θ)$ and

p_{i} (1 - p_{i}) \leq θ \frac{n s - 1}{(θ + n s - 1)^{2}} \sim \frac{n s}{θ}

for all $i$ which follows from the fact that $x (θ + x)^{- 2}$ is increasing for $x < θ$ , it holds that

\frac{1}{σ^{2}} max 1 \leq i \leq n s {p_{i} (1 - p_{i})} \to 0.

Hence, in order for (A.3) to hold, the Lindeberg condition

lim n, s, θ \frac{1}{σ^{2}} n s \sum i = 1 E [| ξ_{i} - p_{i} |^{2} 1 {| ξ_{i} - p_{i} | > ε σ}] = 0

(A.4)

for any $ε > 0$ is necessary where $1 {\cdot}$ is the indicator function, so we see that (A.4) does not hold. Since

sup n, s, θ (s ℓ_{n} (θ)) < \infty,

we can take

ε = inf n, s, θ (\frac{θ}{σ (θ + n)}) > 0.

Then, we have

\frac{p_{i}}{σ} > \frac{θ}{σ (θ + n)} > ε

for all $i = 1, \dots, n s$ , which yields that

E [| ξ_{i} - p_{i} |^{2} 1

A reversal phenomenon in estimation based on multiple samples from the Poisson--Dirichlet distribution

Authors

Fisher-Rao geometry of Dirichlet distributions

Active information requirements for fixation on the Wright-Fisher model of population genetics

Exact Good-Turing characterization of the two-parameter Poisson-Dirichlet superpopulation model

Fisher Information of a Family of Generalized Normal Distributions

The entropy based goodness of fit tests for generalized von Mises-Fisher distributions and beyond

Blackwell dominance in large samples

VC-Dimension Based Generalization Bounds for Relational Learning

1 Introduction

Remark 1.

2 Sampling from the Poisson–Dirichlet distribution

2.1 Sampling procedures

Remark 2.

Remark 3.

Remark 4.

2.2 The falling factorial distribution

3 The Fisher information of two samples

3.1 Sample (i)

Remark 5.

3.2 Sample (ii)

4 Comparing the two samples

4.1 Non-asymptotic results

Theorem 1.

Proof.

Remark 6.

Corollary 2.

Theorem 3.

Proof.

Remark 7.

Remark 8.

Theorem 4.

Proof.

4.2 Asymptotic results

5 Concluding remarks

Appendix A Appendix: The asymptotic normality of $T$

Proposition 5.

Proof.

A reversal phenomenon in estimation based on multiple samples from the Poisson--Dirichlet distribution

Authors

Related Research

Fisher-Rao geometry of Dirichlet distributions

Active information requirements for fixation on the Wright-Fisher model of population genetics

Exact Good-Turing characterization of the two-parameter Poisson-Dirichlet superpopulation model

Fisher Information of a Family of Generalized Normal Distributions

The entropy based goodness of fit tests for generalized von Mises-Fisher distributions and beyond

Blackwell dominance in large samples

VC-Dimension Based Generalization Bounds for Relational Learning

This week in AI

1 Introduction

Remark 1.

2 Sampling from the Poisson–Dirichlet distribution

2.1 Sampling procedures

Remark 2.

Remark 3.

Remark 4.

2.2 The falling factorial distribution

3 The Fisher information of two samples

3.1 Sample (i)

Remark 5.

3.2 Sample (ii)

4 Comparing the two samples

4.1 Non-asymptotic results

Theorem 1.

Proof.

Remark 6.

Corollary 2.

Theorem 3.

Proof.

Remark 7.

Remark 8.

Theorem 4.

Proof.

4.2 Asymptotic results

5 Concluding remarks

Appendix A Appendix: The asymptotic normality of T

Proposition 5.

Proof.

Appendix A Appendix: The asymptotic normality of $T$