Various Optimality Criteria for the Prediction of Individual Response Curves

We consider optimal designs for the Kiefer cirteria, which include the E-criterion as a particular case, and the G-criterion in random coefficients regression (RCR) models. We obtain general the Kiefer criteria for approximate designs and prove the equivalence of the E-criteria in the fixed effects and RCR models. We discuss in detail the G-criterion for ordinary linear regression on specific design regions.

Authors

• 8 publications
11/08/2018

Optimal Designs for Minimax-Criteria in Random Coefficient Regression Models

We consider minimax-optimal designs for the prediction of individual par...
11/13/2019

Optimality regions for designs in multiple linear regression models with correlated random coefficients

This paper studies optimal designs for linear regression models with cor...
07/29/2020

Equivalence theorems for compound design problems with application in mixed models

In the present paper we consider design criteria which depend on several...
09/21/2020

Selection of Regression Models under Linear Restrictions for Fixed and Random Designs

Many important modeling tasks in linear regression, including variable s...
03/29/2018

Generalized Bayesian D criterion for single-stratum and multistratum designs

DuMouchel and Jones (1994) proposed the Bayesian D criterion by modifyin...
03/27/2022

Optimal Design for Estimating the Mean Ability over Time in Repeated Item Response Testing

We present general results on D-optimal designs for estimating the mean ...
06/18/2019

Prediction properties of optimum response surface designs

Prediction capability is considered an important issue in response surfa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The subject of this work is random coefficients regression (RCR) models, which were initially introduced in biosciences and are nowdays popular in many fields of statistical application, for example in agricultural studies or medical research.

Linear criteria as well as a generalized version of the D-criterion in RCR have been discussed in Prus and Schwabe (2016b)

. Some results for the particular case of the c-criterion, interpolation and extrapolation, are presented in

Prus and Schwabe (2016a).

In this work we concentrate on the Kiefer’s -criteria (see e. g. Fedorov and Leonov (2013)

, p. 54, for the fixed effects case), which are based on the eigenvalues of the information matrix in the random coefficient regression models, and the

G- (global) criterion, which aims to minimize the maximal prediction mean squared error over the experimental region.

For fixed effects models, the E-criterion (see e. g Atkinson et al. (2007), ch. 10), which can be recognized as the particular Kiefer’s -criterion for , has been considered in detail in Kiefer (1974). Kiefer and Wolfowitz (1960) have proved the equivalence of D- and G-criteria in models with homoscedastic errors. More recently, Wong (1995) has discussed the G

-optimality in heteroscedastic models. For that models, it has been established that the

D- and G-criteria are in general not equivalent.

Here, we present some results for the general form of Kiefer’s -criteria in RCR models and prove for the E-criterion for largest eigenvalue that the optimal designs for fixed effects models retain their optimality for the prediction. We consider the G-criterion for ordinary linear regression with a diagonal covariance structure on specific experimental regions.

The paper has the following structure: In the second part the random coefficients regression models will be specified and the best linear unbiased prediction of individual random parameters will be presented. The third part provides analytical results for the designs, which are optimal for the prediction. The paper will be concluded by a short discussion in the forth part.

2 Model Specification and Prediction

In this paper we consider the random coefficients regression models, in which the -th observation of individual is given by

 Yij=f(xj)⊤\boldmath{β}i+εij,j=1,…,m,i=1,…,n,x∈X (1)

where is the number of observations per individual, is the number of individuals,

is a vector of known regression functions and

is an experimental region. The observational errors

are assumed to have zero mean and common variance

. The individual parameters have unknown population mean and known positive definite covariance matrix . All individual parameters and all observational errors are assumed to be uncorrelated.

The best linear unbiased predictor of individual parameter is given by

 ^\boldmath{β}i=(F⊤F+D−1)−1(F⊤F^\boldmath{β}i;ind+D−1^\boldmath{β}), (2)

where and for the individual vector of observations , the mean observational vector and the design matrix , which is assumed to be of full column rank.

The mean squared error matrix of the of the vector of all predictors of all individual parameters is given by

 (3)

where

is the vector of length with all elements equal to and denotes the Kronecker product.

3 Optimal Designs

We define here exact designs in the following way:

 ξ=(x1,…,xkm1,…,mk), (4)

where are the distinct experimental settings with the numbers of replications . Approximate designs are defined as

 ξ=(x1,…,xkw1,…,wk), (5)

where and only the conditions and have to be satisfied (integer numbers of replications are not required). Further we will use the notation

 M(ξ)=1mk∑j=1mjf(xj)f(xj)⊤ (6)

for the standardized information matrix from the fixed effects model and for the adjusted dispersion matrix for the random effects. We assume the matrix to be non-singular. With this notation the definition of the mean squared error (3) can be extended for approximate designs to

 (7)

when we neglect the constant term .

3.1 Kiefer’s criterion

We define the Kiefer’s -criterion for the prediction in RCR models for all values of as the following function of the trace of the MSE matrix (3) of the prediction:

 Φq=(1nptr(MSEq))1q,q∈(0,∞). (8)

Then we extend the definition of the criterion for approximate designs and obtain the following result.

Theorem 1.

The Kiefer’s -criterion for the prediction of individual parameters is for approximate designs given by

 Φq(ξ)=(1np(tr(M(ξ)−q)+(n−1)tr((M(ξ)+Δ−1)−q)))1q,q∈(0,∞). (9)
Proof.

As proved in Prus (2015), ch. 5, the eigenvalues of are the eigenvalues of with multiplicity and the eigenvalues of with multiplicity . Then we obtain

 Φq(ξ) = (1nptr(MSE(ξ)q))1q = (1np(p∑s=1ηqs+(n−1)p∑s=1μqs))1q = (1np(tr(M(ξ)−q)+(n−1)tr((M(ξ)+Δ−1)−q)))1q.

Note that the Kiefer’s criterion (9) can be recognized as a weighted sum of the Kiefer’s -criteria in fixed effects and Bayesian models (if we neglect the power ) . The weight of the Bayesian part increases with increasing number of individuals . For models with only one individual optimal designs in fixed effects models are optimal for the prediction.

Particular cases of the Kiefer’s -criterion (for and ), D- and A-criteria, have been considered in detail by Prus and Schwabe (2016b). The E-criterion, which can also be recognized as the limiting Kiefer’s criterion for , will be discussed in the next section.

3.2 E-criterion

We define the E-criterion (eigenvalue criterion) for the prediction as the largest eigenvalue of the MSE matrix (3):

 ΦE=λmax. (10)

For approximate designs we obtain the following result.

Theorem 2.

The E-criterion for the prediction of individual parameters is for approximate designs given by

 ΦE(ξ)=ηmax(ξ), (11)

where denotes the largest eigenvalue of .

Proof.

It is easy to see that in Loewner ordering. Therefore, the smallest eigenvalue of is smaller than the smallest eigenvalue of (see e. g. Fedorov and Leonov (2013), p. 11) and, consequently, the largest eigenvalue of is larger than the largest eigenvalue of . Then making use of the proof of Theorem 1 we obtain the result (11). ∎

Corollary 1.

E-optimal designs in the fixed effects model are E-optimal for the prediction of individual parameters in the random coefficient regression model.

3.3 G-criterion

For the prediction in RCR models we define the G-criterion (global criterion) as the maximal sum of individual expected squared differences of the predicted and real response across all individuals with respect to all possible observational settings:

 ΦG=maxx∈Xn∑i=1E((f(x)⊤(^\boldmath{β}i−\boldmath{β}i))2). (12)

We receive the following criterion for the approximate designs.

Theorem 3.

The G-criterion for the prediction of individual parameters is for approximate designs given by

 ΦG(ξ)=maxx∈X(f(x)⊤(M(ξ)−1+(n−1)(M(ξ)+Δ−1)−1)f(x)). (13)
Proof.
 ΦG = maxx∈Xn∑i=1var(f(x)⊤(^\boldmath{β}i−\boldmath{β}i)) = maxx∈Xn∑i=1tr(Cov(^\boldmath{β}i−\boldmath{β}i)f(x)f(x)⊤) = maxx∈Xtr(MSE(In⊗(f(x)f(x)⊤)))
 ΦG(ξ) = maxx∈Xtr(MSE(ξ)(In⊗(f(x)f(x)⊤))) = maxx∈X(tr(M(ξ)−1f(x)f(x)⊤)+(n−1)tr((M(ξ)+Δ−1)−1f(x)f(x)⊤)) = maxx∈X(f(x)⊤(M(ξ)−1+(n−1)(M(ξ)+Δ−1)−1)f(x))

The following property can be easily verified for the -criterion (13) .

Lemma 1.

Let and be approximate designs of form (5) so that in Loewner ordering. Then it holds for the G-criterion (13) that .

Note that the G-criterion (13) is not differentiable and, therefore, no optimality condition in the sense of an equivalence theorem (see Kiefer and Wolfowitz (1960)) is straightforward to formulate. Therefore, we will consider in detail the following particular model .

Particular model: straight line regression

We consider the linear regression model

 Yij=βi1+βi2xj+εij (14)

for two different experimental regions , , and , .

For this model the function

 Φ(x,ξ)=f(x)⊤(M(ξ)−1+(n−1)(M(ξ)+Δ−1)−1)f(x), (15)

which can be also recognized as the sensitivity function of the D-criterion in RCR models (see Prus and Schwabe (2016b)), is a parabola with a positive leading term (with respect to ). Therefore, achieves its maxima at the ends of the intervals. Then the G-criterion (13) simplifies to

 ΦG(ξ1)=max{Φ(0,ξ1),Φ(a,ξ1)}

or

 ΦG(ξ2)=max{Φ(−b,ξ2),Φ(b,ξ2)}

for all non-singular designs on or on , respectively.

Further we will make use of the following simple lemmas.

Lemma 2.

Let

 ξ=(x1,…,xkw1,…,wk)

be an approximate design in model (14) on . Then it holds for the approximate design

 ξ′=(0a1−w′w′),

where , that in Loewner ordering.

Lemma 3.

Let

 ξ=(x1,…,xkw1,…,wk)

be an approximate design in model (14) on . Then it holds for the approximate design

 ξ′=(−bb1−w′w′),

where , that in Loewner ordering.

Then it follows directly from Lemmas 1, 2 and 3 that at least one of -optimal designs in model (14) on or is of the form

 ξ1=(0a1−w1w1)

or

 ξ2=(−bb1−w2w2),

respectively. Then only the optimal weights and have to be determined.

Further we will additionally assume the diagonal structure of the covariance matrix of the random effects: . Then it is easy to verify that and increase and and decrease with increasing values of and , respectively. Consequently, the -optimal designs are solutions of the equations

 Φ(0,ξ1)=Φ(a,ξ1). (16)

and

 Φ(−b,ξ2)=Φ(b,ξ2). (17)

Note that if equation (16) or (17) has no solutions, the resulting optimal designs on or , respectively, will lead to a singular information matrix.

Then equation (16) may be represented for , , in form

 2w1−1w1(1−w1)=(n−1)a2(1+c1−2w1)(1+c1)(a2w1+c2)−a2w21. (18)

For condition (17) we obtain

 2w2−1w2(1−w2)=(n−1)4b2(1−2w2)(1+c1)(b2+c2)−b2(2w2−1)2. (19)

It is easy to see that the only solution of (19) is given by the optimal weight .

According to Prus and Schwabe (2016b), the optimality condition for the -criterion is given by

 Φ(x,ξ)≤p+(n−1)((M(ξ)+Δ−1)−1M(ξ)), (20)

for all , and equality in (20) for all support points. This condition coincides with (18) for and (19) for , which implies for both design regions the equivalence of the D- and G-criteria in the linear regression model (14) with the diagonal covariance structure.

4 Discussion

We have discussed Kiefer’s the -criteria and the global (G-) criterion in RCR models. The obtained general form of the Kiefer’s criterion can be recognized as the weighted sum of the Kiefer’s criterion in fixed effects and Bayesian models, where the weight of the Baysian part increases with increasing number of individuals. For the E-criterion (particular Kiefer’s -criterion with ), it was proved that optimal designs in fixed effects models retain their optimality for the prediction in RCR models. The G-criterion cannot be factorized in the fixed effects and Bayesian parts. This criterion has been discussed in detail for ordinary linear regression on specific experimental regions, , , and , . For this special case, the equivalence of D- and G-criteria has been established. However, the equivalence of these two criteria does not hold in general for RCR models.

Acknowledgments

This research has been supported by grant SCHW 531/16-1 of the German Research Foundation (DFG). The author thanks to Radoslav Harman and Rainer Schwabe for fruitful discussions.

References

• Atkinson et al. (2007) Atkinson, A. C., Donev, A. N., and Tobias, R. D. (2007). Optimum Experimental Designs, with SAS. Oxford University Press, Oxford.
• Fedorov and Leonov (2013) Fedorov, V. and Leonov, S. (2013). Optimal Design for Nonlinear Response Models. CRC Press, Boca Raton.
• Kiefer (1974) Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Annals of Statistics, 2, 849–879.
• Kiefer and Wolfowitz (1960) Kiefer, J. and Wolfowitz, J. (1960). The equivalence of two extremum problems. Canadian Journal of Mathematics, 12, 363–366.
• Prus (2015) Prus, M. (2015). Optimal Designs for the Prediction in Hierarchical Random Coefficient Regression Models. Ph.D. thesis, Otto-von-Guericke University, Magdeburg.
• Prus and Schwabe (2016a) Prus, M. and Schwabe, R. (2016a). Interpolation and extrapolation in random coefficient regression models: Optimal design for prediction. mODa 11 - Advances in Model-Oriented Design and Analysis, pages 209–216.
• Prus and Schwabe (2016b) Prus, M. and Schwabe, R. (2016b). Optimal designs for the prediction of individual parameters in hierarchical models. Journal of the Royal Statistical Society: Series B, 78, 175–191.
• Wong (1995) Wong, W. K. (1995). On the equivalence of d and g-optimal designs in heteroscedastic models.

Statistics and Probability Letters

, 25, 317–321.