# Equivalence theorems for compound design problems with application in mixed models

In the present paper we consider design criteria which depend on several designs simultaneously. We formulate equivalence theorems based on moment matrices (if criteria depend on designs via moment matrices) or with respect to the designs themselves (for finite design regions). We apply the obtained optimality conditions to the multiple-group random coefficient regression models and illustrate the results by simple examples.

## Authors

• 8 publications
08/12/2018

### Various Optimality Criteria for the Prediction of Individual Response Curves

We consider optimal designs for the Kiefer cirteria, which include the E...
12/22/2018

### Optimal Designs for Prediction in Two Treatment Groups Random Coefficient Regression Models

The subject of this work is two treatment groups random coefficient regr...
07/26/2018

### Optimal Designs in Multiple Group Random Coefficient Regression Models

The subject of this work is multiple group random coefficients regressio...
01/29/2019

### Representation theorems for extended contact algebras based on equivalence relations

The aim of this paper is to give new representation theorems for extende...
10/28/2017

### Optimal designs for regression with spherical data

In this paper optimal designs for regression problems with spherical pre...
03/27/2022

### Optimal Design for Estimating the Mean Ability over Time in Repeated Item Response Testing

We present general results on D-optimal designs for estimating the mean ...
12/16/2019

### Robust Adaptive Least Squares Polynomial Chaos Expansions in High-Frequency Applications

We present an algorithm for computing sparse, least squares-based polyno...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

The subject of this work is compound design problems - optimization problems with optimality criteria depending on several designs simultaneously. Such optimality criteria can be, for example, commonly used design criteria for estimation of unknown model parameters in case when the covariance matrix of the estimation depends on several designs (see e. g.

Fedorov and Jones (2005), Schmelter (2007a)). For such criteria general equivalence theorem proposed in Kiefer (1974) cannot be used directly. In Fedorov and Jones (2005) optimal designs were obtained for specific regression functions. In Schmelter (2007a) particular group-wise identical designs have been discussed.

In this paper we formulate equivalence theorems for two kinds of compound design problems: 1) problems on finite experimental regions and 2) problems with optimality criteria depending on designs via moment (or information) matrices. For both cases we assume the optimality criteria to be convex and differentiable in the designs themselves or the moment matrices, respectively. In case 1) we formulate optimality conditions with respect to the designs directly (as proposed in Whittle (1973) for one-design problems). These results can be useful in situations when design criteria cannot be presented as functions of moment matrices (see e. g. Bose and Mukerjee (2015)). In case 2) optimality conditions are formulated with respect to the moment matrices. Therefore, no additional restrictions of the experimental regions are needed.

We apply the equivalence theorems to multiple-group random coefficient regression (RCR) models. In these models observational units (individuals) are assigned to several groups. Within one group same designs (group-designs) for all individuals have been assumed. Group-designs for individuals from different groups are in general not the same. Most of commonly used design criteria in multiple-group RCR models are functions of several group-designs. The particular case of these models with one observation per individual has been considered in Graßhoff et al. (2012). In Prus (2015), ch. 6, models with group-specific mean parameters were briefly discussed. Bludowsky et al. (2015), Kunert et al. (2010), Lemme et al. (2015) and Prus (2019) considered models with particular regression functions and specific covariance structure of random effects. In Entholzner et al. (2005) and Prus and Schwabe (2016) same design for all observational units have been assumed.

The paper has the following structure: Section 2 provides equivalence theorems for the compound design problems. In Section 3 we apply the obtained optimality conditions to the multiple-group RCR models. In Section 4 we illustrate the results by a simple example. The paper is concluded by a short discussion in Section 5.

## 2 Optimality Conditions for Compound Design Problems

We consider a compound design problem in which

are probability measures (designs) on experimental regions

, respectively, and is a design criterion which depends on simultaneously and has to be minimized. denotes the set of all designs on , . For any , denotes the particular design with all observations at point . For convenience we use the notation

for a vector of designs

, . Then for , where ”” denotes the Cartesian product.

In Section 2.1 we consider compound design problems, where all design regions are assumed to be finite. We formulate an equivalence theorem (Theorem 1) with respect to the designs directly.

In Section 2.2 we consider design criteria depending on designs via moment matrices and we propose an equivalence theorem (Theorem 2) based on this structure. In this case no additional restrictions of the experimental regions are needed.

### 2.1 Optimality conditions in case of finite design regions

In this section we restrict ourselves on optimization problems on finite design regions: for all . denotes a design criterion. We use the notation for the directional derivative of at in direction of :

 Φ(\boldmath{ξ},~\boldmath{ξ})=limα↘01α(ϕ((1−α)\boldmath{ξ}+α~\boldmath{ξ})−ϕ(\boldmath{ξ})). (1)

Further we define the partial directional derivative of at in direction of as follows:

 Φξi′,i′≠i(ξi,~ξi)=Φ(% \boldmath{ξ},˘\boldmath{ξ}), (2)

where with , , , and .

###### Theorem 1.

Let be convex and differentiable.

1. The following statements are equivalent:

1. minimizes

2. .

2. Let minimize . Let be a support point of , . Then .

3. Let minimize . Then the point is a saddle point of in that

 Φ(\boldmath{ξ}∗,\boldmath{ξ})≥0=Φ(% \boldmath{ξ}∗,\boldmath{ξ}∗)≥Φ(~% \boldmath{ξ},\boldmath{ξ}∗),∀\boldmath{% ξ},~\boldmath{ξ}∈Ξ (3)

and the point is a saddle point of in that

 Φξ∗i′,i′≠i(ξ∗i,ξi)≥0=Φξ∗i′,i′≠i(ξ∗i,ξ∗i)≥Φξ∗i′,i′≠i(~ξi,ξ∗i),∀ξi,~ξi∈Ξi, (4)

for all .

###### Proof.
1. (i)(ii):

For this proof we present designs in form of row vectors , where is the weight of observations at , the -th point of the experimental region , , (see also Boyd and Vandenberghe (2004), ch. 7). Then is the full (row) vector of all weights of observations at all points of all experimental regions.

We use the notations for the gradient of with respect to : , and for the gradient of with respect to , which means . (Gradients and are row vectors).

According to convex optimization theory (see e. g. Boyd and Vandenberghe (2004), ch. 4) minimizes iff for all . Then the equivalence of (i) and (ii) follows from

 Φ(\boldmath{ξ}∗,\boldmath{ξ}) = ∇ξϕ(\boldmath{ξ}∗)(\boldmath{ξ}−\boldmath{ξ}∗)⊤ = s∑i=1∇ξiϕ(\boldmath{ξ}∗)(ξi−ξ∗i)⊤ = s∑i=1Φξ∗i′,i′≠i(ξ∗i,ξi).

(ii)(iii): (iii)(ii) Straightforward

(ii)(iii): Let with . Let and , . Then for all we have , which results in

 s∑i=1Φξ∗i′,i′≠i(ξ∗i,ξi) = Φξ∗i′,i′≠i(ξ∗i1,~ξi1)+∑i∈{1,…,s}∖i1Φξ∗i′,i′≠i(ξ∗i,ξ∗i) = Φξ∗i′,i′≠i(ξ∗i1,~ξi1)<0.

(iii)(iv): (iii)(iv) Straightforward

(iv)(iii) Let be the -th point in , . Then the one-point design with all observations at is given by , where is the -th unit (row) vector of length . A design can be written as . Then the directional derivative of at in direction of can be presented in form

 Φξ∗i′,i′≠i(ξ∗i,ξi)=ki∑t=1wit∇ξiϕ(ξ∗)(et−ξ∗i)⊤,

which results in

 Φξ∗i′,i′≠i(ξ∗i,ξi)=ki∑t=1witΦξ∗i′,i′≠i(ξ∗i,δxit)≥0.
2. Let the support point be the -th point in , . Then for we have and

 Φξ∗i′,i′≠i(ξ∗i,ξ∗i)=ki∑t=1w∗itΦξ∗i′,i′≠i(ξ∗i,δxit).

Let . Then since , , we obtain .

3. The left-hand sides of both (3) and (4) are straightforward. From formula (1) and convexity of we obtain

 Φ(~\boldmath{ξ},\boldmath{ξ}∗)≤ϕ(\boldmath{ξ}∗)−ϕ(~\boldmath{ξ}),

which is non-positive for optimal and all . Similarly using formula (2) we obtain the right-hand side of (4).

### 2.2 Optimality conditions based on moment matrices

We use the notation for a matrix which characterizes a design . We assume to satisfy the condition

 Mi(ξi)=∫XiMi(δxi)ξi(dxi) (5)

for all . We call this matrix moment matrix of a design (see e. g. Pukelsheim (1993)). denotes the set of all moment matrices , . For , denotes the set of all , . Then and is convex. is a design criterion. denotes the directional derivative of at in direction of :

 Φ(M,~M)=limα↘01α(ϕ((1−α)M+α~M)−ϕ(M)). (6)

We define the partial directional derivative of at in direction of as follows:

 ΦMi′,i′≠i(Mi,~Mi)=Φ(M,˘M), (7)

where , , , and .

###### Theorem 2.

Let be convex and differentiable.

1. The following statements are equivalent:

1. minimizes

2. .

2. Let minimize . Let be a support point of , . Then .

3. Let minimize . Then the point is a saddle point of in that

 Φ(M(\boldmath{ξ}∗),M(\boldmath{ξ}% ))≥0=Φ(M(\boldmath{ξ}∗),M(% \boldmath{ξ}∗))≥Φ(M(~\boldmath{ξ}),M(\boldmath{ξ}∗)),∀\boldmath{ξ},~\boldmath{ξ}∈Ξ (8)

and the point is a saddle point of in that

 ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξi))≥0 = ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξ∗i)) (9) ≥ΦMi′(ξ∗i′),i′≠i(Mi(~ξi),Mi(ξ∗i)),∀ξi,~ξi∈Ξi,

for all .

###### Proof.
1. (i)(ii):

We use for the gradients of with respect to and the notation

 ∇Miϕ=(∂ϕ∂mkl)k,l,Mi=(mkl)k,l

and

 ∇Mϕ=(∂ϕ∂mkl)k,l,M=(mkl)k,l,

respectively.

minimizes iff for all . The directional derivative can be computed by formula

 Φ(M,~M)=∂ϕ∂α((1−α)M+α~M)|α=0. (10)

Then using some matrix differentiation rules (see e. g. Seber (2007), ch. 17) we receive

 Φ(M(\boldmath{ξ}∗),M(% \boldmath{ξ})) = tr(∇Mϕ(M(\boldmath{ξ}% ∗))(M(\boldmath{ξ})−M(\boldmath{ξ}∗))⊤) = s∑i=1tr(∇Miϕ(M(\boldmath{ξ}∗))(Mi(ξi)−Mi(ξ∗i))) = s∑i=1ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξi)),

which implies the equivalence of (i) and (ii).

(ii)(iii): (iii)(ii) Straightforward

(ii)(iii): Let with . Then for and , , we obtain

 s∑i=1ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξi)) = ΦMi′(ξ∗i′),i′≠i(Mi1(ξ∗i1),Mi1(~ξi1)) +∑i∈{1,…,s}∖i1ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξi)) = ΦMi′(ξ∗i′),i′≠i(Mi1(ξ∗i1),Mi1(~ξi1))<0.

(iii)(iv): (iii)(iv) Straightforward

(iv)(iii) The directional derivative of at in direction of is linear in the second argument:

 ΦMi′,i′≠i(Mi,~Mi)=tr(∇Miϕ(M)(~Mi−Mi)).

Then using formula (5) we obtain

 ΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(ξi))=∫XiΦMi′(ξ∗i′),i′≠i(Mi(ξ∗i),Mi(δxi))ξi(dxi) (11)

for each .

2. The result follows from formula (11), , for all , and .

3. The left-hand sides of both (8) and (9) are straightforward. From convexity of and formula (6) we obtain

 Φ(M(~\boldmath{ξ}),M(\boldmath{ξ}∗))≤ϕ(M(\boldmath{ξ}∗))−ϕ(M(~\boldmath{ξ})),∀~\boldmath{ξ}∈Ξ,

which implies the right-hand side of (8). Similarly using formula (7) we obtain the right-hand side of (9).

## 3 Optimal Designs in Multiple-Group RCR Models

We consider multiple-group RCR models in which observational units are assigned to groups: observational units in the -th group, . The group sizes and the group allocation of observational units are fixed. Experimental designs are assumed to be the same for all observational units within one group (group-design): observations per unit in design points , , in group . However, for units from different groups experimental designs are in general not the same: and (or) , .

Note that the experimental settings in group are not necessarily all distinct (repeated measurements are not excluded).

Note also that observational units (often called individuals in the literature) are usually expected to be people, animals or plants. However, they may also be studies, centers, clinics, plots, etc.

### 3.1 Model specification and estimation of unknown parameters

In multiple-group random coefficient regression models the -th observation of the -th observational unit in the -th group is given by the following -dimensional random column vector

 Yijh=Gi(xih)\boldmath{β}ij+\boldmath{ε}ijh,xih∈Xi,h=1,…,mi,j=1,…,ni,i=1,…,s, (12)

where denotes a group-specific () matrix of known regression functions in group (in particular case : , where is a -dimensional column vector of regression functions), experimental settings come from some experimental region , are unit-specific random parameters with unknown mean and given () covariance matrix , denote column vectors of observational errors with zero mean and non-singular () covariance matrix . All observational errors and all random parameters are assumed to be uncorrelated.

For the vector of observations at the -th observational unit in the -th group we obtain

 Yij=Fi\boldmath{β}ij+\boldmath{ε}ij,j=1,…,ni,i=1,…,s, (13)

where is the design matrix in group and . Then the vector of all observations in group is given by

 Yi=(Ini⊗Fi)% \boldmath{β}i+\boldmath{ε}i,i=1,…s, (14)

where , , is the identity matrix and ”” denotes the Kronecker product, and the total vector of all observations in all groups results in

 Y=⎛⎜⎝1n1⊗F1…1ns⊗Fs⎞⎟⎠\boldmath{β}0+~\boldmath{ε} (15)

with

where is the block-diagonal matrix with blocks , for , and is the column vector of length with all entries equal to .

Using Gauss-Markov theory we obtain the following best linear unbiased estimator for the mean parameters

:

 ^\boldmath{β}0=[s∑i=1ni((~F⊤i~Fi)−1+Di)−1]−1s∑i=1ni((~F⊤i~Fi)−1+Di)−1^\boldmath{β}0,i, (16)

where is the estimator based only on observations in the -th group, and (for the mean observational vector in the -th group and the symmetric positive definite matrix with the property ) are the transformed design matrix and the transformed mean observational vector with respect to the covariance structure of observational errors.

Note that BLUE (16) exists only if all matrices are non-singular. Therefore, we restrict ourselves on the case where design matrices are of full column rank for all .

The covariance matrix of the best linear unbiased estimator is given by

 Cov(^\boldmath{β}0)=[s∑i=1ni((~F⊤i~Fi)−1+Di)−1]−1. (17)

In Fedorov and Jones (2005) similar results were obtained for the multi-center trials models.