Linear regression model with measurement errors in independent variables is of practical importance, and many theoretical and experimental approaches have been studied extensively for a long time. Adcock (1877, 1878) first treated estimation of the slope in a simple linear measurement error model and derived the maximum likelihood (ML) estimator, which nowadays is known as orthogonal regression estimator (see Anderson (1984)). Reiersøl (1950) has investigated identifiability related to possibility of constructing a consistent estimator. For efficient estimation, see Bickel and Ritov (1987) and, for consistent estimation based on shrinkage estimators, see Whittemore (1989) and Guo and Ghosh (2012). A multivariate generalization of univariate linear measurement error model has been considered by Gleser (1981). See Anderson (1984), Fuller (1987) and Cheng and Van Ness (1999) for a systematic overview of theoretical development in estimation of linear measurement error models.
Even though many estimation procedures for the slope have been developed and proposed, each procedure generally has both theoretical merits and demerits. The ML estimator possesses consistency and asymptotic normality. However, the first moment of the ML estimator does not exist and it is hard to theoretically investigate finite-sample properties of the ML procedure. Besides the ML procedure, the most well-known procedure may be the least squares (LS) procedure. The ordinary LS estimator has finite moments up to some order, but is not asymptotically unbiased. The asymptotic biasedness of the LS estimator is called attenuation bias in the literature (see Fuller (1987)).
This paper addresses a simple linear measurement error model in a finite sample setup, and discusses the problem of reducing the bias and the mean square error (MSE) for slope estimators. Suppose that the and the are observable variables for and , where is the number of groups and is the sample size of each group. Suppose also that the and the have the following model:
where and are, respectively, unknown intercept and slope parameters, the are unobservable latent variables, and the and the are random error terms. Assume that the and the are mutually independent and distributed as and , respectively, where and
are unknown. It is important to note that the error variance in independent variables,, can be estimated.
For the latent variables in model (1.1), there are two different points of view, namely, the
are considered as unknown fixed values or as random variables. In the former case, (1.1) is referred to as a functional model and, in the latter case, is called a structural model (Kendall and Stuart (1979), Anderson (1984) and Cheng and Van Ness (1999)). In this paper, we assume the functional model and shall develop a finite-sample theory of estimating the slope .
The remainder of this paper is organized as follows. In Section 2, we simplify the estimation problem in model (1.1), and define a broad class of slope estimators including the LS estimator, the method of moments estimator, and a Stefanski’s (1985) estimator. Also, Section 2 shows some technical lemmas used for evaluating moments. Section 3 presents a unified method of reducing the bias of the broad class as well as that of the LS estimator. In Section 4, we handle the problem of reducing the MSEs of slope estimators. It is revealed that the slope estimation under the MSE criterion is closely related to the statistical control problem (see Zellner (1971) and Aoki (1989)) and also to the multivariate calibration problem (see Osborne (1991), Brown (1993) and Sundberg (1999)). Our approach to the MSE reduction is carried out in a similar way to Kubokawa and Robert (1994), and a general method is established for improvement of several estimators such as the LS estimator and Guo and Ghosh’s (2012) estimator. Section 5 illustrates numerical performance for the biases and the MSEs of alternative estimators. In Section 6, we point out some remarks on our results and related topics.
2 Simplification of the estimation problem
2.1 Reparametrized model
Define for . Consider the regression of the on the . The LS estimator of is defined as a unique solution of
Denote by the resulting ordinary LS estimator of . Then and are given, respectively, by
where and .
Let , and . Define
the identity matrix of orderand by the
-dimensional vector consisting of ones. It is then observed that
for and . Note that , and are mutually independent.
These five statistics, and , are mutually independent, and , , , , and are unknown parameters. Throughout this paper, we suppose that .
From reparametrized model (2.2), the ordinary LS estimators and can be rewritten, respectively, as
Hereafter, we mainly deal with the problem of estimating in reparametrized model (2.2). Denote the bias and the MSE of an estimator , respectively, by
where the expectation is taken with respect to (2.2). The bias of is smaller than that of another estimator if for any . Similarly, if for any , then the MSE of is said to be better than that of , or is said to dominate .
2.2 A class of estimators
If where is a positive value, it follows that and
in probability astends to infinity, and hence
This implies that the ordinary LS estimator is inconsistent and, more precisely, it is asymptotically biased toward zero. This phenomenon is called attenuation bias (see Fuller (1987)).
For reducing the influence of attenuation bias, various alternatives to have been proposed in the literature. For example, a typical alternative is the method of moments estimator
The method of moments estimator converges to in probability as goes to infinity, but does not have finite moments. Noting that and also using the Maclaurin expansion , we obtain the -th order corrected estimator of the form
The above estimator can also be derived from using the same arguments as in Stefanski (1985), who approached to the bias correction from Huber’s (1981) M estimation. However, it is still not known whether or not the bias of is smaller than that of in a finite sample situation.
Convergence (2.4) is equivalent that converges to in probability as goes to infinity. Replacing of with a suitable function of yields a general class of estimators,
2.3 Some useful lemmas
Next, we provide some technical lemmas which form the basis for evaluating the bias and MSE of (2.7).
Let and . Let be a function on the positive real line. Define and denote by the Poisson probabilities for Let be the p.d.f. of .
If then we have
If then we have
When , (i) and (ii) of Lemma 2.1 are, respectively,
Proof of Lemma 2.1. (i) Denote
Let . It turns out that
Denote . Let be a orthogonal matrix whose first row is . Making the orthogonal transformation gives that
Now, for , we make the following polar coordinate transformation
where , , and . The Jacobian of transformation is given by , so (2.10) can be rewritten as
Note here that, for an even ,
and, for an odd, the above definite integral is zero. Thus, it is seen that
The change of variables leads to completeness of the proof of (i).
Using the same arguments as in the proof of (i), we obtain
it is observed that
Hence the proof of (ii) is complete. ∎
Let . Let be a natural number such that . Denote by the Poisson random variable with mean . Then we have
Proof. We employ the same notation as in Lemma 2.1. Note that, when ,
follows the noncentral chi-square distribution withdegrees of freedom and noncentrality parameter . Since the p.d.f. of the noncentral chi-square distribution is given by , it is seen that
for . If , then , so that for . Thus the proof is complete. ∎
The following lemma is given in Hudson (1978).
Let be a Poisson random variable with mean . Let be a function satisfying and . Then we have .
3 Bias reduction
In this section, some results are presented for the bias reduction in slope estimation. First, we give an alternative expression for the bias of the LS estimator .
Let be a Poisson random variable with mean . If , then the bias of is finite. Furthermore, if , the bias of can be expressed as
Proof. Using identity (2.8) gives that for
Hence the proof is complete. ∎
Let be a nonnegative integer. Define a simple modification of , given in (2.6), as
where and for , and . We then obtain the following lemma.
Let be a Poisson random variable with mean . Assume that . If , then can be expressed as
Proof. We prove a case when because the case is equivalent to Lemma 3.1. Note that
which implies from Lemma 3.1 that
which is substituted into (3.3) to obtain
It is here observed that
which yields that, for ,
Hence the proof is complete. ∎
The following theorem specifies a general condition that , given in (2.7), reduces the bias of in a finite sample setup.
Assume that . Let the and the be defined as in (3.2). Assume that is bounded as for any and a fixed natural number . If , then we have for any .