
Covariance Matrix Estimation under Total Positivity for Portfolio Selection
Selecting the optimal Markowitz porfolio depends on estimating the covar...
read it

Minimax Estimation of Bandable Precision Matrices
The inverse covariance matrix provides considerable insight for understa...
read it

Covariance Matrix Estimation with Non Uniform and Data Dependent Missing Observations
In this paper we study covariance estimation with missing data. We consi...
read it

Semiparametric Inference for Nonmonotone MissingNotatRandom Data: the No SelfCensoring Model
We study the identification and estimation of statistical functionals of...
read it

An Apparent Paradox: A Classifier Trained from a Partially Classified Sample May Have Smaller Expected Error Rate Than That If the Sample Were Completely Classified
There has been increasing interest in using semisupervised learning to ...
read it

Graph quilting: graphical model selection from partially observed covariances
We investigate the problem of conditional dependence graph estimation wh...
read it

Generalised Wishart Processes
We introduce a stochastic process with Wishart marginals: the generalise...
read it
Estimating Highdimensional Covariance and Precision Matrices under General Missing Dependence
A sample covariance matrix S of completely observed data is the key statistic in a large variety of multivariate statistical procedures, such as structured covariance/precision matrix estimation, principal component analysis, and testing of equality of mean vectors. However, when the data are partially observed, the sample covariance matrix from the available data is biased and does not provide valid multivariate procedures. To correct the bias, a simple adjustment method called inverse probability weighting (IPW) has been used in previous research, yielding the IPW estimator. The estimator plays the role of S in the missing data context so that it can be plugged into offtheshelf multivariate procedures. However, theoretical properties (e.g. concentration) of the IPW estimator have been only established under very simple missing structures; every variable of each sample is independently subject to missing with equal probability. We investigate the deviation of the IPW estimator when observations are partially observed under general missing dependency. We prove the optimal convergence rate O_p(√(log p / n)) of the IPW estimator based on the elementwise maximum norm. We also derive similar deviation results even when implicit assumptions (known mean and/or missing probability) are relaxed. The optimal rate is especially crucial in estimating a precision matrix, because of the "metatheorem" that claims the rate of the IPW estimator governs that of the resulting precision matrix estimator. In the simulation study, we discuss nonpositive semidefiniteness of the IPW estimator and compare the estimator with imputation methods, which are practically important.
READ FULL TEXT
Comments
There are no comments yet.