Differences of Type I error rates using SAS and SPSS for repeated measures designs
share
We examined Type I error rates of Multilevel Linear Models (MLM) and repeated measures Analysis of Variance (rANOVA) as well as their correction methods for nine and twelve measurement occasions using the SAS and SPSS software packages. As MLM has been proposed for the analysis of small samples, we performed a simulation study with the following specifications: To explore the effect of both the number of measurement occa-sions and the sample size on Type I error, measurement occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15, 20, 25 and 30. The effects of non-sphericity in the population on Type I error for different analysis methods were also examined: 5,000 random samples were drawn from populations that contained neither a within-subject nor a between-group effect. The samples were analyzed with different methods to include the most common options to correct rANOVA and MLM-results: The correction by Huynh and Feldt (1976) for rANOVA (rANOVA-HF) as well as Kenward and Roger's (1997, 2009) correction for unstructured MLM (MLM-UN-KR), which could help to correct progressive bias of uncorrected MLM with an unstructured covari-ance matrix (MLM-UN). The resulting mean Type I error rates showed a progressive bias for the uncorrected rANOVA and heterogeneous results for MLM-UN when sphericity was violated as well as a general progressive bias for MLM-UN for small sample sizes which was stronger in SPSS than in SAS. Moreover, an appropriate correction for Type I error via rANOVA-HF and an insufficient bias correction by MLM-UN-KR for n < 30 were found. The findings plead for a use of rANOVA or MLM-CS if sphericity is not violated and for a correc-tion of a violation rather via rANOVA-HF than MLM-UN-KR. If an analysis requires MLM, how-ever, SPSS yields more accurate Type I error rates for MLM-CS and SAS yields more accurate Type I error rates for MLM-UN.
READ FULL TEXT