Machine Learning-Based Estimation and Goodness-of-Fit for Large-Scale Confirmatory Item Factor Analysis

by   Christopher J. Urban, et al.

We investigate novel parameter estimation and goodness-of-fit (GOF) assessment methods for large-scale confirmatory item factor analysis (IFA) with many respondents, items, and latent factors. For parameter estimation, we extend Urban and Bauer's (2021) deep learning algorithm for exploratory IFA to the confirmatory setting by showing how to handle user-defined constraints on loadings and factor correlations. For GOF assessment, we explore new simulation-based tests and indices. In particular, we consider extensions of the classifier two-sample test (C2ST), a method that tests whether a machine learning classifier can distinguish between observed data and synthetic data sampled from a fitted IFA model. The C2ST provides a flexible framework that integrates overall model fit, piece-wise fit, and person fit. Proposed extensions include a C2ST-based test of approximate fit in which the user specifies what percentage of observed data can be distinguished from synthetic data as well as a C2ST-based relative fit index that is similar in spirit to the relative fit indices used in structural equation modeling. Via simulation studies, we first show that the confirmatory extension of Urban and Bauer's (2021) algorithm produces more accurate parameter estimates as the sample size increases and obtains comparable estimates to a state-of-the-art confirmatory IFA estimation procedure in less time. We next show that the C2ST-based test of approximate fit controls the empirical type I error rate and detects when the number of latent factors is misspecified. Finally, we empirically investigate how the sampling distribution of the C2ST-based relative fit index depends on the sample size.



There are no comments yet.


page 1

page 2

page 3

page 4


Joint Maximum Likelihood Estimation for High-dimensional Exploratory Item Response Analysis

Multidimensional item response theory is widely used in education and ps...

The Lazy Bootstrap. A Fast Resampling Method for Evaluating Latent Class Model Fit

The latent class model is a powerful unsupervised clustering algorithm f...

Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implications

Latent factor models are widely used to measure unobserved latent traits...

A Correlation Thresholding Algorithm for Learning Factor Analysis Models

Factor analysis is a widely used method for modeling a set of observed v...

Improving the Hosmer-Lemeshow Goodness-of-Fit Test in Large Models with Replicated Trials

The Hosmer-Lemeshow (HL) test is a commonly used global goodness-of-fit ...

Approximate Bayesian Computations to fit and compare insurance loss models

Approximate Bayesian Computation (ABC) is a statistical learning techniq...

Informative Features for Model Comparison

Given two candidate models, and a set of target observations, we address...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.