Title: A fast automatically calibrated resampling method for evaluating multinomial model fit
Authors: Ioulia Papageorgiou - Athens University of Economics and Business (Greece) [presenting]
Abstract: Many tests of goodness-of-fit can be reduced to testing a hypothesis about the parameters from a multinomial distribution. However, the classic multinomial goodness-of-fit test based on Pearson's $\chi^2$ is only asymptotic and may be biased in small samples or when the contingency table is sparse. A general goodness-of-fit approach has been recently proposed in the literature using calibrated simulation. Comparative data are generated based on the maximum-likelihood estimates from the observed data to assess systematic discrepancies between observed and simulated data from the fitted model. In common with the posterior predictive $p$-value in Bayesian statistics, the $p$-value of the approach is generally non-uniform under the null model, and calibration is required to check its significance. We introduce a simple variant of the calibrated simulation approach and provide theory-based modifications that result in uniform $p$-values in multinomial goodness-of-fit testing, thereby removing the need for calibration. We illustrate and evaluate the method in a variety of contexts including item response theory modelling, latent class analysis and capture-recapture data modelling. The new method is shown to have nominal type I error rates whilst being computationally much less intensive than alternative resampling methods.