A1241
Title: Testing of variance components in nonlinear mixed models with penalized splines
Authors: Germaine Uwimpuhwe - Durham University (United Kingdom) [presenting]
Reza Drikvandi - Durham University (United Kingdom)
Abstract: Generalized additive models provide a flexible framework for modelling nonlinear relationships when the functional form is unknown. Using penalized spline representations within a nonlinear mixed-effects model (NLMM), smooth functions can be expressed via spline bases with coefficients treated as random effects. This formulation enables joint modelling of nonlinear covariate effects and subject-specific variability. The focus is on testing variance components associated with both spline-related random effects (capturing nonlinear trends) and subject-specific random effects (capturing within-subject correlation). A major challenge in this setting is that, under the null hypothesis, variance components lie on the boundary of the parameter space. As a result, classical likelihood ratio tests are not valid, and standard asymptotic results such as chi-squared or mixture chi-squared distributions do not apply, particularly when spline coefficients are defined at the population level. Moreover, resampling-based approaches such as bootstrap or permutation are not straightforward to implement under these constraints. To address this, a novel non-parametric testing procedure for variance components in such NLMMs is proposed. The method avoids strong distributional assumptions and remains applicable in complex settings. The proposed approach is evaluated through simulation experiments.
