Uncertainty Quantification for Multi-Level Models Using the Survey-Weighted Pseudo-Posterior
Pub. online: 21 July 2026
Type: Computing In Data Science
Open Access
Open Access
Received
7 October 2025
7 October 2025
Accepted
19 June 2026
19 June 2026
Published
21 July 2026
21 July 2026
Abstract
Parameter estimation and inference from complex survey samples typically focuses on global model parameters whose estimators have asymptotic properties, such as from fixed effects regression models. The central challenge is to both mitigate bias induced from potentially unbalanced samples and to incorporate adjustments for differences in effective sample size to get correct variance and interval estimates. We present a motivating example of Bayesian inference for a multi-level or mixed effects model in which estimates of both the local parameters (e.g. group level random effects) and the global parameters need to be adjusted for the complex sampling design. We evaluate the limitations of the survey-weighted pseudo-posterior and an existing automated post-processing method to improve the uncertainty quantification. We propose modifications to the automated process and demonstrate their improvements for multi-level models via a simulation study and a motivating example from the National Survey on Drug Use and Health. Reproduction examples are included in the supplementary material and the updated R package is available via github: https://github.com/RyanHornby/csSampling
Supplementary material
Supplementary MaterialThe supplementary material includes a summary of the Bernstein-von Mises result from Williams and Savitsky (2021). It also includes additional diagnostic summaries for the simulation study and alternative analyses of the NSDUH application. Also included is R code for replicating the simulation studies and R code and data for replicating the NSDUH example. The csSampling package has been updated with the new functionality described in this paper and can be downloaded from github: https://github.com/RyanHornby/csSampling
References
Box GE, Cox DR (1964). An analysis of transformations. Journal of the Royal Statistical Society, Series B, Statistical Methodology, 26(2): 211–243. https://doi.org/10.1111/j.2517-6161.1964.tb00553.x
Breslow NE, Wellner JA (2007). Weighted likelihood for semiparametric models and two-phase stratified samples, with application to Cox regression. Scandinavian Journal of Statistics, 34(1): 86–102. https://doi.org/10.1111/j.1467-9469.2006.00523.x
Bürkner PC (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1): 1–28. https://doi.org/10.18637/jss.v080.i01
Goldstein H, Browne W, Rasbash J (2002). Partitioning variation in multilevel models. Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 1(4): 223–231. https://doi.org/10.1207/S15328031US0104_02
Han Q, Wellner JA (2021). Complex sampling designs: Uniform limit theorems and applications. The Annals of Statistics, 49(1): 459–485. https://doi.org/10.1214/20-AOS1964
Isaki CT, Fuller WA (1982). Survey design under the regression superpopulation model. Journal of the American Statistical Association, 77: 89–96. https://doi.org/10.1080/01621459.1982.10477770
Lee J (2026). Design effect ratios for bayesian survey models: A diagnostic framework for identifying survey-sensitive parameters. arXiv preprint arXiv:2603.07791.
Lele SR, Nadeem K, Schmuland B (2010). Estimability and likelihood inference for generalized linear mixed models using data cloning. Journal of the American Statistical Association, 105(492): 1617–1625. https://doi.org/10.1198/jasa.2010.tm09757
McGuire FH, Beccia AL, Peoples JE, Williams MR, Schuler MS, Duncan AE (2024). Depression at the intersection of race/ethnicity, sex/gender, and sexual orientation in a nationally representative sample of us adults: A design-weighted intersectional maihda. American Journal of Epidemiology, 193(12): 1662–1674. https://doi.org/10.1093/aje/kwae121
Robbins MW, Ghosh SK, Habiger JD (2013). Imputation in high-dimensional economic data as applied to the agricultural resource management survey. Journal of the American Statistical Association, 108(501): 81–95. https://doi.org/10.1080/01621459.2012.734158
Savitsky TD, Toth D (2016). Bayesian estimation under informative sampling. Electronic Journal of Statistics, 10(1): 1677–1708. https://doi.org/10.1214/16-EJS1153
Williams MR, Savitsky TD (2020). Bayesian estimation under informative sampling with unattenuated dependence. Bayesian Analysis, 15(1): 57–77. https://doi.org/10.1214/18-BA1143
Williams MR, Savitsky TD (2021). Uncertainty estimation for pseudo-Bayesian inference under complex sampling. International Statistical Review, 89(1): 72–107. https://doi.org/10.1111/insr.12376
Yeo I, Johnson RA (2000). A new family of power transformations to improve normality or symmetry. Biometrika, 87(4): 954–959. https://doi.org/10.1093/biomet/87.4.954
