<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.0 20120330//EN" "JATS-journalpublishing1.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">JDS</journal-id>
<journal-title-group><journal-title>Journal of Data Science</journal-title></journal-title-group>
<issn pub-type="epub">1683-8602</issn><issn pub-type="ppub">1680-743X</issn><issn-l>1680-743X</issn-l>
<publisher>
<publisher-name>School of Statistics, Renmin University of China</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">JDS1238</article-id>
<article-id pub-id-type="doi">10.6339/26-JDS1238</article-id>
<article-categories><subj-group subj-group-type="heading">
<subject>Computing in Data Science</subject></subj-group></article-categories>
<title-group>
<article-title>Uncertainty Quantification for Multi-Level Models Using the Survey-Weighted Pseudo-Posterior</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0001-8894-1240</contrib-id>
<name><surname>Williams</surname><given-names>Matthew R.</given-names></name><email xlink:href="mailto:mrwilliams@rti.org">mrwilliams@rti.org</email><xref ref-type="aff" rid="j_jds1238_aff_001">1</xref><xref ref-type="corresp" rid="cor1">∗</xref>
</contrib>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0003-0522-2748</contrib-id>
<name><surname>McGuire</surname><given-names>F. Hunter</given-names></name><xref ref-type="aff" rid="j_jds1238_aff_001">1</xref>
</contrib>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0003-1843-3106</contrib-id>
<name><surname>Savitsky</surname><given-names>Terrance D.</given-names></name><xref ref-type="aff" rid="j_jds1238_aff_002">2</xref>
</contrib>
<aff id="j_jds1238_aff_001"><label>1</label><institution>RTI International</institution>, Durham, NC, 27709, <country>USA</country></aff>
<aff id="j_jds1238_aff_002"><label>2</label><institution>U.S. Bureau of Labor Statistics</institution>, Suitland, MD 20746, <country>USA</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>∗</label>Corresponding author. Email: <ext-link ext-link-type="uri" xlink:href="mailto:mrwilliams@rti.org">mrwilliams@rti.org</ext-link>.</corresp>
</author-notes>
<pub-date pub-type="ppub"><year>2026</year></pub-date><pub-date pub-type="epub"><day>21</day><month>7</month><year>2026</year></pub-date><volume content-type="ahead-of-print">0</volume><issue>0</issue><fpage>1</fpage><lpage>20</lpage><supplementary-material id="S1" content-type="archive" xlink:href="jds1238_s001.zip" mimetype="application" mime-subtype="x-zip-compressed">
<caption>
<title>Supplementary Material</title>
<p>The supplementary material includes a summary of the Bernstein-von Mises result from <xref ref-type="bibr" rid="j_jds1238_ref_024">Williams and Savitsky</xref> (<xref ref-type="bibr" rid="j_jds1238_ref_024">2021</xref>). It also includes additional diagnostic summaries for the simulation study and alternative analyses of the NSDUH application. Also included is R code for replicating the simulation studies and R code and data for replicating the NSDUH example. The <italic>csSampling</italic> package has been updated with the new functionality described in this paper and can be downloaded from github: <ext-link ext-link-type="uri" xlink:href="https://github.com/RyanHornby/csSampling">https://github.com/RyanHornby/csSampling</ext-link></p>
</caption>
</supplementary-material><history><date date-type="received"><day>7</day><month>10</month><year>2025</year></date><date date-type="accepted"><day>19</day><month>6</month><year>2026</year></date></history>
<permissions><copyright-statement>2026 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.</copyright-statement><copyright-year>2026</copyright-year>
<license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>Open access article under the <ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/">CC BY</ext-link> license.</license-p></license></permissions>
<abstract>
<p>Parameter estimation and inference from complex survey samples typically focuses on global model parameters whose estimators have asymptotic properties, such as from fixed effects regression models. The central challenge is to both mitigate bias induced from potentially unbalanced samples and to incorporate adjustments for differences in effective sample size to get correct variance and interval estimates. We present a motivating example of Bayesian inference for a multi-level or mixed effects model in which estimates of both the local parameters (e.g. group level random effects) and the global parameters need to be adjusted for the complex sampling design. We evaluate the limitations of the survey-weighted pseudo-posterior and an existing automated post-processing method to improve the uncertainty quantification. We propose modifications to the automated process and demonstrate their improvements for multi-level models via a simulation study and a motivating example from the National Survey on Drug Use and Health. Reproduction examples are included in the supplementary material and the updated R package is available via github: <ext-link ext-link-type="uri" xlink:href="https://github.com/RyanHornby/csSampling">https://github.com/RyanHornby/csSampling</ext-link></p>
</abstract>
<kwd-group>
<label>Keywords</label>
<kwd>Bayesian inference</kwd>
<kwd>complex survey data</kwd>
<kwd>R</kwd>
<kwd>Stan</kwd>
<kwd>survey weights</kwd>
</kwd-group>
</article-meta>
</front>
<back>
<ref-list id="j_jds1238_reflist_001">
<title>References</title>
<ref id="j_jds1238_ref_001">
<mixed-citation publication-type="journal"> <string-name><surname>Binder</surname> <given-names>DA</given-names></string-name> (<year>1996</year>). <article-title>Linearization methods for single phase and two-phase samples: A cookbook approach</article-title>. <source><italic>Survey Methodology</italic></source>, <volume>22</volume>: <fpage>17</fpage>–<lpage>22</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_002">
<mixed-citation publication-type="journal"> <string-name><surname>Box</surname> <given-names>GE</given-names></string-name>, <string-name><surname>Cox</surname> <given-names>DR</given-names></string-name> (<year>1964</year>). <article-title>An analysis of transformations</article-title>. <source><italic>Journal of the Royal Statistical Society, Series B, Statistical Methodology</italic></source>, <volume>26</volume>(<issue>2</issue>): <fpage>211</fpage>–<lpage>243</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1111/j.2517-6161.1964.tb00553.x" xlink:type="simple">https://doi.org/10.1111/j.2517-6161.1964.tb00553.x</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_003">
<mixed-citation publication-type="journal"> <string-name><surname>Breslow</surname> <given-names>NE</given-names></string-name>, <string-name><surname>Wellner</surname> <given-names>JA</given-names></string-name> (<year>2007</year>). <article-title>Weighted likelihood for semiparametric models and two-phase stratified samples, with application to Cox regression</article-title>. <source><italic>Scandinavian Journal of Statistics</italic></source>, <volume>34</volume>(<issue>1</issue>): <fpage>86</fpage>–<lpage>102</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1111/j.1467-9469.2006.00523.x" xlink:type="simple">https://doi.org/10.1111/j.1467-9469.2006.00523.x</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_004">
<mixed-citation publication-type="journal"> <string-name><surname>Bürkner</surname> <given-names>PC</given-names></string-name> (<year>2017</year>). <article-title>brms: An R package for Bayesian multilevel models using Stan</article-title>. <source><italic>Journal of Statistical Software</italic></source>, <volume>80</volume>(<issue>1</issue>): <fpage>1</fpage>–<lpage>28</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.18637/jss.v080.i01" xlink:type="simple">https://doi.org/10.18637/jss.v080.i01</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_005">
<mixed-citation publication-type="other"> Center for Behavioral Health Statistics and Quality (2020). 2019 national survey on drug use and health: Methodological summary and definitions. Retrieved from SAMHSA website.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_006">
<mixed-citation publication-type="book"> <string-name><surname>Fox</surname> <given-names>J</given-names></string-name>, <string-name><surname>Weisberg</surname> <given-names>S</given-names></string-name> (<year>2019</year>). <source><italic>An R Companion to Applied Regression</italic></source>. <publisher-name>Sage</publisher-name>, <publisher-loc>Thousand Oaks CA</publisher-loc>, <edition>third edition</edition>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_007">
<mixed-citation publication-type="journal"> <string-name><surname>Ghosal</surname> <given-names>S</given-names></string-name>, <string-name><surname>Ghosh</surname> <given-names>JK</given-names></string-name>, <string-name><surname>Vaart</surname> <given-names>AWVD</given-names></string-name> (<year>2000</year>). <article-title>Convergence rates of posterior distributions</article-title>. <source><italic>The Annals of Statistics</italic></source>, <volume>28</volume>(<issue>2</issue>): <fpage>500</fpage>–<lpage>531</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_008">
<mixed-citation publication-type="book"> <string-name><surname>Ghosal</surname> <given-names>S</given-names></string-name>, <string-name><surname>Van der Vaart</surname> <given-names>A</given-names></string-name> (<year>2017</year>). <source><italic>Fundamentals of Nonparametric Bayesian Inference</italic></source>, volume <volume>44</volume>. <publisher-name>Cambridge University Press</publisher-name>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_009">
<mixed-citation publication-type="journal"> <string-name><surname>Goldstein</surname> <given-names>H</given-names></string-name>, <string-name><surname>Browne</surname> <given-names>W</given-names></string-name>, <string-name><surname>Rasbash</surname> <given-names>J</given-names></string-name> (<year>2002</year>). <article-title>Partitioning variation in multilevel models</article-title>. <source><italic>Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences</italic></source>, <volume>1</volume>(<issue>4</issue>): <fpage>223</fpage>–<lpage>231</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1207/S15328031US0104_02" xlink:type="simple">https://doi.org/10.1207/S15328031US0104_02</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_010">
<mixed-citation publication-type="journal"> <string-name><surname>Han</surname> <given-names>Q</given-names></string-name>, <string-name><surname>Wellner</surname> <given-names>JA</given-names></string-name> (<year>2021</year>). <article-title>Complex sampling designs: Uniform limit theorems and applications</article-title>. <source><italic>The Annals of Statistics</italic></source>, <volume>49</volume>(<issue>1</issue>): <fpage>459</fpage>–<lpage>485</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1214/20-AOS1964" xlink:type="simple">https://doi.org/10.1214/20-AOS1964</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_011">
<mixed-citation publication-type="journal"> <string-name><surname>Isaki</surname> <given-names>CT</given-names></string-name>, <string-name><surname>Fuller</surname> <given-names>WA</given-names></string-name> (<year>1982</year>). <article-title>Survey design under the regression superpopulation model</article-title>. <source><italic>Journal of the American Statistical Association</italic></source>, <volume>77</volume>: <fpage>89</fpage>–<lpage>96</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1080/01621459.1982.10477770" xlink:type="simple">https://doi.org/10.1080/01621459.1982.10477770</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_012">
<mixed-citation publication-type="journal"> <string-name><surname>Kish</surname> <given-names>L</given-names></string-name> (<year>1995</year>). <article-title>Methods for design effects</article-title>. <source><italic>Journal of Official Statistics</italic></source>, <volume>11</volume>(<issue>1</issue>): <fpage>55</fpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_013">
<mixed-citation publication-type="journal"> <string-name><surname>Kleijn</surname> <given-names>B</given-names></string-name>, <string-name><surname>van der Vaart</surname> <given-names>A</given-names></string-name> (<year>2012</year>). <article-title>The Bernstein-von-Mises theorem under misspecification</article-title>. <source><italic>Electronic Journal of Statistics</italic></source>, <volume>6</volume>: <fpage>354</fpage>–<lpage>381</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_014">
<mixed-citation publication-type="other"> <string-name><surname>Lee</surname> <given-names>J</given-names></string-name> (<year>2026</year>). Design effect ratios for bayesian survey models: A diagnostic framework for identifying survey-sensitive parameters. arXiv preprint arXiv:<ext-link ext-link-type="uri" xlink:href="https://arxiv.org/abs/2603.07791">2603.07791</ext-link>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_015">
<mixed-citation publication-type="journal"> <string-name><surname>Lele</surname> <given-names>SR</given-names></string-name>, <string-name><surname>Nadeem</surname> <given-names>K</given-names></string-name>, <string-name><surname>Schmuland</surname> <given-names>B</given-names></string-name> (<year>2010</year>). <article-title>Estimability and likelihood inference for generalized linear mixed models using data cloning</article-title>. <source><italic>Journal of the American Statistical Association</italic></source>, <volume>105</volume>(<issue>492</issue>): <fpage>1617</fpage>–<lpage>1625</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1198/jasa.2010.tm09757" xlink:type="simple">https://doi.org/10.1198/jasa.2010.tm09757</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_016">
<mixed-citation publication-type="journal"> <string-name><surname>León-Novelo</surname> <given-names>LG</given-names></string-name>, <string-name><surname>Savitsky</surname> <given-names>TD</given-names></string-name> (<year>2019</year>). <article-title>Fully Bayesian estimation under informative sampling</article-title>. <source><italic>Electronic Journal of Statistics</italic></source>, <volume>13</volume>(<issue>1</issue>): <fpage>1608</fpage>–<lpage>1645</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_017">
<mixed-citation publication-type="other"> <string-name><surname>Lumley</surname> <given-names>T</given-names></string-name> (<year>2016</year>). survey: analysis of complex survey samples. R package version 3.32.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_018">
<mixed-citation publication-type="journal"> <string-name><surname>McGuire</surname> <given-names>FH</given-names></string-name>, <string-name><surname>Beccia</surname> <given-names>AL</given-names></string-name>, <string-name><surname>Peoples</surname> <given-names>JE</given-names></string-name>, <string-name><surname>Williams</surname> <given-names>MR</given-names></string-name>, <string-name><surname>Schuler</surname> <given-names>MS</given-names></string-name>, <string-name><surname>Duncan</surname> <given-names>AE</given-names></string-name> (<year>2024</year>). <article-title>Depression at the intersection of race/ethnicity, sex/gender, and sexual orientation in a nationally representative sample of us adults: A design-weighted intersectional maihda</article-title>. <source><italic>American Journal of Epidemiology</italic></source>, <volume>193</volume>(<issue>12</issue>): <fpage>1662</fpage>–<lpage>1674</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1093/aje/kwae121" xlink:type="simple">https://doi.org/10.1093/aje/kwae121</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_019">
<mixed-citation publication-type="journal"> <string-name><surname>Rao</surname> <given-names>JNK</given-names></string-name>, <string-name><surname>Wu</surname> <given-names>CFJ</given-names></string-name>, <string-name><surname>Yue</surname> <given-names>K</given-names></string-name> (<year>1992</year>). <article-title>Some recent work on resampling methods for complex surveys</article-title>. <source><italic>Survey Methodology</italic></source>, <volume>18</volume>: <fpage>209</fpage>–<lpage>217</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_020">
<mixed-citation publication-type="journal"> <string-name><surname>Ribatet</surname> <given-names>M</given-names></string-name>, <string-name><surname>Cooley</surname> <given-names>D</given-names></string-name>, <string-name><surname>Davison</surname> <given-names>AC</given-names></string-name> (<year>2012</year>). <article-title>Bayesian inference from composite likelihoods, with an application to spatial extremes</article-title>. <source><italic>Statistica Sinica</italic></source>, <volume>22</volume>(<issue>2</issue>): <fpage>813</fpage>–<lpage>845</lpage>.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_021">
<mixed-citation publication-type="journal"> <string-name><surname>Robbins</surname> <given-names>MW</given-names></string-name>, <string-name><surname>Ghosh</surname> <given-names>SK</given-names></string-name>, <string-name><surname>Habiger</surname> <given-names>JD</given-names></string-name> (<year>2013</year>). <article-title>Imputation in high-dimensional economic data as applied to the agricultural resource management survey</article-title>. <source><italic>Journal of the American Statistical Association</italic></source>, <volume>108</volume>(<issue>501</issue>): <fpage>81</fpage>–<lpage>95</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1080/01621459.2012.734158" xlink:type="simple">https://doi.org/10.1080/01621459.2012.734158</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_022">
<mixed-citation publication-type="journal"> <string-name><surname>Savitsky</surname> <given-names>TD</given-names></string-name>, <string-name><surname>Toth</surname> <given-names>D</given-names></string-name> (<year>2016</year>). <article-title>Bayesian estimation under informative sampling</article-title>. <source><italic>Electronic Journal of Statistics</italic></source>, <volume>10</volume>(<issue>1</issue>): <fpage>1677</fpage>–<lpage>1708</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1214/16-EJS1153" xlink:type="simple">https://doi.org/10.1214/16-EJS1153</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_023">
<mixed-citation publication-type="journal"> <string-name><surname>Williams</surname> <given-names>MR</given-names></string-name>, <string-name><surname>Savitsky</surname> <given-names>TD</given-names></string-name> (<year>2020</year>). <article-title>Bayesian estimation under informative sampling with unattenuated dependence</article-title>. <source><italic>Bayesian Analysis</italic></source>, <volume>15</volume>(<issue>1</issue>): <fpage>57</fpage>–<lpage>77</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1214/18-BA1143" xlink:type="simple">https://doi.org/10.1214/18-BA1143</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_024">
<mixed-citation publication-type="journal"> <string-name><surname>Williams</surname> <given-names>MR</given-names></string-name>, <string-name><surname>Savitsky</surname> <given-names>TD</given-names></string-name> (<year>2021</year>). <article-title>Uncertainty estimation for pseudo-Bayesian inference under complex sampling</article-title>. <source><italic>International Statistical Review</italic></source>, <volume>89</volume>(<issue>1</issue>): <fpage>72</fpage>–<lpage>107</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1111/insr.12376" xlink:type="simple">https://doi.org/10.1111/insr.12376</ext-link></mixed-citation>
</ref>
<ref id="j_jds1238_ref_025">
<mixed-citation publication-type="other"> <string-name><surname>Yee</surname> <given-names>TW</given-names></string-name> (<year>2025</year>). VGAM: Vector Generalized Linear and Additive Models. R package version 1.1-13.</mixed-citation>
</ref>
<ref id="j_jds1238_ref_026">
<mixed-citation publication-type="journal"> <string-name><surname>Yeo</surname> <given-names>I</given-names></string-name>, <string-name><surname>Johnson</surname> <given-names>RA</given-names></string-name> (<year>2000</year>). <article-title>A new family of power transformations to improve normality or symmetry</article-title>. <source><italic>Biometrika</italic></source>, <volume>87</volume>(<issue>4</issue>): <fpage>954</fpage>–<lpage>959</lpage>. <ext-link ext-link-type="doi" xlink:href="https://doi.org/10.1093/biomet/87.4.954" xlink:type="simple">https://doi.org/10.1093/biomet/87.4.954</ext-link></mixed-citation>
</ref>
</ref-list>
</back>
</article>
