JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1089 10.6339/23-JDS1089 Statistical Data Science Covid-19 Vaccine Efficacy: Accuracy Assessment, Comparison, and Caveats FuWenjiangfuw@math.uh.edu12∗ LiJieni3 ScheetPaul A.4 Department of Mathematics, University of Houston, Houston, Texas 77204, USA Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, Texas 77230, USA Department of Pharmaceutical Health Outcomes and Policy, University of Houston, Houston, Texas 77204, USA Department of Epidemiology, The University of Texas MD Anderson Cancer Center, Houston, Texas 77230, USA Corresponding author. Email: fuw@math.uh.edu. 202473202322145555112022522023 2024 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2024 Open access article under the CC BY license.

Vaccine efficacy is a key index to evaluate vaccines in initial clinical trials during the development of vaccines. In particular, it plays a crucial role in authorizing Covid-19 vaccines. It has been reported that Covid-19 vaccine efficacy varies with a number of factors, including demographics of population, time after vaccine administration, and virus strains. By examining clinical trial data of three Covid-19 vaccine studies, we find that current approach to evaluating vaccines with an overall efficacy does not provide desired accuracy. It requires no time frame during which a candidate vaccine is evaluated, and is subject to misuse, resulting in potential misleading information and interpretation. In particular, we illustrate with clinical trial data that the variability of vaccine efficacy is underestimated. We demonstrate that a new method may help to address these caveats. It leads to accurate estimation of the variation of efficacy, provides useful information to define a reasonable time frame to evaluate vaccines, and avoids misuse of vaccine efficacy and misleading information.

 confidence interval efficacy curve loess smoothing time window weekly efficacy References Abu-Raddad LJ, Chemaitelly H, Butt AA, Vaccinati NSGC (2021). Effectiveness of the BNT162b2 COVID-19 vaccine against the B.1.1.7 and B.1.351 variants. New England Journal of Medicine, 385(2): 187–189. https://doi.org/10.1056/NEJMc2104974 Bernal JL, Andrews N, Gower C, Gallagher E, Simmons R, Thelwall S, et al. (2021). Effectiveness of COVID-19 Vaccines against the B.1.617.2. (Delta) Variant. New England Journal of Medicine, 385(7): 585–594. https://doi.org/10.1056/NEJMoa2108891 CDC (2000). Principles of Epidemiology in Public Health Practice - An Introduction to Epidemiology and Biostatistics, 3rd ed. Center for Disease Control and Prevention, Office of Workforce and Career Development, https://www.cdc.gov/csels/dsepd/ss1978/index.html. Cohen J (2021). One–dose COVID-19 vaccine offers solid protection against severe disease — Johnson & Johnson candidate provides new weapon in vaccine arsenal that could significantly expand world’s supply. Science News Scienceinsider. https://doi.org/10.1126/science.abg7115 FDA (2020a). Development and Licensure of Vaccines to Prevent COVID-19 Guidance for Industry. Food and Drug Administration, https://www.fda.gov/media/139638/download. FDA (2020b). Emergency Use Authorization for Vaccines to Prevent COVID-19. Food and Drug Administration, https://www.fda.gov/media/142749/download. FDA (2020c). FDA Briefing Document: Pfizer and BioNTech COVID-19 Vaccine (BNT162, PF-07302048). Food and Drug Administration, https://www.fda.gov/media/144246/download. FDA (2020d). FDA Briefing Document: Moderna COVID-19 Vaccine. Food and Drug Administration, https://www.fda.gov/media/144434/download. FDA (2021a). FDA Briefing Document: Janssen Ad26.COV2.S Vaccine for the Prevention of COVID-19. Food and Drug Administration, https://www.fda.gov/media/146217/download. FDA (2021b). FDA and CDC Lift Recommended Pause on Johnson & Johnson (Janssen) COVID-19 Vaccine Use Following Thorough Safety Review. Food and Drug Administration, https://www.fda.gov/news-events/press-announcements/fda-and-cdc-lift-recommended-pause-johnson-johnson-janssen-covid-19-vaccine-use-following-thorough. FDA (2022a). Coronavirus (COVID-19) Update: FDA Limites Use of Janssen COVID-19 to Certain Individuals. Food and Drug Administration, https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-limits-use-janssen-covid-19-vaccine-certain-individuals. FDA (2022b). FDA Briefing Document: Novavax COVID-19 Vaccine. Food and Drug Administration, https://www.fda.gov/media/158912/download. Feikin DR, Higdon MM, Abu-Raddad LJ, Andrews N, Araos R, Goldberg Y, et al. (2022). Duration of effectiveness of vaccines against SARS-Cov-2 infection and COVID-19 disease: Results of a systematic review and meta-regression. The Lancet, 399(10328): 924–944. https://doi.org/10.1016/S0140-6736(22)00152-0 Kim D, Keskinocak P, Pelin P, Yildirim I (2022). The balancing role of distribution speed against varying efficacy levels of COVID-19 vaccines under variants. Scientific Reports, 12: 7493. https://doi.org/10.1038/s41598-022-11060-8 Lin DY, Gu Y, Wheeler B, Young H, Holloway S, Sunny SK, et al. (2022). Effectiveness of COVID-19 vaccines over a 9-month period in North Carolina. New England Journal of Medicine, 386: 933–941. https://doi.org/10.1056/NEJMoa2117128 Liu C, Ginn HM, Dejnirattisai W, Supasa P, Wang B, Tuekprakhon A, et al. (2021). Reduced neutralization of SARS-CoV-2 B.1.617 by vaccine and convalescent serum. Cell, 184(16): 4220–4236. https://doi.org/10.1016/j.cell.2021.06.020 Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, et al. (2021). Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19. New England Journal of Medicine, 384: 2187–2201. https://doi.org/10.1056/NEJMoa2101544 Wang P, Nair MS, Liu L, Iketani S, Luo Y, Guo Y, et al. (2021). Antibody resistance of SARS-CoV-2 variants B.1.351 and B.1.1.7. Nature, 593(7857): 130–135. https://doi.org/10.1038/s41586-021-03398-2 Wright BJ, Tideman S, Diaz V, French T, Parsons GT, Robicsek A (2022). Comparative vaccine effectiveness against severe COVID-19 over time in US hospital administrative data: A case-control study. Lancet Respiratory Medicine, 10(6): 557–565. https://doi.org/10.1016/S2213-2600(22)00042-X