JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1018 10.6339/21-JDS1018 Statistical Data Science BDNNSurv: Bayesian Deep Neural Networks for Survival Analysis Using Pseudo Values FengDaidai.feng@abbvie.com1∗ ZhaoLili2 Data and Statistical Sciences, AbbVie Inc., Illinois, USA Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan, USA Corresponding author. Email: dai.feng@abbvie.com. 20211382021194542554 Supplementary Material

The code used for simulation, including R code to simulate data and summarize results, python code to generate initial values for BDNNSurv, python code to run BDNNSurv, and R code to run BART and ANOVA DDP.

2320211572021 2021 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2021 Open access article under the CC BY license.

There has been increasing interest in modeling survival data using deep learning methods in medical research. In this paper, we proposed a Bayesian hierarchical deep neural networks model for modeling and prediction of survival data. Compared with previously studied methods, the new proposal can provide not only point estimate of survival probability but also quantification of the corresponding uncertainty, which can be of crucial importance in predictive modeling and subsequent decision making. The favorable statistical properties of point and uncertainty estimates were demonstrated by simulation studies and real data analysis. The Python code implementing the proposed approach was provided.

 automatic differentiation variational inference Bayesian deep learning neural networks pseudo probability survival outcome References Andersen PK, Klein JP (2007). Regression analysis for multistate models based on a pseudo-value approach, with applications to bone marrow transplantation studies. Scandinavian Journal of Statistics, 34(1): 3–16. Andersen PK, Klein JP, Rosthøj S (2003). Generalised linear models for correlated pseudo-observations, with applications to multistate models. Biometrika, 90: 15–27. Andersen PK, Perme MP (2010). Pseudo-observations in survival analysis. Statistical Methods in Medical Research, 19: 71–99. Bender R, Augustin T, Blettner M (2005). Generating survival times to simulate Cox proportional hazards models. Statistics in Medicine, 24(11): 1713–1723. Binder N, Gerds TA, Andersen PK (2014). Pseudo-observations for competing risks with covariate dependent censoring. Lifetime Data Analysis, 20: 303–315. Blei DM, Kucukelbir A, McAuliffe JD (2017). Variational inference: a review for statisticians. Journal of the American Statistical Association, 112(518): 859–877. Ching T, Zhu X, Garmire LX (2018). Cox-nnet: an artificial neural network method for prognosis prediction of high-throughput omics data. PLoS Computational Biology, 14: e100607. Chipman HA, George EI, McCulloch RE (2010). BART: Bayesian additive regression trees. Annals of Applied Statistics, 4(1): 266–298. De Iorio M, Johnson WO, Müller P, Rosner GL (2009). Bayesian nonparametric nonproportional hazards survival modeling. Biometrics, 65(3): 762–771. Fotso S (2018). Deep neural networks for survival analysis based on a multi-task framework. arXiv preprint: https://arxiv.org/abs/1801.05512 Gal Y, Ghahramani Z (2016). Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, 1050–1059. PMLR. Gensheimer MF, Narasimhan B (2019). A scalable discrete-time survival model for neural networks. PeerJ, 7: e6257. Ghahramani Z (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553): 452–459. Giunchiglia E, Nemchenko A, van der Schaar M (2018). RNN-SURV: a deep recurrent model for survival analysis. In: International Conference on Artificial Neural Networks, volume 90: 15–27. Springer. Hoffman MD, Blei DM, Wang C, Paisley J (2013). Stochastic variational inference. Journal of Machine Learning Research, 14(1): 1303–1347. Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS (2008). Random survival forests. Annals of Applied Statistics, 2(3): 841–860. Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y (2018). Deepsurv: Personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1): 1–12. Kendall A, Gal Y (2017). arXiv preprint: https://arxiv.org/abs/1703.04977. Klein JP, Andersen PK (2005). Regression modeling of competing risks data based on pseudovalues of the cumulative incidence function. Biometrics, 61(1): 223–229. Kucukelbir A, Tran D, Ranganath R, Gelman A, Blei DM (2017). Automatic differentiation variational inference. Journal of Machine Learning Research, 18(1): 430–474. Lee C, Zame W, Yoon J, van der Schaar M (2018). Deephit: a deep learning approach to survival analysis with competing risks. In: Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. Leibig C, Allken V, Ayhan MS, Berens P, Wahl S (2017). Leveraging uncertainty information from deep neural networks for disease detection. Scientific Reports, 7(1): 17816. López-García G, Jerez JM, Franco L, Veredas FJ (2020). Transfer learning with convolutional neural networks for cancer survival prediction using gene-expression data. PLoS ONE, 15(3): e0230536. Luck M, Sylvain T, Cardinal H, Lodi A, Bengio Y (2017). Deep learning for patient-specific kidney graft survival analysis. arXiv preprint: https://arxiv.org/abs/1705.10245 Martinsson E (2016). WTTE-RNN: Weibull Time to Event Recurrent Neural Network, Master’s thesis, University of Gothenburg, Sweden. Nielsen MA (2015). Neural Networks and Deep Learning, volume 25. Determination Press, San Francisco, CA. Salvatier J, Wiecki TV, Fonnesbeck C (2016). Probabilistic programming in python using pymc3. PeerJ Computer Science, 2: e55. Silvestro D, Andermann T (2020). Prior choice affects ability of Bayesian neural networks to identify unknowns. arXiv preprint: https://arxiv.org/abs/2005.04987 Sparapani R, Spanbauer C, McCulloch R (2021). Nonparametric machine learning and efficient computation with Bayesian additive regression trees: the BART R package. Journal of Statistical Software, 97(1): 1–66. Sparapani RA, Logan BR, McCulloch RE, Laud PW (2016). Nonparametric survival analysis using Bayesian additive regression trees (BART). Statistics in Medicine, 35(16): 2741–2753. Tayob N, Murray S (2017). Statistical consequences of a successful lung allocation system – recovering information and reducing bias in models for urgency. Statistics in Medicine, 6: 2435–2451. Tell GS, Fried LP, Hermanson B, Manolio TA, Newman AB, Borhani NO (1993). Recruitment of adults 65 years and older as participants in the cardiovascular health study. Annals of Epidemiology, 3(4): 358–366. Theano Development Team (2016). Theano: A Python framework for fast computation of mathematical expressions. arXiv preprint: https://arxiv.org/abs/1605.02688 Wager S, Hastie T, Efron B (2014). Confidence intervals for random forests: the jackknife and the infinitesimal jackknife. Journal of Machine Learning Research, 15(1): 1625–1651. Wainwright MJ, Jordan MI (2008). Graphical Models, Exponential Families, and Variational Inference. Now Publishers Inc. Xiang F, Murray S (2012). Restricted mean models for transplant benefit and urgency. Statistics in Medicine, 6: 561–76. Zhao L (2020). Deep neural networks for predicting restricted mean survival times. Bioinformatics, 36(24): 5672–5677. Zhao L, Feng D (2020). Deep neural networks for survival analysis using pseudo values. IEEE Journal of Biomedical and Health Informatics, 24(11): 3308–3314. Zhao L, Murray S, Mariani LH, Ju W (2020). Incorporating longitudinal biomarkers for dynamic risk prediction in the era of big data: a pseudo-observation approach. Statistics in Medicine, 39: 3685–3699. Zhou H, Hanson T, Zhang J (2020). spBayesSurv: fitting Bayesian spatial survival models using R. Journal of Statistical Software, 92(9): 1–33. Zhu X, Yao J, Huang J (2016). Deep convolutional neural network for survival analysis with pathological images. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 544–547. IEEE.