JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1084 10.6339/23-JDS1084 Statistical Data Science Random Forest of Interaction Trees for Estimating Individualized Treatment Regimes with Ordered Treatment Levels in Observational Studies ThorpJustin1 https://orcid.org/0000-0002-7553-4264 LevineRichard A.rlevine@sdsu.edu12∗ LiLuo1 FanJuanjuanjjfan@sdsu.edu12∗ Department of Mathematics and Statistics, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182, USA Analytic Studies & Institutional Research, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182, USA Corresponding author. Email: rlevine@sdsu.edu or jjfan@sdsu.edu. 20232220232123914113172022612023 2023 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2023 Open access article under the CC BY license.

Traditional methods for evaluating a potential treatment have focused on the average treatment effect. However, there exist situations where individuals can experience significantly heterogeneous responses to a treatment. In these situations, one needs to account for the differences among individuals when estimating the treatment effect. Li et al. (2022) proposed a method based on random forest of interaction trees (RFIT) for a binary or categorical treatment variable, while incorporating the propensity score in the construction of random forest. Motivated by the need to evaluate the effect of tutoring sessions at a Math and Stat Learning Center (MSLC), we extend their approach to an ordinal treatment variable. Our approach improves upon RFIT for multiple treatments by incorporating the ordered structure of the treatment variable into the tree growing process. To illustrate the effectiveness of our proposed method, we conduct simulation studies where the results show that our proposed method has a lower mean squared error and higher optimal treatment classification, and is able to identify the most important variables that impact the treatment effect. We then apply the proposed method to estimate how the number of visits to the MSLC impacts an individual student’s probability of passing an introductory statistics course. Our results show that every student is recommended to go to the MSLC at least once and some can drastically improve their chance of passing the course by going the optimal number of times suggested by our analysis.

 educational data mining generalized propensity scores individualized treatment effect machine learning student success study National Science Foundation1633130This research was supported in part by the National Science Foundation grant 1633130. References Alemayehu D, Chen Y, Markatou M (2017). A comparative study of subgroup identification methods for differential treatment effect: Performance metrics and recommendations. Statistical Methods in Medical Research, 27(12): 3658–3678. Breiman L (2001). Random forests. Machine Learning, 45: 5–32. Breiman L, Friedman J, Stone C, Olshen R (1984). Classification and Regression Trees. Chapman and Hall/CRC. Chipman H (2010). BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4. Dusseldorp E, Mechelen I (2013). Qualitative interaction trees: A tool to identify qualitative treatment-subgroup interactions. Statistics in Medicine, 33: 219–237. Imbens G (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87: 706–710. Ishwaran H, Kogalur U, Gorodeski E, Minn A, Lauer M (2012). High-dimensional variable selection for survival data. Journal of the American Statistical Association, 105: 205–217. Li L, Levine R, Fan J (2022). Causal effect random forest of interaction trees for learning individualized treatment regimes with multiple treatments in observational studies. Stat, 11(1), e457. Lipkovich I, Dmitrienko A, Denne J, Enas G (2011). Subgroup identification based on differential effect search-a recursive partitioning method for establishing response to treatment in patient subpopulations. Statistics in Medicine, 30(21): 2601–2621. Rosenbaum P, Rubin D (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70: 41–55. Rubin DB (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5): 688–701. Sparapani R, Spanbauer C, McCulloch R (2021). Nonparametric machine learning and efficient computation with Bayesian additive regression trees: The BART R package. Journal of Statistical Software, 97(1): 1–66. Su X, Peña A, Liu L, Levine R (2018). Random forests of interaction trees for estimating individualized treatment effects in randomized trials. Statistics in Medicine, 37(17): 2547–2560. Su X, Tsai CL, Wang H, Nickerson D, Li B (2009). Subgroup analysis via recursive partitioning. Journal of Machine Learning Research, 10: 141–158. R Core Team (2021). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. Thorp J, Li L, Fan J (2022). CERFIT: Causal Effect Random Forest of Interaction Trees. https://cran.r-project.org/web/packages/CERFIT/index.html. Wager S, Athey S (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523): 1228–1242. Zhu Y, Coffman DL, Ghosh D (2015). A boosting algorithm for estimating generalized propensity scores with continuous treatments. Journal of Causal Inference, 3(1): 25–40.