Journal of Data Science

The mean-variance portfolio model, based on the risk-return trade-off for optimal asset allocation, remains fundamental in portfolio optimization. However, its reliance on restrictive assumptions about asset return distributions limits its applicability to real-world data. Parametric copula structures provide a novel way to overcome these limitations by accounting for asymmetry, heavy tails, and time-varying dependencies. Existing methods have been shown to rely on fixed or static dependence structures, thus overlooking the dynamic nature of the financial market. In this study, a semiparametric model is proposed that combines nonparametrically estimated copulas with parametrically estimated marginals to allow all parameters to dynamically evolve over time. A novel framework was developed that integrates time-varying dependence modeling with flexible empirical beta-copula structures. Marginal distributions were modeled using the skewed generalized t-family. This effectively captures asymmetry and heavy tails and makes the model suitable for predictive inferences in real-world scenarios. Furthermore, the model was applied to rolling windows of financial returns from the USA, India, and Hong Kong economies to understand the influence of dynamic market conditions. The approach addresses the limitations of models that rely on parametric assumptions. By accounting for asymmetry, heavy tails, and cross-correlated asset prices, the proposed method offers a robust solution to optimize diverse portfolios in an interconnected financial market. Through adaptive modeling, it allows for better management of risk and return across varying economic conditions, leading to more efficient asset allocation and improved portfolio performance.

We propose a fine-grained attention-based multiple instance classification (FAMIC) model for interpretable word-level sentiment analysis (SA) using only document-level sentiment labels. By operating at the word level, FAMIC enhances interpretability while maintaining competitive performance in document-level classification. The model generates interpretable outputs such as contextual weighting, word neutrality, and negation cues, offering insights into how context shapes sentiment and how the model arrives at its predictions. FAMIC is built on a straightforward yet effective architecture that combines a multiple instance classification framework with self-attention and positionally encoded self-attention blocks. This design enables the model to capture both local and global contextual dependencies, supporting nuanced sentiment interpretation. We evaluate FAMIC on two sentiment classification datasets and provide an extensive analysis of its interpretability and performance.

Inter-rater agreement is fundamental to decision making in medicine, psychology, and the social sciences, as it reflects the quality and reliability of rating systems. ICC (intraclass correlation) has been widely used as a measure of inter-rater agreement. To date, there has been no methodological development that properly assesses improvement in ICC for pre–post studies with ordinal ratings. It remain uninvestigated whether/how correlations between pre- and post-intervention scores impact the estimation and comparison of ICC. We present a Bayesian hierarchical probit framework for evaluating changes in ICCs in such settings. The model incorporates rater- and item-level correlations and compares two parameterizations: an “individual components” prior that separately models variances and correlations, and an inverse Wishart prior. Simulation studies show that accounting for pre–post correlation substantially improves estimation accuracy and power to detect changes in agreement, while ignoring it reduces efficiency. Application to a multicenter study on conjunctival inflammation demonstrates that a novel grading scale markedly increased inter-rater agreement. This framework underscores the importance of modeling ordinal outcomes appropriately and provides a flexible Bayesian tool for evaluating the effectiveness of interventions on inter-rater agreement in pre-post studies.

This study investigates how user ability to manipulate plot features affects graphical perception, by extending a previous graphical study (Vanderplas and Hofmann, 2017) with an interactive framework. Similar to the original study, statistical lineups included two target patterns (a linear trend and a clustering pattern), as well as eighteen null plots generated from three different mixture proportions of the combined cluster and trend models. Participants were asked to select two plots that they perceived as ‘most different’, and were able to interact with the graphics by toggling aesthetic features such as cluster coloring, cluster ellipses, linear trendlines, and regression error bands.

The U.S. Rehabilitation Services Administration (RSA) has partnered with state vocational rehabilitation (VR) agencies since 1973 to improve employment outcomes for individuals with disabilities. A critical resource in this effort is the RSA-911 dataset, a quarterly collection of standardized participant data. However, its complex structure, including high rates of missing or ambiguous values, poses significant challenges for effective analysis. We address these challenges by developing an R package designed to streamline the cleaning and analysis of RSA-911 data, as well as the newly introduced Transition Readiness Toolkit (TRT) scores data (R Core Team, 2021). The TRT assesses participants’ improvement across services and offers a critical measure of VR program effectiveness. Using this R package, our work offers the first analysis of the relationship between TRT pre-post scores and RSA-911 demographic data, providing insights into program outcomes. Additionally, we deliver a user-friendly online dashboard, built with the shiny framework, to allow VR counselors and researchers to independently analyze RSA-911 and TRT data (Chang et al., 2024). This dashboard features intuitive visualizations and workflows, making it easier to generate reproducible analyses without requiring extensive technical expertise. By automating data preparation and providing accessible analysis tools, this project contributes to the field of vocational rehabilitation by facilitating more efficient research and empowering VR professionals with data-driven insights. The tools presented offer a framework for future studies, enhancing the consistency, flexibility, and reproducibility of VR data analysis.

Satellite precipitation products have the potential to be employed for the purpose of better understanding extreme precipitation events in remote mountainous terrain, where weather stations and radar data tend to be sparse. For this reason, it is crucial to assess how closely satellite estimates agree with ground observations during extreme events, and how that agreement varies across such regions. We use asymptotic dependence from multivariate extreme value theory as the primary tool in this study. After presenting two measures of asymptotic dependence and their associated estimators, we illustrate these ideas using simulated data. We then model the level of asymptotic dependence between PERSIANN-CDR and SNOTEL station data over the US Northern Rocky Mountains. We consider both asymptotic dependence estimators, and based on hypothesis tests and visual diagnostics, both estimates of asymptotic dependence indicate positive spatial dependence. We also investigate whether geographical factors influence the levels of asymptotic dependence over this region. Using a spatial correlation analysis, we find that elevation is negatively correlated with both asymptotic dependence estimators and average summer temperature is positively correlated with both asymptotic dependence estimators. However, we did not find any geographical covariates to be statistically significant in the model.

Advances in AI and automation are reshaping qualitative research workflows, making processes more efficient, accurate, consistent, and scalable. This paper presents innovations developed for the Illinois Needs Assessment project, a statewide initiative led by the Illinois State Board of Education and the American Institutes for Research to conduct comprehensive needs assessments for schools that need intensive or comprehensive support. To address the scale and tight timeline requirements of the project, the team designed three interconnected pipelines that work together to produce a finalized report. The first, an Audio Pipeline, uses Whisper and generative AI to automate transcription, text-based speaker role attribution, thematic coding, and insight generation from focus groups and interviews. The second, a Report Generation Pipeline, integrates Airtable automations with AWS infrastructure to produce customized school reports that merge AI-generated findings with survey data, school performance metrics, and contextual comparisons. Third, the Needs Assessment Summary Report automates the assembly of all quantitative and qualitative inputs into a polished, customizable deliverable that combines efficiency with expert review. Together, these pipelines replace ad hoc manual workflows with reproducible, consistent systems that enhance data quality, reduce error, and broaden access for non-technical users. The integrated design demonstrates how automation and generative AI can reduce manual burdens, shorten delivery timelines, and support timely, data-informed, and human-centered decision-making in education.

Land use land cover (LULC) change in the agriculture, is a critical area of concern as it directly impacts food security, environmental health, and economic stability. One of the leading LULC data products is the U.S. Department of Agriculture’s (USDA) Cropland Data Layer (CDL). Produced annually by the USDA National Agricultural Statistics Service (NASS) using satellite imagery, the CDL provides crop-specific data with an estimated classification accuracy of 85% to 95% for major crop types across the U.S. However, several limitations inherent to the CDL, such as crop underestimation bias, pixel misclassification, and difficulty distinguishing certain vegetation types, have raised questions about the accuracy of LULC change estimates derived from this dataset. In this paper, we introduce the R package cdlsim, designed to quantify the sensitivity of CDL-derived metrics through simulations of CDL data at the patch level using NASS published accuracy statistics. We present a case study utilizing landscape metrics calculated with the popular landscapemetrics R package to demonstrate the utility of cdlsim in quantifying the sensitivity of metrics to random perturbations in the data. The case study examines a mixed agricultural and grassland landscape in South Dakota, illustrating how our package enables researchers to achieve a more nuanced representation of land-use change.

Hierarchical linear mixed models are commonly used in many scientific fields. However, without a strong statistical background, it can be hard to understand the relationships between the random effect variables and the inferences that can be made when a model has nested random effects. Visualizing relationships makes it easier for the practitioner to understand what relationships the model is capable of estimating and testing. We present an R package modeldiagramR that seamlessly creates a visualization of the model based on the data and the model object created when fitting a linear mixed model using either lme4 or nlme.