JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1225 10.6339/26-JDS1225 Statistical Data Science Interpretable Word-Level Context-Based Sentiment Analysis https://orcid.org/0000-0002-0346-7541 YangChenyu1 LarsonEric2 CaoJingjcao@smu.edu1∗ Department of Statistics and Data Science, Southern Methodist University, Dallas, Texas, U.S.A Department of Computer Science, Southern Methodist University, Dallas, Texas, U.S.A Corresponding author. Email: jcao@smu.edu. 202675202624231933731720252622026 2026 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2026 Open access article under the CC BY license.

We propose a fine-grained attention-based multiple instance classification (FAMIC) model for interpretable word-level sentiment analysis (SA) using only document-level sentiment labels. By operating at the word level, FAMIC enhances interpretability while maintaining competitive performance in document-level classification. The model generates interpretable outputs such as contextual weighting, word neutrality, and negation cues, offering insights into how context shapes sentiment and how the model arrives at its predictions. FAMIC is built on a straightforward yet effective architecture that combines a multiple instance classification framework with self-attention and positionally encoded self-attention blocks. This design enables the model to capture both local and global contextual dependencies, supporting nuanced sentiment interpretation. We evaluate FAMIC on two sentiment classification datasets and provide an extensive analysis of its interpretability and performance.

 interpretable sentiment analysis multiple instance classification relative positional embedding self-attention References Abnar S, Zuidema W (2020). Quantifying attention flow in transformers. arXiv preprint: https://arxiv.org/abs/2005.00928 Balderas L, Lastra M, Benítez JM (2023). Can persistent homology whiten transformer-based black-box models? A case study on bert compression. arXiv preprint: https://arxiv.org/abs/2312.10702 Bilan I, Roth B (2018). Position-aware self-attention with relative positional encodings for slot filling. arXiv preprint: https://arxiv.org/abs/1807.03052 Carbonneau MA, Cheplygina V, Granger E, Gagnon G (2018). Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition, 77: 329–353. https://doi.org/10.1016/j.patcog.2017.10.009 Chen K, Wang R, Utiyama M, Sumita E (2021). Context-aware positional representation for self-attention networks. Neurocomputing, 451: 46–56. https://doi.org/10.1016/j.neucom.2021.04.055 Das SR, Chen MY (2007). Yahoo! For Amazon: Sentiment extraction from small talk on the web. Management Science, 53(9): 1375–1388. https://doi.org/10.1287/mnsc.1070.0704 Devlin J, Chang MW, Lee K, Toutanova K (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186. Dietterich TG, Lathrop RH, Lozano-Pérez T (1997). Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1): 31–71. https://doi.org/10.1016/S0004-3702(96)00034-3 Fang X, Zhan J (2015). Sentiment analysis using product review data. Journal of Big Data, 2, Article number: 5. https://doi.org/10.1186/s40537-015-0015-2 Ghader H, Monz C (2017). What does attention in neural machine translation pay attention to? arXiv preprint: https://arxiv.org/abs/1710.03348 Go A, Bhayani R, Huang L (2009). Twitter sentiment classification using distant supervision. CS224N project report. Stanford, 1, Article number: 12. Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 Jain S, Wallace BC (2019). Attention is not explanation. arXiv preprint: https://arxiv.org/abs/2005.00928 Katumullage D, Yang C, Barth J, Cao J (2022). Using neural network models for wine review classification. Journal of Wine Economics, 17(1): 27–41. https://doi.org/10.1017/jwe.2022.2 Kim Y (2014). Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1746–1751. Kindermans PJ, Hooker S, Adebayo J, Alber M, Schütt KT, ..., Kim B (2017). The (un)reliability of saliency methods. arXiv preprint: https://arxiv.org/abs/1711.00867 Liu B (2012). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1): 1–167. https://doi.org/10.1007/978-3-031-02145-9 Lundberg SM, Lee S (2017). A unified approach to interpreting model predictions. CoRR. arXiv preprint: https://arxiv.org/abs/1705.07874 Medhat W, Hassan A, Korashy H (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5(4): 1093–1113. https://doi.org/10.1016/j.asej.2014.04.011 Mikolov T, Chen K, Corrado GS, Dean J (2013). Efficient estimation of word representations in vector space. arXiv preprint: https://arxiv.org/abs/1301.3781 Minaee S, Azimi E, Abdolrashidi A (2019). Deep-sentiment: Sentiment analysis using ensemble of cnn and bi-lstm models. arXiv preprint: https://arxiv.org/abs/1904.04206 Nivre J (2005). Dependency grammar and dependency parsing. MSI Report, 5133(1959): 1–32. Pang B, Lee L, Vaithyanathan S (2002). Thumbs up? Sentiment classification using machine learning techniques. Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), 79–86. Petch J, Di S, Nelson W (2022). Opening the black box: The promise and limitations of explainable machine learning in cardiology. Canadian Journal of Cardiology, 38(2): 204–213. https://doi.org/10.1016/j.cjca.2021.09.004 Ray S, Page D (2001). Multiple instance regression. In: ICML (CE Brodley, AP Danyluk, eds.), 425–432. Morgan Kaufmann. Read J (2005). Using emoticons to reduce dependency in machine learning techniques for sentiment classification. Proceedings of the ACL Student Research Workshop, 43–48. Ribeiro MT, Singh S, Guestrin C (2016). “Why should I trust you?”: Explaining the predictions of any classifier. arXiv preprint: https://arxiv.org/abs/1602.04938 Rudin C (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. arXiv preprint: https://arxiv.org/abs/1811.10154 Seonwoo Y, Kim JH, Ha JW, Oh A (2020). Context-aware answer extraction in question answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2418–2428. Shaw P, Uszkoreit J, Vaswani A (2018). Self-attention with relative position representations. arXiv preprint: https://arxiv.org/abs/1803.02155 Singh C, Hsu AR, Antonello R, Jain S, Huth AG, ..., Gao J (2023). Explaining black box text modules in natural language with language models. arXiv preprint: https://arxiv.org/abs/2305.09863 Singh M, Jakhar AK, Pandey S (2021). Sentiment analysis on the impact of coronavirus in social life using the bert model. Social Network Analysis and Mining, 11(1): 33. https://doi.org/10.1007/s13278-021-00737-z Tabinda Kokab S, Asghar S, Naz S (2022). Transformer-based deep learning models for the sentiment analysis of social media data. Array, 14:100157. https://doi.org/10.1016/j.array.2022.100157 Thogesan T, Nugaliyadde A, Wong KW (2025). Integration of explainable ai techniques with large language models for enhanced interpretability for sentiment analysis. arXiv preprint: https://arxiv.org/abs/2503.11948 Tyagi A, Sharma N (2018). Sentiment analysis using logistic regression and effective word score heuristic. International Journal of Engineering and Technology (UAE), 7: 20–23. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, ..., Polosukhin I (2017). Attention is all you need. Advances in Neural Information Processing Systems, 5998–6008. Wu Y, Schuster M, Chen Z, Le QV, Norouzi M,..., Dean J (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint: https://arxiv.org/abs/1609.08144 Wu Z, Ong DC (2021). Context-guided bert for targeted aspect-based sentiment analysis. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35:16, 14094–14102. Xiang M, Grove J, Giannakidou A (2014). Semantic and pragmatic processes in the comprehension of negation: An event related potential study of negative polarity sensitivity. Journal of Neurolinguistics, 38: 71–88. Xiong D, Park S, Lim J, Wang T, Wang X (2024). Bayesian multiple instance classification based on hierarchical probit regression. The Annals of Applied Statistics, 18(1): 80–99. https://doi.org/10.1214/23-AOAS1780 Yang C (2025). Famic: Code and pretrained models. https://github.com/YCY198888/FAMIC. Accessed: 2025-12-27 Yang C, Cao J (2025). Interpretable sentiment analysis using the attention-based multiple instance classification model: An application to wine reviews. Harvard Data Science Review, 7(2). https://doi.org/10.1162/99608f92.caab9466 Zhang S, Zheng D, Hu X, Yang M (2015). Bidirectional long short-term memory networks for relation classification. Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, 73–78. Zhang Y, Wallace BC (2017). A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 253–263. Zhao A, Yu Y (2021). Knowledge-enabled bert for aspect-based sentiment analysis. Knowledge-Based Systems, 227:107220. https://doi.org/10.1016/j.knosys.2021.107220 Zhou B, Khosla A, Lapedriza À, Oliva A, Torralba A (2015). Learning deep features for discriminative localization. CoRR. arXiv preprint: https://arxiv.org/abs/1512.04150