Interpretable Word-Level Context-Based Sentiment Analysis
Volume 24, Issue 2 (2026): Special Issue: The 2025 Symposium on Data Science and Statistics (SDSS 2025),, pp. 319–337
Pub. online: 7 May 2026
Type: Statistical Data Science
Open Access
Open Access
Received
31 July 2025
31 July 2025
Accepted
26 February 2026
26 February 2026
Published
7 May 2026
7 May 2026
Abstract
We propose a fine-grained attention-based multiple instance classification (FAMIC) model for interpretable word-level sentiment analysis (SA) using only document-level sentiment labels. By operating at the word level, FAMIC enhances interpretability while maintaining competitive performance in document-level classification. The model generates interpretable outputs such as contextual weighting, word neutrality, and negation cues, offering insights into how context shapes sentiment and how the model arrives at its predictions. FAMIC is built on a straightforward yet effective architecture that combines a multiple instance classification framework with self-attention and positionally encoded self-attention blocks. This design enables the model to capture both local and global contextual dependencies, supporting nuanced sentiment interpretation. We evaluate FAMIC on two sentiment classification datasets and provide an extensive analysis of its interpretability and performance.
References
Abnar S, Zuidema W (2020). Quantifying attention flow in transformers. arXiv preprint: https://arxiv.org/abs/2005.00928
Balderas L, Lastra M, Benítez JM (2023). Can persistent homology whiten transformer-based black-box models? A case study on bert compression. arXiv preprint: https://arxiv.org/abs/2312.10702
Bilan I, Roth B (2018). Position-aware self-attention with relative positional encodings for slot filling. arXiv preprint: https://arxiv.org/abs/1807.03052
Carbonneau MA, Cheplygina V, Granger E, Gagnon G (2018). Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition, 77: 329–353. https://doi.org/10.1016/j.patcog.2017.10.009
Chen K, Wang R, Utiyama M, Sumita E (2021). Context-aware positional representation for self-attention networks. Neurocomputing, 451: 46–56. https://doi.org/10.1016/j.neucom.2021.04.055
Das SR, Chen MY (2007). Yahoo! For Amazon: Sentiment extraction from small talk on the web. Management Science, 53(9): 1375–1388. https://doi.org/10.1287/mnsc.1070.0704
Devlin J, Chang MW, Lee K, Toutanova K (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186.
Dietterich TG, Lathrop RH, Lozano-Pérez T (1997). Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1): 31–71. https://doi.org/10.1016/S0004-3702(96)00034-3
Fang X, Zhan J (2015). Sentiment analysis using product review data. Journal of Big Data, 2, Article number: 5. https://doi.org/10.1186/s40537-015-0015-2
Ghader H, Monz C (2017). What does attention in neural machine translation pay attention to? arXiv preprint: https://arxiv.org/abs/1710.03348
Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Jain S, Wallace BC (2019). Attention is not explanation. arXiv preprint: https://arxiv.org/abs/2005.00928
Katumullage D, Yang C, Barth J, Cao J (2022). Using neural network models for wine review classification. Journal of Wine Economics, 17(1): 27–41. https://doi.org/10.1017/jwe.2022.2
Kindermans PJ, Hooker S, Adebayo J, Alber M, Schütt KT, ..., Kim B (2017). The (un)reliability of saliency methods. arXiv preprint: https://arxiv.org/abs/1711.00867
Liu B (2012). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1): 1–167. https://doi.org/10.1007/978-3-031-02145-9
Lundberg SM, Lee S (2017). A unified approach to interpreting model predictions. CoRR. arXiv preprint: https://arxiv.org/abs/1705.07874
Medhat W, Hassan A, Korashy H (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5(4): 1093–1113. https://doi.org/10.1016/j.asej.2014.04.011
Mikolov T, Chen K, Corrado GS, Dean J (2013). Efficient estimation of word representations in vector space. arXiv preprint: https://arxiv.org/abs/1301.3781
Minaee S, Azimi E, Abdolrashidi A (2019). Deep-sentiment: Sentiment analysis using ensemble of cnn and bi-lstm models. arXiv preprint: https://arxiv.org/abs/1904.04206
Petch J, Di S, Nelson W (2022). Opening the black box: The promise and limitations of explainable machine learning in cardiology. Canadian Journal of Cardiology, 38(2): 204–213. https://doi.org/10.1016/j.cjca.2021.09.004
Ribeiro MT, Singh S, Guestrin C (2016). “Why should I trust you?”: Explaining the predictions of any classifier. arXiv preprint: https://arxiv.org/abs/1602.04938
Rudin C (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. arXiv preprint: https://arxiv.org/abs/1811.10154
Shaw P, Uszkoreit J, Vaswani A (2018). Self-attention with relative position representations. arXiv preprint: https://arxiv.org/abs/1803.02155
Singh C, Hsu AR, Antonello R, Jain S, Huth AG, ..., Gao J (2023). Explaining black box text modules in natural language with language models. arXiv preprint: https://arxiv.org/abs/2305.09863
Singh M, Jakhar AK, Pandey S (2021). Sentiment analysis on the impact of coronavirus in social life using the bert model. Social Network Analysis and Mining, 11(1): 33. https://doi.org/10.1007/s13278-021-00737-z
Tabinda Kokab S, Asghar S, Naz S (2022). Transformer-based deep learning models for the sentiment analysis of social media data. Array, 14:100157. https://doi.org/10.1016/j.array.2022.100157
Thogesan T, Nugaliyadde A, Wong KW (2025). Integration of explainable ai techniques with large language models for enhanced interpretability for sentiment analysis. arXiv preprint: https://arxiv.org/abs/2503.11948
Wu Y, Schuster M, Chen Z, Le QV, Norouzi M,..., Dean J (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint: https://arxiv.org/abs/1609.08144
Xiong D, Park S, Lim J, Wang T, Wang X (2024). Bayesian multiple instance classification based on hierarchical probit regression. The Annals of Applied Statistics, 18(1): 80–99. https://doi.org/10.1214/23-AOAS1780
Yang C (2025). Famic: Code and pretrained models. https://github.com/YCY198888/FAMIC. Accessed: 2025-12-27
Yang C, Cao J (2025). Interpretable sentiment analysis using the attention-based multiple instance classification model: An application to wine reviews. Harvard Data Science Review, 7(2). https://doi.org/10.1162/99608f92.caab9466
Zhao A, Yu Y (2021). Knowledge-enabled bert for aspect-based sentiment analysis. Knowledge-Based Systems, 227:107220. https://doi.org/10.1016/j.knosys.2021.107220
Zhou B, Khosla A, Lapedriza À, Oliva A, Torralba A (2015). Learning deep features for discriminative localization. CoRR. arXiv preprint: https://arxiv.org/abs/1512.04150
