Jitendra Singh; Geeta Sharma

From Sentiment to Cognition: Task-Driven Deci-sion Framework for Transfer Learning in 4-Class Cognitive Thought Classification

PDF (1377KB), PP.346-367

Views: 0 Downloads: 0

Author(s)

Jitendra Singh ^1,* Geeta Sharma ²

1. Department of Computer Science and Engineering, Lovely Professional University, Phagwara, Punjab, 144401, India

2. School of Computer Applications, Lovely Professional University, Phagwara, Punjab, 144401, India

* Corresponding author.

DOI: https://doi.org/10.5815/ijem.2026.05.19

Received: 11 Jun. 2026 / Revised: 24 Jun. 2026 / Accepted: 9 Jul. 2026 / Published: 8 Oct. 2026

Index Terms

Transfer Learning, Sentiment Analysis, Thought Classification, BiLSTM, Active Learning, Cognitive NLP, Domain Adaptation, PRISMA Review, Mental Health NLP, Decision Framework

Abstract

Traditional sentiment analysis performs well in terms of polarity detection, but it cannot capture cognitive nuances needed for mental health monitoring. In this paper, we address two gaps in the current literature: the lack of task-driven guidance on how to choose transfer learning strategies and the lack of validation on cognitively rich da-tasets, other than standard polarity benchmarks. We present a systematic review of 150 transfer learning studies follow-ing PRISMA guidelines, summarising the findings into a decision framework that maps data availability, domain spec-ificity, and resource constraints to optimal model selection. We empirically validate our decision framework on a novel 4-class cognitive thought classification dataset with human-elicited samples and GPT-4 augmented data. Active learn-ing reduced annotation effort by 60% and achieved high inter-rater agreement (Fleiss’ Kappa = 0.82). According to our framework, 12 models were evaluated. BiLSTM-TF-IDF achieved 93.9% macro F1, outperforming transformer models by 0.5% with 9× less compute, supporting the medium-data/domain-specific path. Specifically, on our 4,944-sample domain-specific cognitive thought dataset using a single NVIDIA RTX 3090 GPU, BiLSTM-TF-IDF outperformed RoBERTa-base by 0.5% macro F1 (93.9% vs. 93.4%, McNemar p=0.004) via uncertainty-sampling active learning with SVM confidence threshold <0.40, while requiring approximately 9.4× less inference time and a 32× smaller model footprint (with a 6× lower peak GPU memory requirement). Uncertainty was quantified using least-confidence sam-pling (selecting samples where max class probability <0.40); the cross-entropy loss function was used for both the SVM-guided active learning and BiLSTM training. We also evaluate cross-dataset performance on IMDb and SST-2, demonstrating reasonable generalisability. Demographic analysis shows cognitive patterns that have implications for mental health. The dataset and code have been published at Zenodo (DOI: 10.5281/zenodo.17444289) to facilitate reproducible cognitive NLP research.

Cite This Paper

Jitendra Singh, Geeta Sharma, "From Sentiment to Cognition: Task-Driven Decision Framework for Transfer Learning in 4-Class Cog-nitive Thought Classification", International Journal of Engineering and Manufacturing (IJEM), Vol.16, No.5, pp. 346-367, 2026. DOI:10.5815/ijem.2026.05.19

Reference

[1]N. Levin, Y. Lipshits-Braziler, and I. Gati, "The identification and validation of five types of career indecision: A latent profile analysis of career decision-making difficulties," Journal of Counseling Psychology, vol. 69, no. 4, pp. 452–462, 2022. https://doi.org/10.1037/cou0000603
[2]E. L. Rice and B. L. Fredrickson, "Do positive spontaneous thoughts function as incentive salience?" Emotion, vol. 17, no. 5, pp. 840–855, 2017. https://doi.org/10.1037/emo0000284
[3]F. Benrouba and R. Boudour, "Emotional sentiment analysis of social media content for mental health safety," Social Network Analysis and Mining, vol. 13, no. 1, Art. 17, 2023. https://doi.org/10.1007/s13278-022-01000-9
[4]F. Safari and A. Chalechale, "Emotion and personality analysis and detection using natural language processing, advances, chal-lenges and future scope," Artificial Intelligence Review, vol. 56, no. S3, pp. 3273–3297, 2023. https://doi.org/10.1007/s10462-023-10603-3
[5]J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language under-standing," in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), vol. 1, Minneapolis, MN, USA, 2019, pp. 4171–4186. https://doi.org/10.18653/v1/N19-1423
[6]Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, "RoBERTa: A robust-ly optimized BERT pretraining approach," arXiv:1907.11692, 2019. https://doi.org/10.48550/arXiv.1907.11692
[7]Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V. Le, "XLNet: Generalized autoregressive pretraining for lan-guage understanding," in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, Vancouver, Canada, 2019, pp. 5753–5763. https://doi.org/10.5555/3454287.3454804
[8]M. Wankhade, A. C. S. Rao, and C. Kulkarni, "A survey on sentiment analysis methods, applications, and challenges," Artificial Intelligence Review, vol. 55, no. 7, pp. 5731–5780, 2022. https://doi.org/10.1007/s10462-022-10144-1
[9]S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, "Deep learning–based text classification: A com-prehensive review," ACM Computing Surveys, vol. 54, no. 3, Art. 62, pp. 1–40, 2021. https://doi.org/10.1145/3439726
[10]T. A. Al-Qablan, M. H. Mohd Noor, M. A. Al-Betar, and A. T. Khader, "A survey on sentiment analysis and its applications," Neural Computing and Applications, vol. 35, no. 29, pp. 21567–21601, 2023. https://doi.org/10.1007/s00521-023-08941-y
[11]A. Rietzler, S. Stabinger, P. Opitz, and S. Engl, "Adapt or get left behind: Domain adaptation through BERT language model fine-tuning for aspect-target sentiment classification," in Proc. 12th Language Resources and Evaluation Conf. (LREC), Marseille, France, 2020, pp. 4933–4941. https://aclanthology.org/2020.lrec-1.607/
[12]M. Ryu, G. Lee, and K. Lee, "Knowledge distillation for BERT unsupervised domain adaptation," Knowledge and Information Systems, vol. 64, no. 11, pp. 3113–3128, 2022. https://doi.org/10.1007/s10115-022-01736-y
[13]L. R. Krosuri and R. S. Aravapalli, "Novel heuristic-based hybrid ResNeXt with RNN for multi-class sentiment analysis," Ma-chine Learning: Science and Technology, vol. 4, no. 1, Art. 015033, 2023. https://doi.org/10.1088/2632-2153/acc0d5
[14]L. Augustyniak, S. Woźniak, M. Gruza, P. Gramacki, K. Rajda, M. Morzy, and T. Kajdanowicz, "Massively multilingual corpus of sentiment datasets and multi-faceted sentiment classification benchmark," arXiv:2306.07902, 2023. https://doi.org/10.48550/arXiv.2306.07902
[15]Saeed, M., Ahmed, N., Ali, D., Ramzan, M., Mohib, M., Bagga, K., ... & Khan, I. M. (2024). In-depth Urdu Sentiment Analysis Through Multilingual BERT and Supervised Learning Approaches. ICCK Transactions on Intelligent Systematics, 1(3), 161-175. https://doi.org/10.62762/TIS.2024.585616
[16]N. J. Prottasha et al., "Transfer learning for sentiment analysis using BERT based supervised fine-tuning," Sensors, vol. 22, no. 11, Art. 4157, 2022. https://doi.org/10.3390/s22114157
[17]F. Khan and S. Shahid, "BiLSTM with self-attention and joint dual input learning for Hindi and Bengali sentiment analysis," Pat-tern Recognition Letters, vol. 154, pp. 1–8, 2022. https://doi.org/10.1016/j.patrec.2022.01.001
[18]C. Schröder and G. Heyer, "Self-training for sample-efficient active learning for text classification with pre-trained language mod-els," in Proc. 2024 Conf. Empirical Methods in Natural Language Processing (EMNLP), Miami, FL, USA, 2024, pp. 11987–12004. https://doi.org/10.18653/v1/2024.emnlp-main.669
[19]K. Margatina and N. Aletras, "On the limitations of simulating active learning," in Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, 2023, pp. 4402–4419. https://doi.org/10.18653/v1/2023.findings-acl.269
[20]Desolda, G., Esposito, A., Lanzilotti, R., Piccinno, A., & Costabile, M. F. (2025). From human-centered to symbiotic artificial intelligence: a focus on medical applications. Multimedia Tools and Applications, 84(27), 32109-32150. https://doi.org/10.1007/s11042-024-20414-5
[21]F. Liu et al., "Application of large language models in medicine," Nature Reviews Bioengineering, vol. 3, no. 6, pp. 445–464, 2025. https://doi.org/10.1038/s44222-025-00279-5
[22]A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, "Learning word vectors for sentiment analysis," in Proc. 49th Annual Meeting of the Association for Computational Linguistics (ACL), Portland, OR, USA, 2011, pp. 142–150. https://aclanthology.org/P11-1015/ (pre-2016 ACL Anthology entry; no DOI assigned)
[23]X. Zhang, J. Zhao, and Y. LeCun, "Character-level convolutional networks for text classification," in Advances in Neural Infor-mation Processing Systems (NeurIPS), vol. 28, Montreal, Canada, 2015, pp. 649–657. https://doi.org/10.5555/2969239.2969312
[24]T. T. Hoang, T. N. Nguyen, T. T. Nguyen, and T. M. Pham, "Aspect-based sentiment analysis: A survey of deep learning ap-proaches," IEEE Access, vol. 9, pp. 142988–143015, 2021. https://doi.org/10.1109/ACCESS.2021.3120969
[25]V. B. Vaghela, J. Bhatt, and D. Patel, "Aspect-based sentiment analysis using self-attention-based LSTM model," Journal of Computer Science, vol. 20, no. 10, pp. 1195–1202, 2024. https://doi.org/10.3844/jcssp.2024.1195.1202
[26]R. Catelli, F. Gargiulo, V. Casola, G. De Pietro, H. Fujita, and M. Esposito, "Cross-lingual transfer learning for sentiment analy-sis of Italian texts," Expert Systems with Applications, vol. 209, Art. 118246, 2022. https://doi.org/10.1016/j.eswa.2022.118246
[27]F. Koto, T. Beck, Z. Talat, I. Gurevych, and T. Baldwin, "Zero-shot sentiment analysis in low-resource languages using a multi-lingual sentiment lexicon," in Proc. 18th Conf. European Chapter of the Association for Computational Linguistics (EACL), Vol-ume 1: Long Papers, St. Julian's, Malta, 2024, pp. 298–320. https://doi.org/10.18653/v1/2024.eacl-long.18.
[28]S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010. https://doi.org/10.1109/TKDE.2009.191
[29]J. Singh and G. Sharma, "Sentiment analysis study of human thoughts using machine learning techniques," in Proc. Int. Conf. on Data Science and Technology (ICDT), 2023, pp. 776–785. https://doi.org/10.1109/ICDT57929.2023.10150917
[30]A. Malte and P. Ratadiya, "Evolution of transfer learning in natural language processing," arXiv:1910.07370, 2019. https://doi.org/10.48550/arXiv.1910.07370
[31]Y. Dai, H. Ye, and L. Dan, "Adversarial training based multi-source unsupervised domain adaptation for sentiment analysis," arXiv:2006.05602, 2020. https://doi.org/10.48550/arXiv.2006.05602
[32]A. Dong, T. Liu, B. Tang, and X. Wang, "Syntax guided domain adaptation for aspect-based sentiment analysis," arXiv:2211.05457, 2022. https://doi.org/10.48550/arXiv.2211.05457
[33]Y. Du, J. Sun, B. Qiu, and J. Li, "Domain adaptation for sentiment analysis using robust internal representation alignment," in Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.769
[34]F. Xu, Z. Pan, and R. Xia, "A domain-adaptive sentiment analysis method integrating attentional mechanisms and adversarial training," The Journal of Supercomputing, vol. 81, Art. 1117, 2025. https://doi.org/10.1007/s11227-025-07608-w
[35]A. Rogers, O. Kovaleva, and A. Rumshisky, "A primer in BERTology: What we know about how BERT works," Transactions of the Association for Computational Linguistics, vol. 8, pp. 842–866, 2020. https://doi.org/10.1162/tacl_a_00349
[36]M. Jiang, H. Chen, and Y. Feng, "Hybrid model combining BERT and CNN for sentiment analysis," Multimedia Tools and Ap-plications, vol. 81, pp. 15439–15459, 2022. https://doi.org/10.1007/s11042-022-12461-3
[37]S. Yang, J. Xing, Z. Liu, and Y. Sun, "Short-text sentiment classification model based on BERT and dual-stream Transformer gated attention mechanism," Electronics, vol. 14, no. 19, Art. 3904, 2025. https://doi.org/10.3390/electronics14193904
[38]C. Sun, X. Qiu, Y. Xu, and X. Huang, "How to fine-tune BERT for text classification?" in Proc. 18th China National Conf. on Computational Linguistics (CCL), Kunming, China, 2019, pp. 194–206. https://doi.org/10.1007/978-3-030-32381-3_16
[39]M. Sahar, H. U. Manzoor, and A. Hanif, " Transfer Learning-Based Framework for Sentiment Classification of Cosmetics Prod-ucts Reviews," Pakistan journal of engineering and technology, 5(3), 38-43, vol. 5, no. 3, pp. 38-43, 2022. https://doi.org/10.51846/vol5iss3pp38-43
[40]A. Singh, S. Mehta, and R. Bhardwaj, "Real-time sentiment analysis of Twitter data using fine-tuned RoBERTa," Procedia Com-puter Science, vol. 199, pp. 1618–1625, 2022. https://doi.org/10.1016/j.procs.2022.01.205
[41]V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter," arXiv:1910.01108, 2019. https://doi.org/10.48550/arXiv.1910.01108
[42]Ping, Z., Sang, G., Liu, Z., & Zhang, Y. Aspect category sentiment analysis based on prompt-based learning with attention mech-anism. Neurocomputing, 565, Article 126994, 2024. https://doi.org/10.1016/j.neucom.2023.126994
[43]Nie, E., Liang, S., Schmid, H., & Schütze, H. Cross-lingual retrieval augmented prompt for low-resource languages. Findings of the Association for Computational Linguistics: ACL 2023, ACL 2023, pp. 8320–8340, 2023. DOI: 10.18653/v1/2023.findings-acl.528
[44]H. Wang, B. Wang, M. Dou, and T. Zhang, "A short text classification method based on N-gram and CNN," Chinese Journal of Electronics, vol. 29, no. 2, pp. 248–254, 2020. https://doi.org/10.1049/cje.2020.01.001
[45]Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, "ALBERT: A lite BERT for self-supervised learning of language representations," in Proc. Int. Conf. on Learning Representations (ICLR), Addis Ababa, Ethiopia, 2020. (OpenReview-hosted; no DOI assigned) https://openreview.net/forum?id=H1eA7AEtvS
[46]J.-R. Chang, L.-S. Chen, and L.-W. Lin, "A novel cluster based over-sampling approach for classifying imbalanced sentiment data," IAENG International Journal of Computer Science, vol. 48, no. 4, pp. 1118–1128, 2021.
[47]X. Yang, J. Liu, and W. Wei, "FinBERT: Financial sentiment analysis with pre-trained language models," arXiv:2006.08097, 2023. https://doi.org/10.48550/arXiv.2006.08097
[48]J. Lee et al., "BioBERT: A pre-trained biomedical language representation model for biomedical text mining," Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020. https://doi.org/10.1093/bioinformatics/btz682
[49]H. Nouri, S. Karim, and N. Habbat, "Customer sentiment analysis and prediction of halal restaurants using machine learning and big data analytics," International Journal of Electrical and Computer Engineering, vol. 13, no. 4, pp. 4504–4515, 2023. https://doi.org/10.11591/ijece.v13i4.pp4504-4515
[50]M. J. Page et al., "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," BMJ, vol. 372, Art. n71, 2021. https://doi.org/10.1136/bmj.n71
[51]Y. Wang, X. Cheng, and X. Meng, "Sentiment analysis with BERT and Bi-LSTM based on multi-head attention," IAENG Inter-national Journal of Computer Science, vol. 50, no. 1, 2023.
[52]S. Gururangan et al., "Don't stop pretraining: Adapt language models to domains and tasks," in Proc. 58th Annual Meeting of the Association for Computational Linguistics (ACL), Online, 2020, pp. 8342–8360. https://doi.org/10.18653/v1/2020.acl-main.740
[53]S. Raj, T. Sharma, and N. Gupta, "Multilingual transfer learning with mBERT for sentiment classification," Applied Intelligence, vol. 51, pp. 712–723, 2021. https://doi.org/10.1007/s10489-020-01863-z
[54]W. Zhao and D. Li, "Sentiment analysis with multilingual transformers for low-resource languages," ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 22, no. 1, pp. 1–25, 2023. https://doi.org/10.1145/3554924
[55]U. Pathak and P. Rai, "Sentiment analysis: Methods, applications, and future directions," International Journal for Research in Applied Science and Engineering Technology, vol. 11, no. 2, pp. 1453–1458, 2023. https://doi.org/10.22214/ijraset.2023.49165
[56]H. Zhang, Y. Liu, J. Wang, and X. Zhao, "A sentiment analysis method based on bidirectional LSTM networks with attention mechanism," Applied Mathematics and Nonlinear Sciences, vol. 8, no. 1, pp. 55–68, 2022. https://doi.org/10.2478/amns.2022.1.00015
[57]J. H. Joloudari et al., "BERT-deep CNN: State of the art for sentiment analysis of COVID-19 tweets," Social Network Analysis and Mining, vol. 13, Art. 99, 2023. https://doi.org/10.1007/s13278-023-01102-y
[58]OpenAI, "Usage policies," OpenAI, San Francisco, CA, USA, 2023. [Online]. Available: https://openai.com/policies/usage-policies (no DOI — website resource)
[59]L. Qiang, X. Sun, and Y. Long, "Sentiment analysis: Comprehensive reviews, recent advances, and open challenges," IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 11, pp. 15092–15112, 2024. https://doi.org/10.1109/TNNLS.2023.3294810
[60]N. Alturayeif and I. Ahmad, "EASE: An enhanced active learning framework for aspect-based sentiment analysis based on sample diversity and data augmentation," Expert Systems with Applications, vol. 261, Art. 125525, 2025. https://doi.org/10.1016/j.eswa.2024.125525
[61]B. Wang, P. Deng, Y. Zhao, and B. Qin, "C2D2 dataset: A resource for analyzing cognitive distortions and its impact on mental health," in Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 2023, pp. 10149–10160. https://doi.org/10.18653/v1/2023.findings-emnlp.680

International Journal of Engineering and Manufacturing (IJEM)