Enhancement of Single Document Text Summarization using Reinforcement Learning with Non-Deterministic Rewards

Full Text (PDF, 706KB), PP.19-27

Views: 0 Downloads: 0


K. Karpagam 1,* A. Saradha 2 K. Manikandan 3 K. Madusudanan 1

1. Dr. Mahalingam College of Engineering & Technology, Pollachi

2. Institute of Road Transport and Technology, Erode

3. Vellore Institute of Technology, Vellore

* Corresponding author.

DOI: https://doi.org/10.5815/ijitcs.2020.04.03

Received: 7 Feb. 2020 / Revised: 4 Mar. 2020 / Accepted: 16 Mar. 2020 / Published: 8 Aug. 2020

Index Terms

Complex Question answering system, Non deterministic Rewards, Reinforcement learning, Machine Learning, Text summarization


A text summarization system generates short and brief summaries of original document for given user queries. The machine generated summaries uses information retrieval techniques for searching relevant answers from large corpus. This research article proposes a novel framework for generating machine generated summaries using reinforcement learning techniques with Non-deterministic reward function.  Experiments have exemplified with ROUGE evaluation metrics with DUC 2001, 20newsgroup data. Evaluation results of proposed system with hypothesis of automatic summarization from given datasets prove that statistically significant improvement for answering complex questions with f- actual vs. critical values.

Cite This Paper

K.Karpagam, A. Saradha, K.Manikandan, K.Madusudanan, "Enhancement of Single Document Text Summarization using Reinforcement Learning with Non-Deterministic Rewards", International Journal of Information Technology and Computer Science(IJITCS), Vol.12, No.4, pp.19-27, 2020. DOI:10.5815/ijitcs.2020.04.03


[1]Aziz Qaroush , Ibrahim Abu Farha, Wasel Ghanem, Mahdi Washaha, Eman Maali.,” An efficient single document Arabic text summarization using a combination of statistical and  semantic features”, Journal of King Saud University - Computer  and Information Sciences  ,Elsevier, 2019. 

[2]Kaichun Yao, Libo Zhang, Tiejian Luo, Yanjun Wu, “Deep reinforcement  learning for extractive document summarization”, Neuro computing, Elsevier, pp. 52– 62, 2018.

[3]Shashi Narayan, Shay B. Cohen, Mirella Lapata , June,“ Summarization with Reinforcement Learning”, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, Pages: 1747–1759, 2018.

[4]Yalias Chali, Sadid A. Hasan & Mustapha Mojahid, ”Reinforcement Learning  Formulation to the Complex Question Answering Problem”, Information Processing & Management, Elsevier, 2015, vol. 51, Issue no 3, pp. 252-272, 2015.

[5]Rasmita Rautray, Rakesh Chandra Balabantaray& Anisha Bhardwaj ,”Document Summarization using Sentence Features”, International  Journal of Information Retrieval Res(IJIRR), Volume 5 Issue 1, pp.36-47,2015.

[6]Ming Tan, Cicero dos Santos, Bing Xiang & Bowen Zhou,” Improved Representation Learning for Question Answer Matching”, Proceedings of  the 54th Annual Meeting of the Association for  Computational Linguistics, August, pp. 7-12,2016. 

[7]SeonggiRyang & Takeshi Abekawa ,” Framework of Automatic Text Summarization using Reinforcement Learning’, In Proceedings of the 2012 Joint Conference on Empirical Methods  in Natural Language Processing & Computational Natural Language Learning, EMNLP- CoNLL Jeju Island, Korea pp. 256–265,2012.

[8]Denis Savenkov & Eugene Agichtein, “CRQA: Crowd- Powered Real-Time Automatic Question Answering System”, Proceedings, The Fourth AAAI Conference on  Human Computation & Crowdsourcing , Association for the Advancement of Artificial Intelligence, pp.189 -198,2016.

[9]Dorota Gowacka, Tuukka Ruotsalo, Ksenia Konyushkova , Kumaripaba Athukorala, Samuel Kaski& Giulio Jacucci, “ Directing Exploratory Search: Reinforcement Learning from User Interactions with Keywords”, IUI’13, Copyright ACM, pp.117-127,2013.

[10]Chin-Yew Lin, “ROUGE: A Package for Automatic Evaluation of Summaries”, In Proceedings of Workshop on Text Summarization branches out, Post- Conference Workshop of Association for Computational Linguistics, Barcelona, Spain , pp. 74–81, 2004,.

[11]Al-Radaideh, Q.A., Bataineh, D.Q, ,” A hybrid approach for arabic text summarization using domain knowledge and genetic algorithms”, Cognitive Computer, 2018.

[12]Rautray, R., Balabantaray, R.C. , ,An evolutionary  framework for multi-document summarization using cuckoo search approach, Mdscsa. Application Computer Information, volume 14,issue 2, pp.134–144, 2018.

[13]Smucker, M.D., Allan, J. and Carterette, B.,”A comparison of statistical significance tests for information retrieval evaluation”, In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, ACM, 623-  632,2007.

[14]Karpagam.K & Saradha.A, “Text Summarization using Machine Learning Approaches for Question Answering System”, International Journal of Advances in Computer and Electronics Engineering, ISSN: 2456 - 3935, Volume 4, Issue 2, pp.1–5,2019.

[15]Gunnar Schröder, Maik Thiele &Wolfgang Lehner , ‘Setting Goals & Choosing Metrics for Recommender System Evaluations ‘, 5th ACM   Conference on Dresden University of Technology   Recommender Systems Chicago, October 23th, 2011.

[16]Hua, Y.H., Chenb, Y.L., Chou, H.L,” Opinion  mining from  online hotel reviews a text  summarization approach”, Information Process Management 53 (2), pp.436–449,2017.

[17]Abdelkrime, A., Djamel Eddine, Z., Khaled Walid, H. ,“All summarizer system at  multilingual single and multi-document  summarization” Proceedings of the SIGDIAL 2015 Conference,pp.237–244, 2015.

[18]Litvak, M., Vanetik, N., Last, M., Churkin, E.. Museec,” A multilingual text summarization tool”, pp. 73–78,2016,.

[19]Thomas, S., Beutenmüller, C., de la Puente, X., Remus, R., Bordag, S,,., “Extractive text Summarizer”, Proceedings of the SIGDIAL 2015 Conference, pp. 260–269,2015.

[20]Jan Frederik Forst, Anastasios Tombros& Thomas Roelleke, ‘Less Is More: Maximal Marginal Relevance as a Summarization Feature’, Advances in Information Retrieval Theory, ICTIR, pp. 350-353, 2009.

[21]Youzheng Wu, Chiori Hori, Hideki Kashioka &Hisashi Kawai, “Leveraging social  QA collections for improving complex question answering”, Computer Speech & Language, Volume 29 Issue 1, pp.1-19,2015.

[22]Hiroyuki Sakai & Shigeru Masuyama, ,”A Multiple-Document Summarization System with User Interaction”, In Proceedings of the 20th International Conference on Computational   Linguistics, COLING ‘04, C04-1144, ACL Anthology, pp.1001- 1007,2004.