Meta-analysis of Systematic Literature Review Methods

Full Text (PDF, 409KB), PP.17-25

Views: 0 Downloads: 0


Iman Tikito 1,* Nissrine Souissi 2

1. Mohammed V University in Rabat, EMI-SIWEB Team, Rabat Morocco

2. Mines-Rabat School, Computer Sciences Department, Rabat Morocco

* Corresponding author.


Received: 19 Nov. 2018 / Revised: 10 Dec. 2018 / Accepted: 16 Dec. 2018 / Published: 8 Feb. 2019

Index Terms

SLR, Systematic Literature Review, Data, Steps.


Plenty data are generated each second under different locations and ways. In return, several valuable data are not used due to neglect or a wrong criteria selec-tion, affecting the results. To have the accurate decision the exact information is needed. But, in case of lack of data, an overall vision about all existing findings make a big difference especially in Medicine.
The approach of Systematic Literature Review give the possibility to have this clear vision related to a specific topic, for the right input. We aim to identify all important steps to conduct a high quality of systematic literature review, independently of research domain.
We first explain the reason behind our research and the significance of Systematic Literature Review in research fields, one of the way for minimizing the loss of infor-mation. Since our goal is to have an explicit methodolo-gy, we select several papers mentioning the procedure in different years to have a general view. Subsequently, we explain the approaches used to select the key steps to follow for selected articles.
We settle this study with a complete step resuming all finding key phases. This research highlight also the im-portance of the use of an explicit methodology to vanish any misunderstanding or ignoring basic points to accom-plish a valuable and high quality work.
This paper make a clear complete methodology to fol-low, in any domain and specially medicine, to conduct a best quality of Systematic Literature Review and reach the right gain using best practices.

Cite This Paper

Iman Tikito, Nissrine Souissi, "Meta-analysis of Systematic Literature Review Methods", International Journal of Modern Education and Computer Science(IJMECS), Vol.11, No.2, pp. 17-25, 2019.DOI: 10.5815/ijmecs.2019.02.03


[1]Y. WANG, L. KUNG and T. A. BYRD, "Big data analyt-ics: Understanding its capabilities and potential benefits for healthcare organizations," vol. 126, pp. 3-13, 2016.
[2]Kim, S. Song and B.-Y. Choi, Data Deduplication for Data Optimization for Storage and Network Systems, Springer, 2017.
[3]GOUGH, S. OLIVER and J. THOMAS, An introduction to systematic reviews, Sage, 2017.
[4]DUMEZ Hervé, "Faire une revue de littérature: pourquoi et comment?," Le Libellio d'Aegis, vol. 7, no. 2-Eté, pp. 15-27, 2011.
[5]Andrew Booth, Diana Papaioannou and Anthea Sutton, Systematic Approaches to a Successful Literature Review, SAGE, 2012.
[6]B. Hilda, G. Paul and C. Iain, "Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?," PLoS Med, vol. 7, no. 9, p. e1000326, 2010.
[7]IOANNIDIS John, "The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses," The Milbank Quarterly, vol. 94, no. 3, pp. 485-514, 2016.
[8]Ramiro S., "SP0028 Systematic literature review: where to start," Annals of the Rheumatic Diseases, vol. 76, p. 8, 2017.
[9]PETTICREW Mark, "Systematic reviews from astronomy to zoology: myths and misconceptions," BMJ: British Medical Journal, vol. 322, no. 7278, p. 98, 2001.
[10]HADDAWAY Neal R. and BILOTTA Gary S., "System-atic reviews: Separating fact from fiction," Environment international, vol. 92, pp. 578-584, 2016.
[11]FISHER Charles, PATEL Alpesh A. and VACCARO Alexander R, "High-Impact Literature: What It Requires Beyond Evidence-Based Medicine," Spine, vol. 39, no. 1, pp. E49-E51, 2014.
[12]Thomas M. Connolly, Elizabeth A. Boyle, Ewan MacAr-thur, Thomas Hainey and James M. Boyle, "A systematic literature review of empirical evidence on computer games and serious games," Computers & Education, vol.59, no. 2, pp. 661-686, 2012.
[13]EVANGELISTA Laura, BRIGANTI Alberto, FANTI Stef, JONIAU Stephen, RESKE Sven, SCHIAVINA Ric-cardo, STIEF Christian, THALMANN N. George and PICCHIO Maria, "New clinical indications for 18 F/11 C-choline, new tracers for positron emission tomography and a promising hybrid device for prostate cancer staging: a systematic review of the literature," European urology, vol. 70, no. 1, pp. 161-175, 2016.
[14]MOORE ADAMS, Brianne L. JONES, W. Monty and COHEN Jonathan, "Learning to teach online: a systematic review of the literature on K-12 teacher preparation for teaching online," Distance Education, vol. 37, no. 3, pp. 333-348, 2016.
[15]J. CRUZ-BENITO, "Systematic Literature Review & Mapping," 2016.
[16]LIGHT, Richard and SMITH, Paul, "Accumulating evi-dence: Procedures for resolving contradictions among dif-ferent research studies," vol. 41, no. 4, pp. 429-471, 1971.
[17]COCHRANE, Archibald Leman and et al., "Effectiveness and efficiency: random reflections on health services," 1972.
[18]CHALMERS, Iain., "The Cochrane collaboration: prepar-ing, maintaining, and disseminating systematic reviews of the effects of health care," vol. 703, no. 1, pp. 156-165, 1993.
[19]M. PETTICREW and H. ROBERTS, "Systematic reviews in the social sciences: A practical guide," 2006.
[20]DIESTE, M. LÓPEZ and F. RAMOS, "Formalizing a systematic review updating process," Software Engineering Research, Management and Applications, 2008. SERA'08. Sixth International Conference on. IEEE, pp. 143-150, 2008.
[21]SIDDAWAY, "What is a Systematic Literature Review and how do I do one," University of Stirling, pp. 1-13, 2014.
[22]KITCHENHAM and P. BRERETON, "A systematic review of systematic review process research in software engineering," Information and software technology, vol. 55, no. 12, pp. 2049-2075, 2013.
[23]BOAZ, Annette, ASHBY, Deborah, YOUNG, Ken and et al, Systematic reviews: what have they got to offer evidence based policy and practice?, London: ESRC UK Centre for Evidence Based Policy and Practice, 2002.
[24]B. Kitchenham and S. Charters, Guidelines for performing systematic literature reviews in software engineering (ver-sion 2.3), Keele University, UK, Tech. Rep. EBSE Tech-nical Report EBSE-2007-01, 2007.
[25]DUFF, Alistair, "The literature search: a library-based model for information skills instruction," Library review, vol. 45, no. 4, pp. 14-18, 1996.
[26]Cook DJ, Sackett DL and Spitzer WO, "Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis," no. 48:167–71, 1995.
[27]BHANDARI, Mohit, GUYATT, Gordon H., MONTORI, Victor and et al, "User's guide to the orthopaedic literature: how to use a systematic literature review," JBJS, vol. 84, no. 9, pp. 1672-1682, 2002.
[28]J. Biolchini, P.G. Mian, A.C.C. Natali and G.H. Trava, "Systematic Review in Software Engineering," in Technical Report TR – ES 679/05, COPPE/UFRJ, 2005.
[29]KITCHENHAM, Barbara, "Procedures for performing systematic reviews," Keele, UK, Keele University, vol. 33, no. 2004, pp. 1-26, 2004.
[30]H. SACKS S., J. BERRIER, D. REITMAN and e. al., "Meta-analyses of randomized controlled trials," vol. 316, no. 8, pp. 450-455, 1987.
[31]FELDMAN and Kenneth A., "Using the work of others: Some observations on reviewing and integrating," pp. 86-102, 1971.
[32]COOK, Deborah J., MULROW, Cynthia D. and HAYNES, R. Brian, "Systematic reviews: synthesis of best evidence for clinical decisions," Annals of internal medicine, vol. 126, no. 5, pp. 376-380, 1997.
[33]ROSENTHAL, Robert, "Writing meta-analytic reviews," vol. 118, no. 2, p. 183, 1995.
[34]B. KITCHENHAM, O. P. BRERETON, D. BUDGEN and . al., "Systematic literature reviews in software engi-neering–a systematic literature review," Information and software technology, vol. 51, no. 1, pp. 7-15, 2009.
[35]Egger M, Davey Smith DG and Altman DG, Systematic reviews in health care: meta analysis in context, London: BMJ Books, 2001.
[36]ALDERSON, Phil, GREEN, Sally, HIGGINS, J. P. T. and et al., "Cochrane Reviewers’ Handbook 4.2. 2.," The Cochrane Library, vol. 1, 2004.
[37]SCIE, Systematic Research Reviews: Guidelines (2nd edition), UK, Wales, Northern Ireland: Social Care Institute for Excellence, 2010.
[38]P. CRONIN, F. RYAN and M. COUGHLAN, "Undertak-ing a literature review: a step-by-step approach," British journal of nursing, vol. 17, no. 1, pp. 38-43, 2008.
[39]BOOTH, D. PAPAIOANNOU and A. SUTTON, "Sys-tematic approaches to a successful literature review," 2012.
[40]T. HALL, S. BEECHAM, D. BOWES and e. al, "A sys-tematic literature review on fault prediction performance in software engineering," IEEE Transactions on Software Engineering, vol. 38, no. 6, pp. 1276-1304, 2012.
[41]Alison Mackey and Susan M. Gass, Second Language Research: Methodology and Design, London: Lawrence Erlbaum Associates, 2005.
[42]V. R. BASILI and D. M. WEISS, "A methodology for collecting valid software engineering data," IEEE Trans-actions on software engineering, no. 6, pp. 728-738, 1984. DOI: 10.1109/TSE.1984.5010301.
[43]ZOZUS Meredith, The Data Book: Collection and Man-agement of Research Data, CRC Press, 2017.
[44]JONES, Tina and EVANS, David, "Conducting a system-atic review," Australian Critical Care, vol. 13, no. 2, pp. 66-71, 2000.
[45]BADGER, D., NURSTEN, J., WILLIAMS, P. and et al., "Should all literature reviews be systematic?," Evaluation & Research in Education, vol. 14, no. 3-4, pp. 220-230, 2000.
[46]NEEDLEMAN, Ian G, "A guide to systematic reviews," Journal of clinical periodontology, vol. 29, no. s3, pp. 6-9, 2002.
[47]CARR, Alan B, "Systematic reviews of the literature: the overview and meta-analysis," Dental Clinics of North America, vol. 46, no. 1, pp. 79-86, 2002.
[48]BIGBY, Michael and WILLIAMS, Hywel, "Appraising systematic reviews and meta-analyses," Archives of der-matology, vol. 139, no. 6, pp. 795-798, 2003.
[49]KHAN, Khalid S., KUNZ, Regina, KLEIJNEN, Jos and et al. , "Five steps to conducting a systematic review," Journal of the Royal Society of Medicine, vol. 96, no. 3, pp. 118-121, 2003.
[50]T. DYBA, T. DINGSOYR and G. K. HANSSEN, "Applying systematic reviews to diverse study types: An experience report. In : Empirical Software Engineering and Measurement, 2007. ESEM 2007," First International Symposium on. IEEE, pp. 225-234, 2007.
[51]L. CHEN, M. ALI BABAR and C. CROWLEY, "A status Report on the evaluation of variability management ap-proaches," 2009.
[52]M. Pai, M. McCulloch, J. Gorman, N. Pai, W. Enanoria, G. Kennedy, P. Tharyan and J. J. Colford, "Systematic re-views and meta-analyses: An illustrated step-by-step guide," The National Medical Journal of India, vol. 17, no. 2, pp. 86- 95, 2004.
[53]WHITE, Adrian and SCHMIDT, Katja, "Systematic liter-ature reviews," Complementary therapies in medicine, vol. 13, no. 1, pp. 54-60, 2005.
[54]P. MIAN, T. CONTE, A. NATALI and e. al, "A systemat-ic review process for software engineering," in In : ESELAW’05: 2nd Experimental Software Engineering Latin American Workshop, 2005.
[55]R. Haynes, D. Sackett, G. Guyatt and a. et, Clinical Epi-demiology: How to Do Clinical Practice Research, 3rd edn, Philadelphia: Lippincott Williams & Wilkins, 2006.
[56]P. BRERETON, B. A. KITCHENHAM, D. BUDGEN and . al, "Lessons from applying the systematic literature review process within the software engineering domain," Journal of systems and software, vol. 80, no. 4, pp. 571-583, 2007.
[57]B. A. KITCHENHAM, D. BUDGEN and P. BRERETON, "Evidence-Based Software engineering and systematic reviews," CRC Press, 2015.
[58]B. D. A. C. Jorge, M. G. Paula, N. C. A. Candida and al., "Scientific research ontology to support systematic review in software engineering," Advanced Engineering Infor-matics, vol. 21, no. 2, pp. 133-151, 2007.
[59]R. ARMSTRONG, E. WATERS, N. JACKSON and e. al., Systematic reviews of health promotion and public health interventions, Melbourne, 2007.
[60]D. MOHER, A. LIBERATI, J. TETZLAFF and e. al., "Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement," PLoS medicine, vol. 6, no. 7, p. e1000097, 2009.
[61]D. Denyer and D. Tranfield, "Producing a systematic review," in Buchanan, D. and Bryman, A. (Eds), The Sage Handbook of Organizational Research Methods, Sage Publications, London, pp. 671-689, 2009.
[62]Centre for Reviews and Dissemination, Systematic re-views: CRD’s guidance for undertaking reviews in healthcare, UK: University of York, 2009.
[63]C. OKOLI and K. SCHABRAM, "A guide to conducting a systematic literature review of information systems re-search," Sprouts Work. Pap. Inf. Syst, vol. 10, no. 26, 2010.
[64]RYAN, "Guidance notes on planning a systematic review," James Hardiman Library, 2010. [Online]. Available: http://www. library. nuigalway. ie/media/jameshardimanlibrary/-content/documents/support/Guidance% 20on% 20planning% 20a% 20systematic% 20review. pdf.
[65]ZHANG and M. A. BABAR, "An empirical investigation of systematic reviews in software engineering," Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on. IEEE, pp. 87-96, 2011.
[66]J. JESSON, L. MATHESON and F. M. LACEY, "Doing your literature review: Traditional and systematic tech-niques," in Sage, 2011.
[67]L. S. UMAN, "Systematic reviews and meta-analyses," Journal of the Canadian Academy of Child and Adolescent Psychiatry, vol. 20, no. 1, p. 57, 2011.
[68]J. BETTANY-SALTIKOV, "How to do a systematic literature review in nursing: a step-by-step guide," McGraw-Hill Education (UK), 2012.
[69]C. COLICCHIA and F. STROZZI, "Supply chain risk management: a new methodology for a systematic literature review," Supply Chain Management: An International Journal, vol. 17, no. 4, pp. 403-418, 2012.
[70]J. C. CARVER, E. HASSLER, E. HERNANDES and . al., "Identifying barriers to the systematic literature review process," In : Empirical Software Engineering and Meas-urement, 2013 ACM/IEEE International Symposium on. IEEE, pp. 203-212, 2013.
[71]E. HASSLER, J. C. CARVER, N. A. KRAFT and e. al., "Outcomes of a community workshop to identify and rank barriers to the systematic literature review process," In : Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. ACM, p. 31, 2014.
[72]KOTHARI Chakravanti Rajagopalachari, Research meth-odology: Methods and techniques, New Age International, 2004.
[73]N. STEMBERT, P. CONRADIE, I. MULDER and S. CHOENNI, "Participatory data gathering for public sector reuse: Lessons learned from traditional initiatives," Inter-national Conference on Electronic Government. Springer, Berlin, Heidelberg, pp. 87-98, 2013. DOI: 10.1007/978-3-642-40358-3_8.
[74]Z. STAPIĆ, E. G. LÓPEZ, A. G. CABOT and . al., "Per-forming systematic literature review in software engineer-ing," In : CECIIS 2012-23rd International Conference, 2012.