Improving Systematic Mapping Reviews

Abstract

Context: Systematic mapping reviews are now common in software engineering. However, if there are rigorous guidelines for systematic literature reviews in software engineering, the same is not true for mapping reviews. Objective: Develop a set of rigorous guidelines for mapping reviews. Method: The guidelines were based on some orientations for mapping reviews, mapping studies and on adaptations from existent guidelines for systematic literature reviews. Results: A set of guidelines, presented as a method, consisting of six stages. Conclusion: The guidelines form a well structured method for mapping reviews with the potential to attain high-quality reviews.

References

1. C. Torgerson, Systematic Reviews, ser. Research methods series. Continuum International Publishing Group, 2003.
2. B. Kitchenham and S. Charters, "Guidelines for performing Systematic Literature Reviews in Software Engineering," Keele University and Durham University Joint Report, Tech. Rep. EBSE 2007-001, 2007.
3. B. A. Kitchenham, D. Budgen, and O. P. Brereton, "Using mapping studies as the basis for further research: A participant-observer case study," Information and Software Technology, vol. 53, no. 6, pp. 638--651, 2011, special Section: Best papers from the APSEC.
4. B. Kitchenham and P. Brereton, "A systematic review of systematic review process research in software engineering," Information and Software Technology, vol. 55, no. 12, pp. 2049--2075, 2013.
5. B. Kitchenham, P. Brereton, and D. Budgen, "The educational value of mapping studies of software engineering literature," in Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, ser. ICSE '10. New York, NY, USA: ACM, 2010, pp. 589--598.
6. C. Wohlin, P. Runeson, P. A. da Mota Silveira Neto, E. Engstrom, I. do Carmo Machado, and E. S. de Almeida, "On the reliability of mapping studies in software engineering," Journal of Systems and Software, vol. 86, no. 10, pp. 2594--2610, 2013.
7. K. Petersen, S. Vakkalanka, and L. Kuzniarz, "Guidelines for conducting systematic mapping studies in software engineering: An update," Information and Software Technology, vol. 64, pp. 1--18, 2015. {Online}. Available: http://www.sciencedirect.com/science/article/pii/ S0950584915000646
8. K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, "Systematic mapping studies in software engineering," 12th International Conference on Evaluation and Assessment in Software Engineering, vol. 17, no. 1, pp. 1--10, 2007.
9. H. Arksey and L. O'Malley, "Scoping studies: towards a methodological framework," International journal of social research methodology, vol. 8, no. 1, pp. 19--32, 2005.
10. J. Clapton, D. Rutter, and N. Sharif, "Scie systematic mapping guidance," Social Care Institute for Excellence (SCIE), Tech. Rep. 03, April 2009.
11. J. Bailey, D. Budgen, M. Turner, B. Kitchenham, P. Brereton, and S. Linkman, "Evidence relating to object-oriented software design: A survey," in Empirical Software Engineering and Measurement, 2007. ESEM 2007. 1st International Symposium on, sept. 2007, pp. 482--484.
12. M. Petticrew and H. Roberts, Systematic reviews in the social sciences: A practical guide. Blackwell Publishing, 2006.
13. A. Booth and A. Fry-Smith, "Developing the research question," Etext on Health Technology Assessment (HTA) Information Resources, 2004, last access: 19 November, 2014. {Online}. Available: http://www.nlm.nih.gov/archive/ 20060905/nichsr/ehta/chapter2.html
14. U. of York. Centre for Reviews, Dissemination, and J. Akers, Systematic reviews: CRD's guidance for undertaking reviews in health care. Centre for Reviews and Dissemination, 2009.
15. P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, "Lessons from applying the systematic literature review process within the software engineering domain," Journal of Systems and Software, vol. 80, no. 4, pp. 571--583, 2007.
16. S. Casteleyn, I. Garrig'os, and J.-N. Maz'on, "Ten years of rich internet applications: A systematic mapping study, and beyond," ACM Trans. Web, vol. 8, no. 3, pp. 18:1--18:46, Jul. 2014.
17. B. Kitchenham, "What's up with software metrics? - a preliminary mapping study," J. Syst. Softw., vol. 83, no. 1, pp. 37--51, Jan. 2010.
18. D. Budgen, M. Turner, P. Brereton, and B. Kitchenham, "Using mapping studies in software engineering," in PPIG - Psychology of Programming Interest Group 2008. PPIG, 2008, pp. 195--204.
19. D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and T. P. Group, "Preferred reporting items for systematic reviews and meta-analyses: The prisma statement," Annals of Internal Medicine, vol. 151, no. 4, pp. 264--269, 2009.
20. J. Higgins and G. (editors), Cochrane handbook for systematic reviews of interventions 5.1.0. Cochrane Collaboration, 2011, available from:. {Online}. Available: www.cochrane-handbook.org
21. B. Kitchenham, T. Dyba, and M. Jorgensen, "Evidence-based software engineering," in Proceedings of the 26th International Conference on Software Engineering, ser. ICSE '04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 273--281.