A Case of Mobile App Reviews as a Crowdsource

Full Text (PDF, 371KB), PP.39-47

Views: 0 Downloads: 0


Mubasher Khalid 1,* Usman Shehzaib 1 Muhammad Asif 1

1. Dept. of Computer Science, COMSATS Institute of Information Technology, Sahiwal, 57000, Pakistan

* Corresponding author.

DOI: https://doi.org/10.5815/ijieeb.2015.05.06

Received: 5 Jun. 2015 / Revised: 2 Jul. 2015 / Accepted: 5 Aug. 2015 / Published: 8 Sep. 2015

Index Terms

App reviews, Crowdsourcing, crowd capital, user experience, m-business


Crowdsourcing is a famous technique to get innovative ideas and soliciting contribution from a large online community particularly in e-business. This technique is contributing towards changing the current business techniques and practices. It is also equally famous in analysis and design of m-business services. Mobile app stores are providing an opportunity for its users' to participate and contribute in the growth of mobile app market. App reviews given by users usually contain active, heterogeneous and real life user experience of mobile app which can be useful to improve the quality of app. Best to our knowledge, the strength of mobile app reviews as a crowdsource is not fully recognized and understood by the community yet. In this paper, we have analysed a crowdsourcing reference model to find out which features of crowdsource are present and are related to our case of mobile app reviews as a crowdsource. We have analyzed and discussed each construct of the reference model from the perspective of mobile app reviews. Moreover, app reviews as a crowdsourcing technique is discussed by utilizing the four pillars of the reference model: the crowd, the crowdsourcer, the crowdsourcing, and the crowdsourcing platform. We have also identified and partially validated certain constructs of the model and highlighted the significance of app reviews as a crowdsource based on existing literature. In this study, only one crowdsourcing reference model is used which can be a limitation of our study. The study can be further investigated and compared with other crowdsourcing reference models to get better insights of app reviews as a crowdsource. We believe that the understanding of app reviews as a crowdsourcing technique can lead to the further development of the mobile app market and can open further research opportunities.

Cite This Paper

Mubasher Khalid, Usman Shehzaib, Muhammad Asif, "A Case of Mobile App Reviews as a Crowdsource", International Journal of Information Engineering and Electronic Business(IJIEEB), vol.7, no.5, pp.39-47, 2015. DOI:10.5815/ijieeb.2015.05.06


[1]Brabham, D.C., Crowdsourcing as a model for problem solving an introduction and cases. Convergence: the international journal of research into new media technologies, 2008. 14(1): p. 75-90.

[2]Chanal, V. and M.-L. Caron-Fasan. How to invent a new business model based on crowdsourcing: the Crowdspirit ® case. In EURAM (Lubjana, Slovenia, 2008).

[3]Estellés-Arolas, E. and F. González-Ladrón-de-Guevara, Towards an integrated crowdsourcing definition. Journal of Information science, 2012. 38(2): p. 189-200.

[4]Wikipedia. http://www.wikipedia.org/.

[5]Foncubierta Rodríguez, A. and H. Müller. Ground truth generation in medical imaging: a crowdsourcing-based iterative approach. In Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia. 2012. ACM.

[6]Yu, B., et al., Crowdsourcing participatory evaluation of medical pictograms using Amazon Mechanical Turk. Journal of medical Internet research, 2013. 15(6): p. e108.

[7]Whitla, P., Crowdsourcing and its application in marketing activities. Contemporary Management Research, 2009. 5(1).

[8]Gao, H., et al., Harnessing the crowdsourcing power of social media for disaster relief. 2011, DTIC Document.

[9]Fraternali, P., et al., Putting humans in the loop: Social computing for Water Resources Management. Environmental Modeling & Software, 2012. 37: p. 68-77.

[10]Prpic, J., A. Taeihagh, and J. Melton. A Framework for Policy Crowdsourcing. 2014.

[11]Ali, R., et al., Social adaptation: when software gives users a voice. 2012.

[12]Gebauer, J., Y. Tang, and C. Baimai, User requirements of mobile technology: results from a content analysis of user reviews. Information Systems and e-Business Management, 2008. 6(4): p. 361-384.

[13]Pagano, D. and W. Maalej. User feedback in the appstore: An empirical study. in Requirements Engineering Conference (RE), 2013 21st IEEE International. 2013. IEEE.

[14]Pagano, D. and B. Bruegge. User involvement in software evolution practice: a case study. In Proceedings of the 2013 international conference on Software engineering. 2013. IEEE Press.

[15]Groen, E.C., J. Doerr, and S. Adam, Towards Crowd-Based Requirements Engineering. A Research Preview, in Requirements Engineering: Foundation for Software Quality. 2015, Springer. p. 247-253.

[16]Prpić, J., et al., How to work a crowd: Developing crowd capital through crowdsourcing. Business Horizons, 2015. 58(1): p. 77-85.

[17]Hoon, L., et al. A preliminary analysis of vocabulary in mobile app user reviews. In Proceedings of the 24th Australian Computer-Human Interaction Conference. 2012. ACM.

[18]Hosseini, M., et al. The four pillars of crowdsourcing: A reference model. In Research Challenges in Information Science (RCIS), 2014 IEEE Eighth International Conference on. 2014. IEEE.

[19]Zhao, Y. and Q. Zhu, Evaluation on crowdsourcing research: Current status and future direction. Information Systems Frontiers, 2014. 16(3): p. 417-434.

[20]Ross, J., et al. Who are the crowd workers? Shifting demographics in mechanical Turk. In CHI'10 extended abstracts on Human factors in computing systems. 2010. ACM.

[21]Prahalad, C.K. and V. Ramaswamy, Co-creating unique value with customers. Strategy & leadership, 2004. 32(3): p. 4-9.

[22]Guzman, E. and W. Maalej. How do users like this feature? a fine grained sentiment analysis of app reviews. In Requirements Engineering Conference (RE), 2014 IEEE 22nd International. 2014. IEEE.

[23]Google Play. https://play.google.com/store/apps/.

[24]Platzer, E., Opportunities of automated motive-based user review analysis in the context of mobile app acceptance. Proc. CECIIS, 2011: p. 309-316.

[25]Fu, B, et al. Why people hate your app: Making sense of user feedback in a mobile app store. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 2013. ACM.

[26]Vasa, R., et al. A preliminary analysis of mobile app user reviews. In Proceedings of the 24th Australian Computer-Human Interaction Conference. 2012. ACM.

[27]Finkelstein, A., et al., App Store Analysis: Mining App Stores for Relationships between Customer, Business and Technical Characteristics. RN, 2014. 14: p. 10.

[28]McIlroy, S., et al., Analyzing and automatically labeling the types of user issues that are raised in mobile app reviews. Empirical Software Engineering, 2015: p. 1-40.

[29]Khalid, H. On identifying user complaints of iOS apps. In Software Engineering (ICSE), 2013 35th International Conference on. 2013. IEEE.

[30]Google, Developer Console Help- Fake ratings & reviews. https://support.google.com/googleplay/android-developer/answer/2985810?hl=en.

[31]Iacob, C. and R. Harrison. Retrieving and analyzing mobile apps feature requests from online reviews. In Mining Software Repositories (MSR), 2013 10th IEEE Working Conference on. 2013. IEEE.

[32]Asif, Muhammad, and John Krogstie. "Research Issues in Personalization of Mobile Services." International Journal of Information Engineering and Electronic Business (IJIEEB) 4.4 (2012): 1.