International Journal of Information Technology and Computer Science(IJITCS)

ISSN: 2074-9007 (Print), ISSN: 2074-9015 (Online)

Published By: MECS Press

IJITCS Vol.11, No.5, May. 2019

Factors Affecting Users’ Measure of Interest: A Study of the Effect of Task, Document Difficulty and Document Familiarity

Full Text (PDF, 358KB), PP.47-57

Views:0   Downloads:0


Stephen Akuma, Chrisina Jayne

Index Terms

Document difficulty;Document familiarity;implicit indicators;Information Retrieval;Task Type


Data on the web is constantly growing which may affect users’ ability to find relevant information within a reasonable time limit. Some of the factors previously studied that affect users searching behaviour are task difficulty and topic familiarity. In this paper, we consider a set of implicit feedback parameters to investigate how document difficulty and document familiarity affects users searching behaviour in a task-specific context. An experiment was conducted and data was collected from 77 undergraduate students of Computer science. Users’ implicit features and explicit ratings of document difficulty and familiarity were captured and logged through a plugin in Firefox browser. Implicit feedback parameters were correlated with user ratings for document difficulty and familiarity. The result showed no correlation between implicit feedback parameters and the rating for document familiarity. There was, however, a negative correlation between user mouse activities and document difficulty ratings. 

Also, the dataset of all the participants in the experiment was grouped according to task type and analysed. The result showed that their behaviour varies according to task type. Our findings provide more insight into studying the moderating factors that affect user searching behaviour.

Cite This Paper

Stephen Akuma, Chrisina Jayne, "Factors Affecting Users’ Measure of Interest: A Study of the Effect of Task, Document Difficulty and Document Familiarity", International Journal of Information Technology and Computer Science(IJITCS), Vol.11, No.5, pp.47-57, 2019. DOI: 10.5815/ijitcs.2019.05.06


[1]G. Jawaheer, P. Weller and P. Kostkova, '"Modeling User Preferences in Recommender Systems: A Classification Framework for Explicit and Implicit User Feedback," ACM Transactions on Interactive Intelligent Systems, vol. 4, pp. 1-26.

[2]X. Zhu, J. Huang, B. Zhou, A. Li and Y. Jia, '"Real-time personalized twitter search based on semantic expansion and quality model," Neurocomputing, vol. 254, pp. 1339-1351.

[3]S. Akuma, R. Iqbal, C. Jayne and F. Doctor, '"Comparative analysis of relevance feedback methods based on two user studies," Comput.Hum.Behav., vol. 60, 7, pp. 138-146.

[4]B. Mobasher, R. Cooley and J. Srivastava, '"Automatic Personalization Based on Web Usage Mining," Commun ACM, vol. 43, no. 8, pp. 142-151.

[5]A. Alhindi, U. Kruschwitz, C. Fox and M. Albakour, '"Profile-Based Summarisation for Web Site Navigation," ACM Transactions on Information Systems, vol. 33, no. 1, pp. 1-40.

[6]J. Liu, C. Liu and N. Belkin, '"Examining the effects of task topic familiarity on searchers' behaviors in different task types," Proceedings of the ASIST Annual Meeting, vol. 50, no. 1.

[7]N.J. Belkin, D. Hienert, P. Mayr and C. Shah, '"Data requirements for evaluation of personalization of information retrieval - A position paper," CEUR Workshop Proc., vol. 1866.

[8]S. Akuma, C. Jayne, R. Iqbal and F. Doctor, '"Implicit predictive indicators: Mouse activity and dwell time," IFIP Advances in Information and Communication Technology, vol. 436, September 19-21, pp. 162-171.

[9]D. Kelly and C. Cool, '"The effects of topic familiarity on information search behavior," Proceedings of the ACM International Conference on Digital Libraries, pp. 74-75.

[10]J. Liu, C. Liu, J. Gwizdka and N.J. Belkin, '"Can search systems detect users' task difficulty? Some behavioral signals," SIGIR 2010 Proceedings - 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 845-846.

[11]H. Lee and N. Pang, '"Understanding the effects of task and topical knowledge in the evaluation of websites as information patch," J.Doc., vol. 74, no. 1, pp. 162-186.

[12]S.K. Bhavnani, '"Domain-specific search strategies for the effective retrieval of healthcare and shopping information," Conference on Human Factors in Computing Systems - Proceedings, pp. 610-611.

[13]S.K. Bhavnani, '"Important cognitive components of domain-specific search knowledge,", pp. 571-578.

[14]B. Allen, '"Topic knowledge and online catalog search formulation," Library Quarterly, vol. 61, no. 2, pp. 188-213.

[15]X. Zhang, H.G.B. Anghelescu and X. Yuan, '"Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study," Information Research, vol. 10, no. 2.

[16]Y. Li and N.J. Belkin, '"A faceted approach to conceptualizing tasks in information seeking," Information Processing and Management, vol. 44, no. 6, pp. 1822-1837.

[17]P. Bennett, K. Collins-Thompson, D. Kelly, R. White and Y. Zhang, '"Overview of the Special Issue on Contextual Search and Recommendation," ACM Transactions on Information Systems, vol. 33, no. 1, pp. 1-7.

[18]D.K. Limbu, A.M. Connor, R. Pears and S.G. MacDonell, '"Improving web search using contextual retrieval," ITNG 2009 - 6th International Conference on Information Technology: New Generations, pp. 1329-1334.

[19]M. Busby, '"Learn Google,", 2003.

[20]R. Iqbal, A. Grzywaczewski, J. Halloran, F. Doctor and K. Iqbal, '"Design implications for task-specific search utilities for retrieval and reengineering of code," Enterprise Information Systems, pp. 1751-7575.

[21]A. Crescenzi, R. Capra and J. Arguello, '"Time pressure, user satisfaction and task difficulty," Proceedings of the ASIST Annual Meeting, vol. 50, no. 1.

[22]J. Mao, Y. Liu, K. Zhou, J. Nie, M. Zhang, S. Ma, J. Sun and H. Luo, '"When does Relevance Mean Usefulness and User Satisfaction in Web Search?" SIGIR '16 Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, July 17 -21, 2016, pp. 463-472.

[23]J. Jiang, D. He, D. Kelly and J. Allan, '"Understandingephemeral state of relevance,", March 07-11, 2017, pp. 137-146.

[24]K. Järvelin and P. Ingwersen, '"Information seeking research needs extension toward tasks and technology," Information Research, vol. 10, no. 1.

[25]R.A. Hamid, J.A. Thom and D.N.F.A. Iskandar, '"Effects of relevance criteria and subjective factors on web image searching behaviour," J.Inf.Sci., vol. 43, no. 6, pp. 786-800.