BIROn - Birkbeck Institutional Research Online

    How and why do users change their assessment of search results over time?

    Zhitomirsky-Geffet, M. and Bar-Ilan, J. and Levene, Mark (2016) How and why do users change their assessment of search results over time? Proceedings of the Association for Information Science and Technology 52 (1), pp. 1-4. ISSN 2373-9231.

    Full text not available from this repository.


    In this study we investigate whether and why users change their preferences when assessing search engine results over time. We conducted a study with 35 subjects who were asked to rank and assign relevance scores to the same set of search results for three times, with a few weeks period between each round. The subjects were then exposed to the differences in their judgements and were asked to explain them. A new coefficient to measure change was introduced to assess the results of the experiment. We found that all the subjects judge the vast majority of the results differently in every round. However, there was less change in relevance judgements than in rankings. Most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time influences the relevance assessment. Our analysis reveals that the main factors that caused these changes were due to categorical thinking, influence of the learnt information, and environmental and emotional changes.


    Item Type: Article
    Keyword(s) / Subject(s): Ranking, relevance judgements, search engines, change over time, locality, categorical thinking
    School: Birkbeck Schools and Departments > School of Business, Economics & Informatics > Computer Science and Information Systems
    Depositing User: Administrator
    Date Deposited: 15 Dec 2016 10:22
    Last Modified: 15 Dec 2016 10:22


    Activity Overview

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item