BIROn - Birkbeck Institutional Research Online

    A method to assess search engine results

    Bar-Ilan, J. and Levene, Mark (2011) A method to assess search engine results. Online Information Review 35 (6), pp. 854-868. ISSN 1468-4527.

    Full text not available from this repository.


    Purpose – The aim of this paper is to develop a methodology for assessing search results retrieved from different sources. Design/methodology/approach – This is a two phase method, where in the first stage users select and rank the ten best search results from a randomly ordered set. In the second stage they are asked to choose the best pre-ranked result from a set of possibilities. This two-stage method allows users to consider each search result separately (in the first stage) and to express their views on the rankings as a whole, as they were retrieved by the search provider. The method was tested in a user study that compared different country-specific search results of Google and Live Search (now Bing). The users were Israelis and the search results came from six sources: Google Israel,, Google UK, Live Search Israel, Live Search US and Live Search UK. The users evaluated the results of nine pre-selected queries, created their own preferred ranking and picked the best ranking from the six sources. Findings – The results indicate that the group of users in this study preferred their local Google interface, i.e. Google succeeded in its country-specific customisation of search results. Live Search was much less successful in this aspect. Research limitations/implications – Search engines are highly dynamic, thus the findings of the case study have to be viewed cautiously. Originality/value – The main contribution of the paper is a two-phase methodology for comparing and evaluating search results from different sources.


    Item Type: Article
    School: School of Business, Economics & Informatics > Computer Science and Information Systems
    Research Centres and Institutes: Birkbeck Knowledge Lab
    Depositing User: Sarah Hall
    Date Deposited: 31 May 2013 08:00
    Last Modified: 02 Dec 2016 13:23


    Activity Overview

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item