Bar-Ilan, J. and Levene, Mark and Mat-Hassan, M. (2006) Methods for evaluating dynamic changes in search engine rankings: a case study. Journal of Documentation 62 (6), pp. 708-729. ISSN 0022-0418.
Abstract
Purpose The objective of this paper is to characterize the changes in the rankings of the top ten results of major search engines over time and to compare the rankings between these engines. Design/methodology/approach The papers compare rankings of the top‐ten results of the search engines Google and AlltheWeb on ten identical queries over a period of three weeks. Only the top‐ten results were considered, since users do not normally inspect more than the first results page returned by a search engine. The experiment was repeated twice, in October 2003 and in January 2004, in order to assess changes to the top‐ten results of some of the queries during the three months interval. In order to assess the changes in the rankings, three measures were computed for each data collection point and each search engine. Findings The findings in this paper show that the rankings of AlltheWeb were highly stable over each period, while the rankings of Google underwent constant yet minor changes, with occasional major ones. Changes over time can be explained by the dynamic nature of the web or by fluctuations in the search engines' indexes. The top‐ten results of the two search engines had surprisingly low overlap. With such small overlap, the task of comparing the rankings of the two engines becomes extremely challenging. Originality/value The paper shows that because of the abundance of information on the web, ranking search results is of extreme importance. The paper compares several measures for computing the similarity between rankings of search tools, and shows that none of the measures is fully satisfactory as a standalone measure. It also demonstrates the apparent differences in the ranking algorithms of two widely used search engines.
Metadata
| Item Type: | Article |
|---|---|
| School: | Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences |
| Depositing User: | Sarah Hall |
| Date Deposited: | 25 May 2021 19:00 |
| Last Modified: | 09 Aug 2023 12:50 |
| URI: | https://eprints.bbk.ac.uk/id/eprint/44428 |
Statistics
Additional statistics are available via IRStats2.
Archive Staff Only (login required)
