BIROn - Birkbeck Institutional Research Online

    Blocking My Crawl

    Eve, Martin Paul (2023) Blocking My Crawl. ,

    [img] Text
    2023-11-09-blocking-my-crawl.markdown - Published Version of Record
    Available under License Creative Commons Attribution.

    Download (3kB)


    My day job involves quite a lot of crawling lists of websites to determine statistics about Crossref members and their behaviours. A good example is something I wanted to know recently: in the current sample, how many members display the title and doi (according to the latest display guidelines) on that page? In other words, how many members are doing good things on their landing pages? This is also important because, if a high number of these pages are behaving well, then we can use this as a marker of semantic shift/change. That is, we can use it to detect when a new domain owner comes along, for instance, and changes the content so that it no longer reflects the original.


    Item Type: Article
    School: Birkbeck Faculties and Schools > Faculty of Humanities and Social Sciences > School of Creative Arts, Culture and Communication
    Depositing User: Martin Eve
    Date Deposited: 23 Jan 2024 12:33
    Last Modified: 23 Jan 2024 12:33


    Activity Overview
    6 month trend
    6 month trend

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item