BIROn - Birkbeck Institutional Research Online

    Reviewing the Reviewers: Training Neural Networks to Read Peer Review Reports

    Eve, Martin Paul and Gadie, Robert and Odeniyi, Victoria and Parvin, Shahina (2022) Reviewing the Reviewers: Training Neural Networks to Read Peer Review Reports. In: Jaillant, Lise (ed.) Archives, Access and Artificial Intelligence: Working with Born-Digital and Digitised Archival Collections. Bielefeld: Bielefeld University Press, pp. 131-156. (In Press)

    Book2_CHAPTER-5.pdf - Published Version of Record
    Available under License Creative Commons Attribution Non-commercial.

    Download (353kB) | Preview


    The study of academic peer review is often difficult owing to the confidentiality of reports. As an occluded genre of writing that nonetheless underpins scientific publication, relatively little is known about the ways that academics write and behave, at scale, in their reviewing practices. In this chapter, we describe for the first time the database of peer review reports at PLOS ONE, the largest scientific journal in the world, to which we had unique access. Specifically, we detail the approach that we took to training a multi-label, multi-class text classifier using the TenCent NeuralClassifier toolkit to examine the peer review reports. Although this resulted in a predictable failure to produce accurate levels of recall and precision, we argue that as these technologies further develop there are a range of uses – for both good and ill – that could be used to machine-read these archives.


    Item Type: Book Section
    School: Birkbeck Faculties and Schools > Faculty of Humanities and Social Sciences > School of Creative Arts, Culture and Communication
    Depositing User: Martin Eve
    Date Deposited: 22 Mar 2021 16:17
    Last Modified: 09 Aug 2023 12:50


    Activity Overview
    6 month trend
    6 month trend

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item