---
title: "Open peer review and its rhythms"
layout: post
image: 
    feature: header_review.png
---
I was lucky enough, recently, to get a slightly-ahead-of-general-release opportunity to openly peer review Kathleen Fitzpatrick's most recent book manuscript, _Leading Generously_. It's over [on HCommons](https://leadinggenerously.hcommons.org/) for those who want to take a look and feed in.

While there's much, content-wise, in the book on which I might remark, I wanted, instead, to take a few minutes to reflect on the process of "open peer review" here and how it led me to act. I also want to draw attention to the ways that this was different to more conventional review modes.

First, it's abundantly clear how many more of the (existing) comments of others are embedded early in the manuscript. That is: it seems, to me, that many more people make remarks early in the book's flow. Perhaps this is _always_ the case with peer review of books and, as reviewers' attentions wander, you get fewer remarks on later portions. This might be either because the reviewers are sold on what you are saying or because they have become tired of reading by that point! That said, because these reviews are unfolding in real-time between different actors, it could be simply because other reviewers were only part-way through and will return. Certainly, I worked through it in two sessions.

Second, seeing the remarks of other commentators altered my flow of reading and thought. There's [good evidence that online paratextual commentaries affect how we read and understand articles themselves](https://doi.org/10.1111/1468-5973.12215). The same goes for open peer-review contexts. I was influenced by the thoughts that others had had, as I read.

Third, because of the commenting format of the manuscript, I read this book at my computer. This is not my normal _modus operandi_. Yes, I work almost entirely digitally. But I would conventionally transfer a paginated PDF to my Remarkable tablet and annotate it there, before returning to my desk to synthesize my notes into a summary review/report (see next point). In this case, I worked at my screen. I do not know what the determinate effects of this may be, except that [backlit screens encourage us to read in F-shaped patterns](https://www.scientificamerican.com/article/reading-paper-screens/). The CommentPress format invites the screen-based reading, because it's so much more convenient for leaving notes...

Which brings me to my fourth point. Conventionally, when I am asked to review book manuscripts, I will make my annotations and then type them up into a narrative report that is supposed to recommend publication or rejection, often with a set of revisions requested. That wasn't what was being asked here. And I also wasn't being asked by an intermediary (a press) to make such a judgement. Instead, the process here felt like one of open annotation, where I could express my thoughts as I read, without being beholden to making a synthesizing decision on my thoughts on the MS as a whole. This was more like an opportunity to "think with" Kathleen through her paragraphs, rather than to judge whether they were wholly correct in my view. It felt like a more generous form of review, because I wasn't being asked to reject at any point. (Although perhaps that curtails my freedom, so it's less generous to my own liberty... But I don't really think that.)

It was also far more of a processual experience. Perhaps I should have read the whole thing and then gone back and commented. But I didn't. I worked through it paragraph by paragraph adding my thoughts as they came to me. This means that, on occasion, the queries I had were addressed a few paragraphs later. But it also meant that I was able to query, through my cumulative questioning, the logical order in which things were constructed. Perhaps I should have been more patient... However, when I keep asking "where is X?" when it hasn't yet appeared, that probably also speaks to the expectations of other readers, too. This type of structural unveiling might tell us something about writing and readership and expectation.

Of course, there's also an element of performance in this type of review work. Everyone will see what I thought and typed. And sometimes, this veered onto dangerous ground. Discussing the problems of academic tenure, while a tenured-equivalent (as close as, in the UK context) academic, is difficult terrain. I know that others will be able to logon and see my comments. This is far more exposing than in other contexts of review, where everything happens behind closed doors: "and thanks to the two anonymous readers at the Press who..."

It's also interesting to see how many people commented and how substantive the feedback is, compared to reader reports on books that have not been through such a process. At the present moment, six external commentators have dived in. Most of the cumulative remarks are not equal, in volume, to what I would hope for from a reader report for a monograph in the conventional process. But some of the remarks are incredibly astute. And some commentators have really gone above and beyond. Of course, volume means nothing when compared to quality (as the book itself states, several times, in its rejection of quantitative metrics in favour of stories and narrative). But there is also a fundamental question of "what makes for a good review?" at work here. As a book series editor, I've seen completely inappropriately short reviews and I've seen reviews that could be used as weapons of assault, if printed. In [our work on the peer review corpus at _PLOS ONE_](https://www.cambridge.org/gb/academic/subjects/literature/printing-and-publishing-history/reading-peer-review-plos-one-and-institutional-change-academia?format=PB&isbn=9781108742702), we found a review that was longer than the paper on which it commented. Was it a good peer review? I'm not sure...

In any case, reading and commenting on this work was enjoyable, interesting, fun, enlightening, depressing (at times, thinking about HE and its strictures), useful, humbling, and thoughtful. I enjoyed it in a way that I don't normally necessarily get from the usual peer review process. Good stuff from Kathleen, but also great in that it has let me think through this process of open review and how it felt, experientially, as I worked through the manuscript in my mind.

## Addenda
A few additional points occurred to me overnight that I thought it worth adding here.

It is important to recognize that this book is not typical of scholarly work. It is written by an extremely prominent/distinguished author and it is in a field that applies to every area of university life. Everybody in every discipline in the university will have encountered "university leadership". This means that this text will probably have received much greater engagement than a more nuanced work ("Church Pews, 1950-1951"). A question then arises as to how far this review process can be taken as a guide and how far it is idiosyncratic/unique. Certainly, other open peer review experiments have required explicit invitations for people to participate.

It's also interesting to note the experiential elements of annotation in conducting this review. For instance, at times I felt that I needed to be careful about what I said, particularly as examples come from one's own experience. At other times, though, I did almost somehow forget the publicness of the performance of review...