---
title: "Accreditation and research-assessment practices (on DORA and the Tickell review)"
layout: post
categories: [scholarly communications, open access]
tags: [scholarly communications, open access]
published: True
---
Last week, the [Tickell review of open access in the United Kingdom](https://www.gov.uk/government/publications/open-access-to-research-independent-advice) was published. There are no unwelcome nasty surprises in the review and, in fact, there are a number of extremely progressive elements, most notably the formation of a monographs sub-committee to address this increasingly important area of practice.

One of the elements that struck me in the review, though, is that the very first of the recommendations is to "[e]ncourage universities to sign-up to the San Francisco Declaration on Research [Assessment]" ([DORA](http://www.ascb.org/dora/)). The number one recommendation of DORA is: "[d]o not use journal-based metrics [...] as a surrogate measure of the quality of individual research articles, to assess an individual[...]’s contributions, or in hiring, promotion, or funding decisions."

The reasons for this are many but for me there are several key rationales behind it:

1. Every "top" journal has published dud work and any good researcher can choose to publish elsewhere. By giving a list of journals or by ranking journals, we encourage people to ignore the specificity of work while also reifying the status of a "top" journal, even when we are unable to vouch for the anonymous peer community behind each article (we are merging a heterogeneous set of individuals and their expertise into a homogeneous journal title).

2. This reification also fixes economic hierarchies of publishing. Being on an accredited list means that those publishers will fare well as scholars will feel they must submit to such venues. This makes it harder for young journals -- even those with credible peer-review practices -- to break in and contributes to the rise of monopolistic publishers.

3. It actually restricts academic freedom in where to publish when we give lists like this, even if not ranked. Young scholars will have little choice -- if they wish to be evaluated -- but to go to these venues.

And this all comes about because we _do_, collectively as academics, want to be evaluated. Whatever anyone says -- and regardless of how brilliantly kind people are as individuals -- academia is hierarchical. We may wish for a world where this wasn't necessary but, in the world that exists, those wanting posts and promotion want their outputs to be recognised for this, not just for disseminating knowledge. Dissemination _and_ assessment.

In my view, the culture that DORA aims to challenge is one situated in the area of labour time for readers. (I have been deeply influenced in my thinking here by Geoff Bilder.) People seek proxies for quality because they have insufficient time to read everything. So finding short-hands seems a tidy way in which we can accelerate this process. If we have 200 CVs before us on a hiring panel, do we read the work of absolutely everyone who applied? We surely should and we should not be doing a checkbox exercise for publication venue and for publication type. Undoubtedly, when labour time is short, though, panels fall back on such practices.

What we should be doing, however, is using our critical faculties to evaluate beyond the frame of publication, even if this is difficult in current labour climates. We need to explicitly state that legitimate and high-quality publications should be rewarded wherever they are found and within whichever communities and practices that can produce those publications. We can reward communities (journals) and practices (types of peer review) that contribute towards good work. But doing so at the expense of allowing people the freedom to create new communities and new practices through the soft-power exercise of scarcity-finance, mediated by hiring and promotion panels, stifles the academic world.