Can you de-couple outputs?
- 05 September 2017
- Posted in: Education, Science & Technology
As REF2021 approaches (at the time of writing we are now 39 months away from the expected submission date) we have gone from the certainty of the REF2014 rules, through various reviews and consultations and are now on the cusp of knowing the new rules of the game. And not a moment too soon either - we are also 44 months (51 for impact) - well over halfway into the assessment period.
The Initial decisions on the Research Excellence Framework 2021 cast some light on the proceedings but the big issues are still being consulted upon, namely Questions on staff and output portability with responses being solicited until Sept 29th - perfect timing for this Research Impact: Strengthening the Excellence Framework Open Forum event!
In REF2014, as in previous research assessment exercises, outputs were firmly tied to academic staff, with four required per person - or some reasons needed as to why a lower number were included. One of the Stern Review recommendations was that, as we are assessing departments and not individuals, outputs should be decoupled from individuals and just provided at the departmental (UOA submission) level. This goes hand in hand with not having to select staff (always a divisive issue) for submission, but rather just using the staff FTE to determine how many outputs should be assessed. Surely this is all good? Well, the halfway house proposal so far from the Funding Councils talks about having a minimum (perhaps just one) number of outputs associated with all eligible staff, and a maximum (six was mooted) too.
This partial decoupling is perhaps the worst of all worlds. Having a minimum means that either staff with no outputs (or perhaps one really poor output) will have to be included at a penalty to the submission, or somehow excluded. If staff members are being excluded, how does one exclude them? They would need to become ineligible somehow, perhaps by exception (which would reintroduce the onerous Individual Staff Circumstances used in REF2014 for arguing for a no penalty reduction in the number of outputs an individual needed to submit), but there is the risk of the exception not being approved. Another solution is to somehow render them totally ineligible, the most obvious way being to change their contract of employment to no longer include research - even more divisive than non-selection. And there is of course additional burden in either approach, not to mention the angst for those individual members of academic staff involved.
The maximum appears, initially, to be less of an issue, although one wonders what the rationale for a particular maximum is, why six? In fact, why have one at all? Informal feedback from panels suggested that it was extremely rare for all four outputs for a member of staff last time to be judged as being 4*. How likely is it that more than six would be 4*? And if they are, given that we are decoupling outputs, why shouldn't a department be able to submit them anyway?
And, back to my favourite subject of burden, even if we ignore the additional variables added by jointly authored publications, in order to determine the optimum collection, many more outputs will need to be assessed internally before submission, and to a greater degree. In REF2014, the top four per person needed to be identified - whilst it was not always easy to agree on what each output might be graded as, it was relatively easy to select the best four from someone's portfolio. Now with the option of (perhaps) up to six, not only do more need to be reviewed, but also compared across all the other top six outputs from the other staff in the department – much trickier when trying to compare outputs from different sub areas. This could be a huge burden for departments.
It is not yet too late to resolve some of these burdensome issues, so please do respond to the consultation, but I feel that overall we may not make much progress on that front.
And finally with Impact Case Studies becoming 56% more valuable (up from 16% [80% of 20%] to 25%), I can see a burgeoning market for Research Impact Professionals - now there is an acronym to conjure with...
Article by Simon Kerridge
Simon is a veteran of the past four REF/RAE submissions and was a panel secretary for parts of the 2008 and 2014 exercises. He is Director of Research Services at the University of Kent, chair of CASRAI, and immediate past chair of ARMA.
