Measuring research output

Last night John Denham, the Secretary of State for Innovation, Universities and Skills gave a speech at the RSA in which he called for scientists to engage more fully with policy-makers and the public. He saw this as a key enabler for meeting the challenges of the next century.

This call in itself is not new, but Denham went further in actually considering some of the reasons why the research community are often reluctant to leave their ivory towers.

I think many will welcome this statement but will want the minister to look further at how the RAE has affected other parts of academic culture.

The RAE has been a reality of academic life for the last 20 years and few would doubt it has had a positive impact on the quality of academic output in the UK – but it has also narrowed the focus of what it means to be a researcher, particularly in science and technology subjects where as Denham concedes, the system is weighted towards publishing as much as possible in high impact-factor journals.

Another area that may suffer from the pressure of the RAE is interdisciplinary and risky/adventurous research. The argument goes that such research will not be published in traditional high impact-factor journals and that it is more difficult and takes longer, reducing published output.

As part of the consultation mentioned by Denham, HEFCE published a piece of statistical research by Evidence Ltd which indicated that using publication metrics (relative numbers of citations) to assess the quality of research will not disadvantage interdisciplinary research, as has been widely believed.

This is interesting, but in many ways does not solve the problem as there is still a widely held perception that such research is at a disadvantage. Whilst the perception exists, some department heads will still steer their younger colleagues into ‘safe’ well established areas of research.

And before we jump to condemn such behaviour it worth remembering that the RAE holds the key to significant amounts of funding which can make or break an academic department.

Going back to John Denham’s original concerns regarding policy and public engagement, I wonder if he is opening a wider debate on exactly what should be the role of the academic and indeed academic institutions in society? Can we develop an academically excellent but more innovative and outward looking ‘Academy’ with links between disciplines and to industry, policy makers and the wider public?

Our present system, whilst paying lip service to all these wider relationships does not provide the right structure of incentives. This is the challenge for HEFCE in planning its new model to assess research.