Oral evidence: Research integrity, HC 350

Oral evidence, 30 January 2018.

Witnesses

Professor Sir Bernard Silverman and James Parry.

Chair

Welcome. It is very good to see you both. Would you like to introduce yourselves briefly?

Professor Silverman

I am Bernard Silverman. I am the chair of the UK Research Integrity Office.

James Parry

Good morning. I am James Parry. I am the chief executive of the UK Research Integrity Office.

Chair

Brilliant. All of us will ask you questions. We are under quite tight time constraints, so will you try to keep your answers as tight as possible? Do not feel that both of you have to answer everything. The UKRIO describes itself as a provider of support and advice to the sector on matters of research integrity. Do you provide that support passively on request, or do you have a leadership role in proactively trying to drive change in practice?

Professor Silverman

Both. We initiate projects to provide generic advice and discuss ways in which we should do that. For example, one important event is our annual conference. We go to a lot of trouble to decide which topics people would like to know about and invite senior figures to speak about them. In that way, we inculcate a good culture. Of course, we also offer a confidential advisory and support service; James can give more details of its use. That provides what I would call reactive advice, so we are both active and passive.

Chair

Do you want to add anything, James?

James Parry

There is also a crossover. For example, we have benchmark publications—a code of practice for research and a procedure for the investigation of misconduct in research—which are out there for anyone to use. We also provide tailored support to help institutions to embed those into their systems and practices, both in the generic sense—for example, by establishing or revising a procedure for investigating alleged research fraud—and by helping with practical advice during investigations. There is a lot of crossover in those elements.

Chair

You are funded in part by universities. Sixty-five universities are subscribers.

Professor Silverman

Our only source of funding is the subscriptions that we get. Each of our subscribers pays a modest £2,600 a year. They include not just universities but other kinds of research institutes and so on. The majority are UK universities.

Chair

My sense, as an ex-lawyer, is that that provides something of a conflict of interest. There is no problem in providing advice and support, but if you are seeking to achieve higher standards and to play a leadership role does it not compromise you in demanding change, calling out bad practice, naming and shaming, or whatever may be necessary to drive a change in culture? Is that an issue?

Professor Silverman

I do not think so.

Chair

Why not?

Professor Silverman

We work hard to make sure that there are not conflicts of interest.

Chair

If your survival depends on payments by universities, that inevitably has an impact, does it not?

James Parry

Because pretty much everyone pays a flat-rate subscription, we are not beholden to any one institution. If a university says, “We don’t like the way you are handling this particular case,” we can simply say, “That is all very well and good. We are happy for you to unsubscribe.” We do not tell our subscribers whom we are helping or how we are helping them. If someone comes to us from a university that is also a subscriber, we will declare to that person, “Just so you know, these people are one of our many funders. We will not share any confidential information with your institution about your inquiry.” The institutions know that and do not press.

Chair

You are described as the Research Integrity Office. Do you have any official status?

Professor Silverman

I do not think so. That is the title. We are probably called that for historic reasons.

James Parry

Yes. It mimics the titles of institutions in other countries.

Chair

I think that I know the answer to this question from your earlier answer. What support, financial or otherwise, do you currently receive from Government for your work?

Professor Silverman

None. We are independent of Government in that respect.

Chair

It is interesting that you are independent of Government, but not independent of the institutions that are doing the work that might be subject to concern.

Professor Silverman

If Government would care to give us a large subvention, we would not have to charge subscribers.

Chair

That begs a further question. Do you think that they should, so that you would be independent of the institutions that are doing the work that you are monitoring and where you are seeking to improve standards?

Professor Silverman

I would prefer to have a diversified source of funding. The levels of funding are so modest that I do not think that we should worry too much about it. My main role, as chair of the trustees, is to ensure the financial stability of the charity, of course. I am happy at the moment, because we have a lot of different subscribers. The numbers are going up all the time. We are diversifying in international coverage and so on. I do not worry where the money comes from, as long as we are transparent about it.

Chair

You would not suggest that Government should provide funding support, to enable you to be independent.

Professor Silverman

We would be delighted, because then we would not have to spend time collecting subscriptions and so on. If Government would care to give us a subvention, we would be delighted. Of course, there is always the risk that they would take it away again. In a way, having a subscriber base gives longer-term stability. We do not see this as a big issue.

Vicky Ford

All that I wanted to hear was whether it was an issue.

Professor Silverman

No.

Chair

Do you wish to comment, James?

James Parry

Bernard has dealt with the point very well.

Bill Grant

My question is about how you measure your performance. How do you establish whether you are doing a good job? Are you subject to any formal evaluation by an external organisation? Has anybody looked at what you are doing? Has anybody measured your performance—hopefully, with a good outcome?

Professor Silverman

I do not think that we have had an external measure of our performance. Would you like to tell us about that, James?

James Parry

During our pilot phase, from 2006 to 2010, we had external evaluations, because at that stage we were hosted by Universities UK and funded by the higher education funding councils, the Royal Society and other bodies. The pilot was assessed and found to be a success. At that point, we spun out, became operationally independent from Universities UK in legal governance and became a charity. We have not had external evaluations since then. The metrics we look at are growth in the use of our advisory service, which has happened year on year, growth in the number of subscribers, which has increased every year, and growth in the number of proactive approaches by institutions to try to get things right, which I think is most important. I refer not to crisis management, but to institutions asking, “How do we make sure that this particular project or type of research goes well? How do we make sure that, if we have to investigate anything, it runs smoothly? How do we make sure that our systems for ensuring integrity and reviewing research ethics and research governance go well?” That category of request for support has grown every year, as has the number of requests for training and development.

Bill Grant

Are you using the expansion and growth in the number of subscribers as a measure of success and as a good indicator?

James Parry

It is a measure—personally, I do not think that it is the most important measure. For our financial stability, it is useful. It shows that institutions are more willing to engage with us, because we give more long-term, in-depth support. I would say that the main metrics are the increasing use of the advisory service and, in particular, the number of proactive requests for support by institutions. That has grown as well.

Bill Grant

Are you indicating that you are evolving to become a trusted organisation?

James Parry

I would like to think so. Quite frankly, when we moved to the subscription model of funding in 2010—Bernard was not there at the time—it was a bit of a leap into the unknown. We had never done it before. Within two years, we had 24 universities signed up. Two years later, it was just under 50, from memory. Now we have 79, including non-universities and universities outside the UK. Therefore, there is an element of being a trusted brand.

Bill Grant

That is looking to external activities. Do you have any internal targets or measures you report against? Do you have any internal reporting?

James Parry

Yes. We have our internal appraisals and the like.

Professor Silverman

The chief executive reports to me. The board of trustees and I are the people to whom internal reporting is done. What I am particularly pleased about as regards performance is the diversity. I must clarify that one of our subscribers is a Government organisation. Is that right, James?

James Parry

Yes—the Ministry of Defence Research Ethics Committee.

Professor Silverman

The Ministry of Defence Research Ethics Committee is one of our subscribers. I apologise for not making that clear earlier.

Bill Grant

The majority of UK universities are not subscribers to your organisation. I sense that you must find that disappointing. Are you minded to change that? If so, what efforts will you make to change it? Finally, what reasons do those who do not subscribe suggest to you for not wanting to engage with you? That may be quite a complex question.

Professor Silverman

One does not like to distinguish all the time, but the majority of strong research universities are members. Only a very small number are not. Those are the ones we make proactive efforts to recruit.

Chair

Are all Russell Group universities members?

James Parry

Twenty out of the 24 are members.

Professor Silverman

The vast majority of the Russell Group are members. If we had a request for help from a non-member university, we would encourage it to become a subscriber. We do not restrict our advice to subscribers, but we encourage universities to become subscribers. Our numbers are growing all the time. One of James’s targets each year from me is to increase the number of subscribers. He more than meets that each time.

Bill Grant

Having captured the subscribers, have you lost any? Has any university failed to renew its subscription, and given you a reason for why it has not renewed?

James Parry

Over the years, three dropped out, citing financial constraints. Two of those three have since re-subscribed, one in less than a year and one about two years later.

Bill Grant

You can take some heart from that. They left and returned.

James Parry

Yes. Often the people doing the practical work on issues of research integrity in universities are not the people who make decisions about whether a university makes a corporate subscription. They are going to other budget holders and making a request.

Bill Grant

Finally, I have one question on an important aspect of research. The Research Integrity Office provides model research misconduct processes. How many universities use those model processes?

James Parry

It has been a few years since we did a survey. A few years ago, we surveyed 75 institutions. Over 50 were making use in some way of our research misconduct procedure and our code of practice for research, which are our core publications. Those have also been endorsed by funding bodies. They do not necessarily mandate their adoption, but they recommend them. Last year, for example, Research Councils UK said that a couple of provisions in our research misconduct procedure—for example, having an external member on a formal investigation panel for research misconduct—must be adopted by institutions as a condition of their grants.

Chair

In your view, how many of your subscriber organisations meet the requirement to have an external member? Do you think that it is uniform among your subscriber members?

James Parry

I think that the vast majority meet it. Both of these documents first came out about 10 years ago. Since they were published, we have worked with institutions to adapt them and to embed them in their procedures. For example, we always recommend that, if you have a formal investigation panel, you should have at least one external member on it. You should also have the option of the chair of the panel being an external member—for example, if the university is dealing with an allegation concerning research that has been the subject of much public debate and concern. We also recommend that, in complex inquiries, you should have provision for getting an external member in the screening stage, too.

Professor Silverman

We cannot monitor every single inquiry to discover whether they are using our procedures, but by being a trusted organisation and having published procedures we hope that we give a model for the institutions themselves.

Vicky Ford

At previous hearings, one of the reasons given for why research is sometimes misleading was that sometimes the researchers themselves do not have a clear enough understanding of statistics. Do you agree? Could you help to provide training in that? How do we embed that understanding of statistics in our scientists?

Professor Silverman

As you may know, I am a statistician myself.

Vicky Ford

That may be why I asked the question.

Professor Silverman

I read the transcripts of those sessions. I thought that what was said there—particularly by David Hand, who is a very old and close friend of mine—was absolutely to the point. I do not think that it is the job of the UK Research Integrity Office to provide training in statistics. I would look to the Royal Statistical Society and places like that.

Chair

Do you think that there is a need for it?

Professor Silverman

In my capacity as a former president of the Royal Statistical Society, yes, I do, but I do not think that it is particularly a matter for the Research Integrity Office. In today’s capacity, I would not offer an opinion.

Vicky Ford

But you would not disagree with them.

Professor Silverman

I would not disagree with Professor Hand. I am speaking in a personal capacity, not on behalf of the UKRIO.

Carol Monaghan

I am slightly concerned that, because an institution does not subscribe to the UKRIO, its research is somehow downgraded and not considered to be as solid and acceptable as that of other places. Is there a danger of that? If so, are institutions almost being held to ransom over joining?

James Parry

While we are a trusted brand, I do not think that we are that much of a trusted brand. Funders and publishers neither mandate that institutions subscribe to us nor check whether an institution is a subscriber when research grants are applied for and research is submitted to publishers for publication. I do not think that there is a two-tier system in that sense. I would like to think that we are seen as worth subscribing to because we provide practical help, not because, if you do not, you will be considered somehow substandard.

Carol Monaghan

So you are a carrot, rather than a stick.

James Parry

I would like to think that. When it comes to research integrity and research culture in general, it is easier to build sticks into systems, rather than carrots. We should look more at carrots in general terms.

Professor Silverman

We publish our list of subscribers, don’t we?

James Parry

On our website.

Professor Silverman

If you wanted to use membership of our organisation as a stick, you could. That would please the trustees, in the sense that our financial situation would be better. To be serious, we would not be very pleased to be used in that way.

Carol Monaghan

UKRIO has highlighted to us that it about to embark on a research project on research integrity. Will you tell us a bit about the timescale and methodology for that and the reasons behind it? Do you think that any recommendations will be made off the back of it?

Professor Silverman

It is a fairly small-scale project. It is research into research misconduct, because we feel that there is not a clear enough picture of what research misconduct there actually is. We know what types of things one should worry about, but it would be very interesting to get an up-to-date picture of how much of each problem there is. It is a fairly small-scale project, using existing sources of information that are either in the public domain or can be obtained by making inquiries, if necessary, through FOIs, to universities and so on. We reckon that it will take about six months. It will be funded by ourselves. We are scoping the exact details at the moment. Our output will be a high-level summary and discussion of conclusions. Mark Walport says that there are three main areas of concern: research misconduct, questionable practices and sloppiness. It will be interesting to see how much of each of those things there is. You ask about recommendations. We are not seeking evidence for recommendations that we have already formulated, but we intend that recommendations will come from this. When we have the research, we will look at it and make recommendations.

Carol Monaghan

Can we expect the recommendations in the next six months, or will they be on a longer timescale?

Professor Silverman

They will be made when the research is done; I would rather do it in that way. I think that we will be able to move to that quite quickly. It will also help to shape our own future work.

Chair

Do you go into this project with any view about the scale of the problem?

Professor Silverman

The evidence is that the problem is not vast. There is not a big problem, but the issue is important and, therefore, always needs attention. In other words, as Mark Walport says, there is a small amount of genuine research misconduct. There will be a larger shadow of sloppiness and so on. I do not think that there is any more of a problem than there ever has been. I am very glad that the research community is taking issues of integrity seriously.

Graham Stringer

That is an assertion without any evidence, isn’t it?

Professor Silverman

Retraction Watch had some interesting evidence in a previous session. It looks at the number of retractions per billion dollars spent on research. The absolute level is quite low. That is within a culture where we are now much more likely to retract bad research than we were in the past. That is a piece of evidence. I do not have numerical evidence about sloppiness, because it is a very difficult thing to quantify, but I read the journals in my own field and others. In general, the papers that you read are produced to a good standard, but inevitably there are some that are not like that. But you are quite right, Mr Stringer; I agree with you.

Chair

There is a POSTnote on this whole issue. It refers to a 2009 study in which researchers were asked for their private view on the issues. It says that “2% said they had falsified, fabricated or altered data”—this is international, incidentally—and that “34% admitted” some “questionable research practices.” That is quite significant.

Professor Silverman

Yes, it is. Of course, some “questionable research practices” may be once they had done something that they subsequently regretted—but, yes, I agree.

Chair

The slight worry is that university after university returns nil inquiries each year. There is a sense, if there is any truth in that, that there is an issue that remains covered up—that the misconduct and questionable practices that are happening are not getting exposed. Is that your view?

James Parry

The 2009 study you are referring to was by Daniele Fanelli, who was then at the University of Edinburgh. The meta-analysis was based on existing studies. One of the reasons we want to do this piece of research is that a lot of those studies are either high-level international ones or based on data from the United States, so we are not sure that we have a true national [i.e. UK] picture. I definitely agree that the numbers quoted by Fanelli are too high and that things need to be done. If an institution reported year after year that it never had any allegations, I would be somewhat sceptical about whether that figure was accurate. At a meeting of the Royal Society a couple of years ago, a panel I was on was asked, “What is the right number of allegations for an institution to have per year?” The answer is, “However many are honestly reported.” If the number is zero consistently, you may need to look at your practices and overall research culture. What I also find interesting about the Fanelli study is the level of questionable research practices. It is vital that institutions—and funders and publishers; it is not just the universities—address research misconduct robustly, fairly and as transparently as they can, under employment law.

Chair

Do you think that more needs to be done?

James Parry

I think that more needs to be done by everybody. Based on the inquiries that come to us and studies like Fanelli’s in 2009, questionable research practices are an area of concern, because they are more common and, perhaps, more insidious. It is worth noting that the research integrity concordat asks for data on “formal investigations” in the annual statements. That does not mean investigation that has been undertaken under a formal process—it means the second stage of an investigation, where they have assessed an issue, to see whether there is a case to answer, and decided that there is some evidence and there may have been misconduct. The issue then goes to a formal panel. Historically, that has been the level of information that has been asked for by funders, although last year Research Councils UK changed it. It said that—along with the Wellcome Trust, I believe—it wants to know about allegations that are being screened. It will get better reporting in that way.

Professor Silverman

Every university ought to publish research integrity statements that are easily accessible. We have tried to find all of them, and not all universities do it. It would be good if all universities did that.

Chair

And it should be a requirement.

Professor Silverman

We cannot make—

Chair

Whether you can or cannot, do you think that it should be?

Professor Silverman

Speaking personally, yes, I do.

Carol Monaghan

Would you consider publishing an annual report on the state of research integrity? What might that cover?

James Parry

As a charity and advisory body, we can publish the data on the concerns that people bring to us. We would be wary of saying that that is utterly reflective of the UK, because there are people who may decide not to access our support. We can certainly publish that sort of information. I have talked about institutions using our services, but just under 50% of our work is with individual researchers and members of the public. We want to make sure that they feel that they can come to us and that their concerns are confidential. We would not want to release data where someone could say, “Oh, that must be about my inquiry,” but we can certainly look at publishing a degree of information.

Professor Silverman

The research project that we are doing now will be a model for the kinds of things that we should do. We will take that one away. Obviously, one would produce summary data, not individual data. We would not say, “The University of X did this, that and the other.” However, that is a very good recommendation. It is one that we will take away.

Carol Monaghan

Do you feel that the frequency with which your advice has been sought is increasing? Are you collecting data on that to measure it?

James Parry

The use of our services is growing year on year. Like other things, such as the rate of retractions, that may be simply because there is more awareness of these issues and of us. For example, after the publication of the concordat in 2012, we got a lot more attention, because we were name-checked in that document and because it brought research integrity afresh to those at a very high level of institutions.

Professor Silverman

I do not think that there is more of a problem, but we are delighted that our services are being used more.

Carol Monaghan

It is more about awareness.

Professor Silverman

Indeed.

Carol Monaghan

Does the word “pauchling” mean anything to you? Maybe that is a Scottish word.

Chair

You have flummoxed us all.

Carol Monaghan

I am glad we have got that on the parliamentary record. “Pauchling” is cooking the books, often in a scientific manner. You cover all sections of research integrity, including the private sector. Do you detect any differences in the approach or attitude towards research integrity—or, possibly, the frequency of problems—between the two sectors? Where do you think that most of your work is required?

James Parry

We cover any research that is done under the auspices of the UK. That includes research in any sector and discipline, whether it is done by an individual or by an organisation. Historically, the private sector has been a relatively low user of our services.

Chair

Do no private sector organisations subscribe to you?

James Parry

We have one private sector subscriber—the RAND Corporation, which joined recently. We do not have any other private sector subscribers. However, we hear from the private sector on some occasions. When institutions and individuals from the university sector come to us, it may be about a project where they are partnering with the national health service, the private sector or a charity.

Carol Monaghan

Does it concern you that the private sector does not engage to the same extent as the academic sector?

James Parry

We would welcome it if it engaged more.

Carol Monaghan

But it has not happened so far.

James Parry

That is correct.

Martin Whitfield

May we explore the concordat a little more? You say that it was signed in 2012 and that you were name-checked in it, but you are not actually a signatory to it, are you?

James Parry

That is correct. We wanted to maintain our independence.

Martin Whitfield

That is to maintain your independence from the concordat. Do you press or advocate the concordat? Do you inquire of your signatories and other users about their involvement with the concordat, or does it sit separately from you?

Professor Silverman

The concordat shapes all our activities. Everything that we do is within the principles of the concordat. We want to help it to work out and to do things that will support it. The matter that I just mentioned—publishing research integrity statements—is an example of something that I would press a little, perhaps even through this inquiry.

Martin Whitfield

For all intents and purposes, you advocate all the views of the concordat as being a positive way forward, both for you as an organisation and your signatories, and for everyone who is in the field of research, across the board. Whose responsibility do you think it is that the concordat is implemented? Where does the buck stop?

Professor Silverman

Within the concordat, there are places where the buck stops. It places obligations on institutions, funders and so on. What we have here is a complicated system with lots of moving parts. The concordat provides a framework for that.

Martin Whitfield

Who do you think should take responsibility for the concordat to be implemented? I refer not just to its terms, but to making sure that people who are signed up to it—

Professor Silverman

Do you mean the overview of the concordat?

Martin Whitfield

Yes.

Professor Silverman

At the moment, it is like morality and ethics in general, I suppose. There is no overarching body. Is that fair?

James Parry

As is the case in every country, the system relies on self-regulation by researchers. On top of that, you have the universities’ responsibility to make sure that researchers are aware of their responsibility to self-regulate, to support them in doing it and to take action if they do not. The funders are there to monitor the universities’ progress. We see our role as being to help institutions and individuals to implement the concordat and to shy away from trying to tick the boxes for the sake of contractual compliance. A lot of work that we do with institutions and individuals is to help them to dial down this document, which is quite high level and overarching, and is meant to cover all types of research in the UK, so that they can make practical changes, using the document and other good-practice examples, to make things better in their institution. For example, we have a self-assessment tool for the concordat, which includes guidance on how to write an annual statement. We are doing another guidance note on that topic, which should be out later this year, hopefully.

Martin Whitfield

I am sorry to interrupt you; I would like to come back to your toolkit in a moment. Do you think that you could take the moral high ground and be the overseers of the concordat’s implementation? It seems to fit very much with your ethos of independence and being removed from the universities, as we discussed in relation to funding, and you recognise its importance. I am not asking you to make a decision today.

Professor Silverman

We see ourselves as overseeing it informally, in the sense that it shapes our activities. I do not think that we would seek a formal status, because that would make it harder for us to be an adviser and a supporter at the same time. It is about being a formal regulator—that is the word that we use—which would not sit comfortably with the way in which we like to work.

Martin Whitfield

But you are content to continue to advocate the importance of the concordat.

Professor Silverman

Yes. Although we would not necessarily advocate there being a formal regulator, we would work within whatever system there was.

Martin Whitfield

We have discussed the potential for an annual report. Do you see your role extending to an annual review of where the concordat sits within that in relation to research within the UK?

Professor Silverman

The concordat would be the jumping-off point for any annual report.

Martin Whitfield

That is useful. You mentioned the self-assessment toolkit. It is used by your funders—by the signatories to it—but it is not available to everyone.

Professor Silverman

It is.

James Parry

We made it available recently.

Professor Silverman

It is now available to everyone.

Martin Whitfield

So, as well as having an offer of annual oversight, we have the tool to ascertain how well the concordat is being followed.

James Parry

It is a tool. I should point out that it is not endorsed by the funders. We developed it separately, looking at the themes that we feel cut across the concordat’s commitments and offering suggestions on how you can translate them into practical terms.

Martin Whitfield

Effectively, you are acting as the interface between the concordat and its users. You are prepared—quite sensibly, because you are where the knowledge rests—to give advice, understanding and implementation to it. That is very positive. Thank you.

Chair

It has been in place for some years now. You have identified one area—annual reporting, as we have described—where you think that it could be tightened. Do you think that it is time for it to be refreshed? You described it as “high level.” Do you think that it needs to be clearer in any areas about the universities’ obligations? It includes recommendations about ways in which it can be complied with. Do we need to look at it again?

James Parry

As you will be aware, Universities UK did a review of the concordat in 2016. We canvassed our subscribers extensively and then presented anonymised views. One thing that came out was that the concordat is designed to promote research integrity but is also an element of contractual compliance with research funders. Therefore, slightly improved clarity from funders on what their expectations are and how they are monitoring them would be helpful. It is good that Research Councils UK, for example, publishes a summary of its compliance data from the institutions. It would be useful to look at those datasets from other funders as well. The concordat approach has garnered a lot of international interest. The Republic of Ireland set up its own research integrity concordat model based on the UK system. Our international subscribers, which include institutions from Ireland and elsewhere, have found our self-assessment tool for the concordat useful as a way of measuring their own research integrity provision. There are many routes that a country can take to ensure integrity in its research. The UK’s is getting a lot of attention from other countries.

Chair

The point is not to argue that it is not the right approach, but to argue whether it can be improved, essentially.

James Parry

Yes. Every country has adopted slightly different variations. Some have gone down the very heavy regulatory route, but often those regulators regulate only certain elements of research integrity, certain funding streams or certain disciplines. Others, such as the UK, have an advisory approach. In Germany, the organisation is an ombudsman that regulates disputes between academics. Serious fraud is investigated by the institution. There are a variety of approaches out there. In the global research integrity community—that is probably giving it a grander title than it deserves—everyone is very keen to learn from everyone else. I have found that to be the case within the UK as well. A lot of what we do is sharing good practice. If an institution comes to us for advice, we will give advice as best we can, but we will also say, “We are aware of other institutions that have grappled with this issue. We will put you in contact with them.” There has not had to be regulation to make sure that institutions talk to one another, share lessons learned and work together to improve things.

Professor Silverman

I come back to the question. We do not think that it is necessary to revisit the concordat itself. It is more important to continue to make sure that it is embedded, in the ways we have discussed.

Damien Moore

Good morning. You touched on other countries in your last answer. What assessment have you made of the merits of other countries’ models, such as those of Denmark, Australia and the United States, for regulating and responding to research misconduct? You have mentioned a couple of countries, but have you looked at misconduct regulation there?

Professor Silverman

Most of the regulation in other countries is regulation of investigations. I have looked at the American model. One thing that we have to understand is that culture is different in different places. The American model can be a bit legalistic. There are very strong sanctions within the American model, but it is mostly about regulation of the way in which formal investigations are carried out, once they take place. Some aspects of different international ways of working are relevant to us, and others are not. In general, the larger issue of things like how to encourage a good research culture, training and how to deal with things informally is not specifically regulated elsewhere. That is the vast majority of the work in this area, anyway.

Damien Moore

In your view, should there be a change in the processes and structures for tackling research misconduct, through external oversight or regulation?

Professor Silverman

I do not think so. Our view is that it is the responsibility of employers. In the end, I suppose, the legal responsibility rests with the employer. What we do is provide templates of good practice for universities and other research employers to use in the unfortunate event that there is a formal investigation.

Damien Moore

What are the pros and cons from a legislative perspective of setting up an independent body to handle this instead?

Professor Silverman

I am not a lawyer. My concern with it is that it becomes an excessively bureaucratic process. We are agnostic, I suppose, on the way in which serious investigations are dealt with, but I think that it is not straightforward to set up such a body. If the decision were made to do so, we would, of course, be ready and willing to give advice on how we felt it could best be done. I think that there are a lot of difficulties in setting up a formal body of that kind, but we do not have a strong view on that.

Damien Moore

Do you find that a whistleblower would turn to you? Have they turned to you, if they are not happy with a university’s investigation process?

James Parry

Yes. We give advice to people with concerns about research misconduct, to people who are witnesses in cases and to respondents. We have found that we can effect change. I remember that in our very early years, 2006 to 2007, we found that institutions that were hesitant to take action—these days it is very different—could be prompted by a complainant writing back to them saying, “I think that you should look into this matter. I have contacted the UK Research Integrity Office, anonymously and in confidence, and am responding to advice received.” The idea is that a knowing third party prompted people. These days institutions and publishers—which deal with allegations as well—recognise that it is part of good governance to deal with these matters properly. They cannot keep these things behind closed doors any more, as they may have wanted to do 10, 20, 30 or 40 years ago. Things get into the public domain. Both good practice and a degree of enlightened self-interest are driving this. People think, “If we don’t deal with this properly, it will come out sooner or later.”

Damien Moore

You never find that there is a conflict of interest, considering that universities are your main funders. This is a decision that you take.

James Parry

I go back to what was said earlier. If someone within our organisation had a personal conflict of interest, obviously we would not involve them. For example, Bernard had a position at Oxford, so we would not involve him in a matter relating to Oxford. However, we simply do not tell the funders—the people who subscribe to us—about inquiries relating to their own institution. We keep that confidential. Given the way in which our funding model is set up, if any funder ever kicked off, we could simply say, “Fair enough. You do not have to subscribe to us any more.”

Damien Moore

If the Government were to introduce external regulation in some form, would you see the future for your institution being the same as it is now, or would you question your own existence?

Professor Silverman

We would probably have a stronger future, because we would be providing advice within that structure. That is why we are agnostic, in a sense. We do not think that we should be the people formally to supervise things; we hope that the universities will do it for themselves. If Government felt that that should be the way, we would work with that, too.

James Parry

We have a body of evidence to inform the creation of a regulator, if that is felt to be desirable.

Graham Stringer

Is not the real regulator of science replication? The problem is that we do not know what is going on because people are trying to replicate science less and less. That has become a culture. People want to break ground, to get high impact in journals. They do not do what is really the slogging work of science, which is to try to repeat the experiments.

Professor Silverman

That is interesting. In my view, it is actually the other way around. My view is that we now have so much research going on that, within every project, you are replicating a lot of what has gone on before. In my own field, it is now considered standard that, if you produce a new statistical technique or work on some data, you make your technique publicly available in detail, so that your computer programme and so on is all in the public domain. You also put your data in the public domain, as far as you can. When I started in research 40 years ago, that was not the general practice at all. Especially with the internet—the web—it becomes easier to replicate things. I completely agree about replication and continuously revisiting things being the way in which science is actually regulated, but I do not think that there is less of it than there used to be; in fact, I think there is more. One of the reasons why Retraction Watch has seen more retractions may be that replicability is now more accessible. This is a matter of opinion that we could discuss for a long time.

Graham Stringer

That is interesting. Staying with the culture of research, a lot of the written evidence that has come in has said that the problem of research is the culture of science—that scientists are looking for high impact and have short-term funding. Putting fraud to one side, where there are mistakes and shoddiness, are not you dealing with the symptoms, rather than the causes?

Professor Silverman

There is undoubtedly a very competitive culture. Actually, there always has been. There are cases 200 years ago of people stealing one another’s work because of the competitive culture. There is tension between pursuing a co-operative enterprise of trying to uncover truth, knowledge and so on, and doing so in a competitive way. I do not think that that necessarily leads to sloppy practices, because it means that people are always looking over one another’s shoulders. Are we addressing the causes rather than the symptoms? The underlying cause is that there is always a small number of people who, unfortunately, will try to get around the system and do things in the wrong way. That is true in every aspect of human life. We have to understand that. We are doing our best to encourage people not to be like that.

Graham Stringer

We have HEFCE and Research Councils UK next. Do you think that they are doing enough to improve the culture and integrity of science?

Professor Silverman

They are very—

Graham Stringer

What questions would you ask them? Can you do our job for us?

Professor Silverman

Yes; exactly. I would ask RCUK and HEFCE whether they feel that they are expecting students and researchers to be sufficiently informed and to put integrity sufficiently at the front of their minds. Is there more that could be done to ensure, for example, that people who do PhDs have proper training in this area? That is one thing. I said earlier that maybe HEFCE should encourage or require universities to publish research integrity statements prominently. Maybe it should audit them to make sure that they are not just lip service. I believe that HEFCE and RCUK take this seriously. In constructing the new REF guidelines, they have tried to produce guidelines that genuinely encourage co-operative research and the assessment of institutions and groups, rather than just individuals, and that ask people to document impact and so on. Those are all positive steps in this regard.

Stephen Metcalfe

You mentioned PhD training. Two questions spring from that. First, do you think that that should be formal training, with formal recognition—a qualification to research, if you like? Secondly, I go back to the point about statistics. Do all research projects use statistics? If so, should that form part of the formal qualification to conduct a PhD research project?

Professor Silverman

First, do I think that there should be a formal qualification for PhD students in research integrity? This is not the considered view of the UKRIO, but I think that it would be a good thing to work towards. It might need to be tailored somewhat to the subject area and might be a matter of accrediting training that universities are providing, but I would say that it would be a good thing. After all, a PhD is not just doing research—it is a training in research. I think that some formal work on research integrity would be very worth while, just as university lecturers nowadays get formal training in teaching. I had that when I became a university lecturer, but I can tell you that not all my colleagues did. You ask about formal training in statistics. Again, I am now speaking personally. I think that it would be difficult to require everybody to have formal training in statistics. I suggest that you go back to some of the witnesses in your earlier sessions to ask them about that. Of course, what can I say? The more training in statistics there is, the better.

James Parry

The question of training is important, potentially, at all career stages. Sometimes the basic standards for research are perceived as quite obvious, but the calls to our advisory service and other issues suggest that things are perhaps more challenging to put into practice. The question is, are we teaching researchers what they need to know, or are they picking things up as they go along? If it is the latter, are they picking up good habits or bad? Authorship in academic publications is a good example. Some disciplines have formalised authorship criteria. In others, it is more a matter of custom and tradition: this is the way in which it is done. But both types of disciplines still have problems with authorship disputes. Those are insidious, because they can damage research teams. Authorship disputes can also stop papers being published, so this is an example of a questionable practice having quite serious consequences. An element of research integrity and ethics training interleaved throughout career progression as a researcher—not just for PhD students—could be very helpful.

Stephen Metcalfe

The question was about whether it should be formally recognised, as a licence to research, so that you could point to it and, potentially, remove it at some point.

Professor Silverman

I think that removing it would be difficult.

Chair

Do you think that someone who is found out—someone who has failed to meet the high standards of integrity in research—should continue to receive funding for further research?

Professor Silverman

If someone is a truly egregious offender, they should not, but how you enforce that is difficult.

Chair

We have identified that deliberate misconduct is a small but very serious issue. We know that it is there. We know that there is a bigger issue of sloppy standards and so forth. You have said very clearly that more needs to be done. We all need to do more. In a way, you are in the perfect position, given the work that you do, to make a judgment about what needs to be done. In guiding us, you will shape the recommendations in our report. You have talked about the need for better training and, perhaps, making annual reporting mandatory. What other clear recommendations do you have? Rather than just saying, “We need to do more,” can you give us your specific prescriptions for what needs to be done?

James Parry

There is a lot that needs to be looked at in terms of culture and leadership. It is important that we do not micromanage researchers, straitjacket them with bureaucracy or treat them all as potential research criminals. A compliance culture gets you only so far. Training and development through a researcher’s career progression, from PhD student to veteran professor, is very important. We also need to look at how universities, publishers and every other actor in the system are contributing to the broader issues of research culture, and how we can make sure that researchers feel that they can put their hands up and say, “I have got a problem. I am not sure what to do.” To me, the most important line in the concordat is the one that says that there should be no stigma attached to researchers who find themselves in need of help. Institutions—in which I include publishers, regulators and funders—should defend researchers who stand up for research integrity in difficult circumstances. For me, those are the two most important bits of the concordat. That is about culture—the climate within a research team, a faculty or a school.

Chair

Sir Bernard, do you have any additional points?

Professor Silverman

I do not. The difficulty in trying to steer culture is what levers you can use to do that. I do not have any specific suggestions of particular levers, other than those that we have already made. I am encouraged by the fact that there is now a general understanding that this is an important issue. I hope that the work that we do will continue to push in that direction.

James Parry

We are working with the Royal Society on issues of research culture. There will be some outputs later this year. We will certainly keep the Committee informed of that work.

Professor Silverman

We are very pleased to work with the Royal Society. That is an example of our working with a highly respected and influential organisation. For this to be high on the agenda on the Royal Society was quite a win. That is the sort of thing we are very pleased with.

Chair

Thank you very much for your time. It is enormously appreciated.

Professor Silverman

Thank you very much. Examination of witnesses

Witnesses

Dr Tony Peatfield and Dr Steven Hill.

Chair

Welcome, both of you. Do you want to introduce yourselves quickly?

Dr Hill

Good morning. I am Steven Hill. I am head of research policy at the Higher Education Funding Council for England. As the Committee will be aware, the council’s research and knowledge exchange functions will transfer to a new organisation, Research England, in April. I will be in a similar role in Research England in the future.

Dr Peatfield

I am Tony Peatfield. I am director of corporate affairs at the Medical Research Council, as my day job. I am here today mainly because I also chair the RCUK good research conduct network, which is the body that joins together the research councils in the area of good research conduct.

Chair

I have a question for both of you. How would you describe your roles in relation to research integrity as funders? Where do you fit into the research integrity system as a whole?

Dr Hill

We are a substantial funder of research in universities in England. Our role around research integrity is to provide incentives and frameworks within which autonomous universities can deal with matters of research integrity. The primary route by which we do that at the moment is related to the concordat you have already talked about this morning. We make compliance with the concordat a condition of our grants. All universities that receive research funding from us are required to comply with the concordat.

Chair

We have heard that the concordat, welcome though it is, is quite high level and leaves quite a lot of scope for interpretation. Do you think that compliance with the concordat is enough in itself? Could you use your power as funders to drive up standards in other ways?

Dr Hill

There are positives to the scope for interpretation, as well as the negatives that your question implies. We have a very diverse higher education sector. Would one suggest that exactly the same processes and requirements should apply to the Royal Northern College of Music and University College London? We need some scope for nuance within that. I agree that, following the UUK review, there are areas where we need to clarify with institutions what our expectations are. We made some endeavours on that when we made it a condition of grant, and there is some guidance that we make available to institutions. However, in areas like publishing narrative statements, making clear who the named contacts are within universities and so on, there is scope for us to be clearer about our requirements.

Chair

Do you have a plan to set out new requirements in the near future?

Dr Hill

The other thing that is emerging from the UUK review is a new high-level forum to discuss these matters. We intend to discuss them in the context of that forum and then to decide how to move forward.

Dr Peatfield

The research councils fund a lot of research, both in the UK and overseas. We have a responsibility to make sure that our money is well spent. For many years, we have had initiatives to try to tighten up the requirements for that. The terms and conditions of receiving research council grants refer to the fact that researchers have to abide by RCUK guidance and policy, which is fairly detailed on what we expect those who receive our funds to do. Some of the research councils are also employers of researchers, so some of us have a separate role—just like any university, in a sense—as regards how we manage and train our own staff. As funders, we have quite a lot of power, as we are the source of the money.

Chair

Do you share Steven’s view that more can be done and that you can use your power as funders more effectively to drive up standards?

Dr Peatfield

Yes. We have done quite a lot in the past two or three years to make some of our guidance clearer. We now have frequently asked questions on our website, to tighten up some of that. There is probably more that can be done, but we have done quite a lot recently. As was referred to earlier, one thing that we did was ask universities and those we fund to let us know earlier in the process when they receive allegations. It is difficult to ascribe cause and effect, but that appears to have encouraged more reporting to us. The number of cases or allegations has gone up.

Chair

By how much?

Dr Peatfield

It is difficult to say, because it came in only in April last year, but I would guess that it is has gone up by roughly 50%. The numbers are small, of course.

Chair

How do research councils and funding councils across the UK work together on research integrity?

Dr Peatfield

We are both involved in the concordat. We have both subscribed to that, obviously. We were involved not only in developing it at the outset, but in the working group that keeps an eye on it and in the review that was done. The research councils have a funding assurance programme, dipsticking universities every year. We share that information. Most of it is to do with financial matters, rather than research integrity, but two or three years ago we added some research integrity questions, which are very specific. It is a dipsticking thing, so it does not give a complete picture, but it is information that we can share with the funding councils. They have a similar assurance process.

Chair

Do you work to develop common processes and protocols between you?

Dr Peatfield

Not in an active sense.

Chair

Should you?

Dr Peatfield

We do, in the sense that we are all singing from the same hymn sheet. It is not as if there are any differences between us.

Dr Hill

We work together closely. We currently have separate assurance and audit processes, which Tony has already mentioned. It is worth mentioning that, as well as working very closely with the research councils, historically HEFCE has worked with the other UK funding bodies in Scotland, Wales and Northern Ireland, which have similar policy approaches to us. We have ensured consistency across that. On close working, it is also worth saying that, in a couple of months’ time, Tony and I will be in the same organisation. That gives us some opportunities for increased sharing.

Chair

Do you think that UKRI provides us with the opportunity for greater co-ordination of processes, protocols and so forth on research integrity?

Dr Hill

We co-ordinate a lot already, but having a single organisation will make that more streamlined and straightforward.

Chair

But you will maintain the link with the devolved Administrations.

Dr Hill

Clearly, it is a very important role for Research England within the context of UKRI to maintain the links with our colleagues in Scotland, Wales and Northern Ireland. That is definitely very high on our agenda.

Dr Peatfield

It will also bring in Innovate UK, which funds more into the private sector. That may address the question that we had earlier about the extent of private sector interest in this.

Chair

Sure. RCUK guidelines now require research organisations to report allegations of misconduct to the relevant research council. Will RCUK publish summary data from the information that it receives in that way?

Dr Peatfield

Through the funding assurance programme, we receive information from selected universities. We have published that information on the RCUK website for the past five years. It was one of the requirements in the concordat that we produce a narrative statement, so that information has been published. Obviously, it is a selective sample, so one cannot read too much into the information there. As I said, since April, we have been collecting information from universities more completely.

Chair

At the earlier stage of investigations.

Dr Peatfield

Yes. Of course, we can ask only for information that involves RCUK funding or RCUK involvement in some form, be it through training, through funding or through referees and panel members, so it will not be a complete UK picture. However, it will be a much more complete picture than we had in the past.

Chair

Will that be included in what is published?

Dr Peatfield

We will include that information—the high-level numbers, certainly—in what we publish in future narrative statements.

Martin Whitfield

I want to look again at the concordat. Unlike the first panel, you are both signatories to the concordat. Just to give an overview, will you say what proportion of institutions you think currently comply with it?

Dr Peatfield

There are many aspects of the concordat.

Martin Whitfield

Hence the use of the word “overview.”

Dr Peatfield

It is difficult to say. As we discussed earlier, there is a very wide range of universities in the UK, which covers some that are research intensive and some that are not. It is almost an impossible question to answer. One would have to be more specific and to drill down into specific questions.

Martin Whitfield

It was a slightly unfair question. How about we turn it the other way? As funders, have you ever withdrawn funding from an institution that is not complying with it?

Dr Peatfield

The research councils much prefer carrots to sticks. We tend to try to work with the institutions, where there are cases. As we said in our written evidence, often by the time that a case is proven the research grant will have ended, so withdrawing that funding is not feasible. From memory, I do not think that we have prevented an organisation from applying to us for funding because of some previous misconduct arrangement.

Dr Hill

To go back to your first question, we have information on universities in England. I can tell you—this will lead into answering your second question—that in last year’s funding return we had two institutions that were not compliant and declared themselves as not compliant. Like Tony, we favour carrots, rather than sticks. That triggered a process with those institutions that involved very close contact with them, the requirement to develop an action plan and following up that action plan. I am pleased to say that in this year’s return both those institutions are now compliant. This year we have two different institutions that are not compliant. Both are in the part of the sector that carries out very little research; in fact, one of them does not receive research funding from us. However, we will talk to the other institution and go through the same process, to ensure compliance. The objective here is to develop the culture whereby institutions recognise where they have shortcomings and are open to working with us in order to address them.

Martin Whitfield

And those institutions and organisations recognise the significance that you place on the concordat and the potential nightmare scenario of not getting funding. We have heard that there are a lot of carrots—a lot of assistance and help—between the two. Do you think that the wording of the concordat is strong enough as it stands, or would you like to see improvements in it? Would you like to see it develop?

Dr Hill

There are some areas where more clarity would be helpful. We have talked about the publishing of narrative statements, where the wording of the concordat is that universities “should” do that. We are of the mind that we need to make clear that that is not just an option, but a requirement. There are examples like that, but the evidence from the UUK review of the concordat is that it is driving culture change and change in behaviour within institutions. We need to continue to support that.

Dr Peatfield

One thing that is not clear in the concordat is that, in my view, universities should make public on their websites the name of the person who is responsible for research misconduct.

Chair

It is a recommendation.

Dr Peatfield

Sometimes it is just a job title, rather than a name. I suspect that giving a job title is convenient, because the email address will not change if the individual changes role. However, I think that it rams home the personal responsibility of those people if their names are on the website. That is one area where it could be a bit clearer.

Chair

And the name for whistleblowers to contact.

Dr Peatfield

Yes. It will be the same person, possibly.

Dr Hill

I very much agree with that.

Martin Whitfield

Do you think that there should be an obligation on that named person to produce an annual narrative report?

Dr Peatfield

They are supposed to, as part of the concordat, so yes.

Martin Whitfield

A more useful report, shall we say? A more transparent report, possibly.

Dr Peatfield

In my view, transparency is always good. There are issues about confidentiality and deductive disclosure that universities have to worry about, if they release too much information that appears to be anonymous, but people can work out who is involved.

Martin Whitfield

RCUK gets organisations to answer six questions about compliance with the concordat. Are those responses published?

Dr Peatfield

Not individually.

Martin Whitfield

May I ask why not?

Dr Peatfield

It is something that we could consider.

Chair

You have just said that you support transparency.

Dr Peatfield

I do. It is possible. We have not done it up to now, but we certainly could.

Martin Whitfield

Do you see that there would be a benefit to publishing that?

Dr Peatfield

The high-level information is published. We publish the numbers.

Martin Whitfield

I meant the information that drills down, in response to the questions that you put. They are posed for very good reasons.

Dr Peatfield

Yes.

Chair

Should we not all see whether universities that are receiving public money are meeting the standards for something they have signed up to?

Dr Peatfield

Yes. Sometimes the responses are not black and white, but I agree with you in principle.

Chair

So that is something that you will consider.

Dr Peatfield

Yes, we will consider that.

Martin Whitfield

My final question relates to an issue that was mentioned with the first panel. Do you think that there should be an annual report on the state of research integrity? At the minute, you are dipsticking and sampling, rather than doing a fuller report. Do you think that there would be value in that?

Dr Peatfield

Personally, I think that there would be value in it. As I have said before, as funders, we can require only information concerning our own funding.

Martin Whitfield

Absolutely.

Dr Peatfield

In that sense, it would not be complete. However, one could envisage a situation where, if universities complied with the concordat fully, somebody could assemble all those inputs and create an overall report from them. That would certainly be possible.

Dr Hill

I would support that, too, with the same proviso that there is a challenge in how you collect all the information. There is some work that could be done at a national aggregate level, rather than necessarily looking at individual institutions. Maybe the UKRIO study you heard about earlier will look at that. I go back to the Retraction Watch database. It is assembling a database of retractions, as you have already heard. Retractions are not a perfect proxy for issues of research integrity. They are an important part of the scientific process, and there are many perfectly valid reasons why things get retracted, but research misconduct will certainly feature in them. I looked yesterday, and the Retraction Watch database now has nearly 17,000 retractions on it. It is possible to mine the data to look at how the UK performs relative to other countries. UK authors appear on papers that account for just under 3% of the retractions. That is set against the UK contribution to journal articles, which is about 6% of the global share. That indicates that the UK is well below what you might expect. It is not dissimilar to the US, where the retraction figure is about 10% and the global share is around 20%. Again, they are doing reasonably well. You can contrast that with other countries like China. China accounts for 20% of global publications, but nearly 50% of the retractions on the database involve authors from China. Using data like those, you can benchmark national performance, as well as look at individual countries.

Martin Whitfield

We have spoken a lot about improving the culture of research. Would the elements that we have discussed today, although not answering that, go a long way towards improving the culture of research and, more importantly, research integrity? Do they all form part of the jigsaw of the answer?

Dr Peatfield

Yes, absolutely.

Chair

Do you share the view that institutions that are carrying out a lot of research and that repeatedly give nil returns for inquiries in their narrative statements should at least give rise to some concern? Would you expect there to be issues an institution should be looking at?

Dr Hill

I go back to the comments by the earlier panel. The current data, until April this year, relates to formal investigations, rather than the screening phase. If I can turn your question around, I would be more concerned if that broader definition of what needed to be reported led to zero. If nothing was even being screened for investigation, that would be concerning, but I am not sure that seeing no cases that go to formal investigation is necessarily concerning.

Chair

That begs the question, should the institution’s annual report, which the concordat encourages, include those early-stage investigations, so that there is a full picture of what is happening? In a lot of institutions, we are just seeing nil return, nil return, nil return. That gives rise to some concerns.

Dr Hill

Certainly, the RCUK changes are moving in that direction. I am supportive of that.

Chair

Of the narrative statements each year showing the full picture.

Dr Hill

Yes.

Carol Monaghan

We have been told in one submission that pressures of the REF contribute to problems with the research culture. Do you agree?

Dr Hill

This is probably a question for me. I agree that there is a perception and that there are potentially conditions within universities that relate the REF to pressure to publish. However, we have to look critically at the features of the REF and the extent to which they are really linked to pressure to publish. The first comment that I would make is that the REF is a UK-specific activity—it does not apply in other countries—yet we see that the UK performs quite well, both in terms of the retraction data I have just talked about and in terms of the quality of publications. A context where there is strong pressure to publish tends to lead to an increase in the number of publications, with a commensurate decrease in their quality. There is some evidence for that in other countries. However, what we have seen in the UK, from the ways in which we can measure that, is an increase in quality over time. That suggests that the UK is not in a “pile ’em high, sell ’em cheap” mentality when it comes to research publications. In the REF itself, there is a strong focus on things other than publications. In the last REF, 20% of the outcome was focused on broader societal impact and 15% was focused on the research environment—which, by the way, includes considerations of research integrity. In the previous exercise, the publications element—what we refer to as the research outputs element—required researchers to produce only four publications over a six-year period. In many disciplines, that is quite a low requirement, in terms of the total volume. In disciplines where long-form research outputs are more common, we have provisions that allow double-weighting of outputs, which reduces the number further. For the future, we are taking the average number per individual down lower, to 2.5 research outputs per individual, on average.

Carol Monaghan

Is that annually?

Dr Hill

No, it is over the whole period. That is a seven-year period, in the case of REF 2021. The fact that we are talking about an average per person reflects something that Sir Bernard mentioned earlier—that we are moving away from the REF being very strongly linked to individuals, and their research outputs, to looking much more consistently at the output of research units and groups.

Carol Monaghan

So you are pushing more for quality than for quantity with those numbers.

Dr Hill

Yes. To be frank, the REF, and the RAE before it, has always had a strong pressure towards quality, rather than quantity.

Carol Monaghan

May I push you a little more? You said that an institution’s research integrity was included in the judgments. In what way is that done? Is there scope for making research integrity a larger part of the REF judgments?

Dr Hill

Yes. In the 2014 exercise, two of the broad subject groupings took into account matters related to research governance and integrity in the environment section. Those were the grouping covering medical and life sciences research and the grouping covering social science research.

Carol Monaghan

Who judges that?

Dr Hill

It is judged by the panel members, who are members of the academic community, together with users of research. In the medical sphere, for example, you would have practitioners, patient groups and others represented. For the upcoming exercise in 2021, we are currently at the stage where we are assembling the panels and they are coming up with their criteria. We will require all of them to consider how research integrity and misconduct issues can be covered in the research environment section. That will be a new requirement. It will need to be looked at in the context of the different disciplinary communities. Clearly, what matters in the space for medical research is rather different from what matters in research in the humanities. However, we will require all the panels to consider that and to assess it as part of the exercise.

Carol Monaghan

I am slightly concerned by something you said. Perhaps I am being overly-concerned, but you said that the judgments had been made by other academics and those using it. Is there not a potential conflict of interest there? Surely an independent judge would be better.

Dr Hill

Those panels are made up of respected people from their fields. The judgments require that expertise. We have very strong processes in place to avoid conflicts of interest where panel members’ own universities are being assessed—they would always leave the room and not be involved in those discussions. These processes are not unlike those used by the research councils. When the panels assemble, they are making assessments on behalf of their disciplinary communities and making judgments between institutions, so they have a strong incentive to do that in a robust way that makes clear where there are strengths and weaknesses.

Carol Monaghan

You mentioned that part of the REF was looking at the wider societal impact. One submission we have had refers to “… disproportionately rewarding high-profile outputs that are the kind of flashy, breakthrough-type research.” That was a comment about the REF. Does the REF make it difficult to secure funding for replication studies and meta-research—research into research methods?

Dr Hill

I do not think it does. The panels judge research outputs against three criteria: originality; significance; and rigour. Those criteria are applied to all sorts of different types of outputs, not just journal articles: meta-reviews and different pieces of work.

Carol Monaghan

I think the comment was about how you prove the societal impact if all you are doing is researching research.

Dr Hill

I think we have two elements of the REF. We look at the research outputs that are judged against their academic impact and the difference they are making in their disciplines in advancing knowledge, and we have a separate section that looks at the broader societal impacts. The section that looks at the broader societal impacts looks back over a 20‑year horizon rather than focusing on just the assessment period, and it is looking at all different types of societal impacts. There is a huge diversity within those impacts submitted for assessment. Drug discoveries will potentially come from research outputs in the past that were very high profile and regarded as being huge in their discipline but have led to drug development. You also have examples that are on a different scale and involve research that can be very local and focused on particular problems and communities, so there is a huge diversity.

Carol Monaghan

I understand that, but the problem is that, if you are not able to show, even on a local scale with a small project, that this research has had an impact, there will be difficulties. That brings into question who is going to fund the research into research methods, which is what we are talking about today.

Dr Hill

Research into research methods, if it meets the criteria of originality, significance and rigour, could be submitted as part of the REF exercise and would score very well, so there is no barrier to that sort of work.

Stephen Metcalfe

You will have heard Professor Silverman say earlier that he was generally supportive of the idea of some formal qualification in research. Would you consider introducing some form of licence or qualification to demonstrate that you understand what research integrity is, which could be withdrawn if there was a problem? It is a bit like the GMC striking a doctor off the register.

Dr Peatfield

It would be very bureaucratic and difficult to implement. Research is a worldwide activity. If you are recruiting somebody from Germany to the UK, what would you expect them to bring in the form of licensed activity? At one level, a PhD is deemed to be a scientific qualification that defines you as a scientist. That harks back to what we discussed earlier about PhD training and the extent to which research integrity is included within it. It is possible to set up the equivalent of the General Medical Council with a register of researchers. There would be huge turnover with people coming and going. You would have to have criteria in addition to those people must have to get their PhDs. Therefore, those criteria might be very different for a clinical triallist compared with an astrophysicist. How do you judge those criteria? Who decides? You have to have a process for striking off people. Somebody would have to complain, and then you go through a legal process to strike them off, because you are depriving somebody of the right to work. My personal view is that it would be extremely bureaucratic and expensive to set up and probably will not work very well.

Dr Hill

I tend to agree with the analogy with the GMC. There is a clearly defined body of knowledge and practice related to being a medical practitioner. It is rather difficult to define what that would be across the spectrum of research. As Tony said, even within the sciences there is a huge amount of diversity, and when you move into the social sciences and humanities that diversity is greater. I agree with Sir Bernard that more training for PhD students and researchers across the spectrum in matters of research integrity would be a good thing. We already have another concordat on research and development that universities have been implementing for a decade. Associated with that is a Researcher Development Framework that does indeed include those issues, so we already have some soft measures in place that encourage that training. It is also part of the research integrity concordat. I think it is an area we should look at in firming up the potential.

Stephen Metcalfe

I hear what you say about a licence to research, which we might want to explore further. Will you explain what sanctions are available to you where you find there has been a problem with research?

Dr Peatfield

A lot of the sanctions are local, so the employer does the investigation and is the body that decides whether there has been a proven case. Therefore, the primary sanctions are from the employer. Certainly, within the research councils as employers of researchers it is part of the contract. If somebody is proven to have committed misconduct it is a sackable offence, and I think it is the same with universities. Therefore, the first line of sanctions is with the local employer. Funders can apply sanctions, and I have listed in our written evidence what the range is. We can terminate a grant if it is still running; if we wanted to, we could stop a clinical organisation applying for new grant, but we have never done that.

Chair

Do you ever do it?

Dr Peatfield

I do not think we have ever stopped an organisation applying for a grant. On occasion we have gone back where there has been a case against an individual and worked with universities to take that individual off the research project, or ensure much better supervision within the research project. We have acted more with a carrot than a stick to try to make sure the research continues, because we have funded it up to a certain point and it is in our interests and the researchers’ interests to try to draw it to a satisfactory conclusion. Therefore, we work with universities to try to do that. The one case where we applied a sanction arose because somebody was found to have plagiarised. They were a Research Councils referee and we removed them from the list of potential referees for that reason. I am not a lawyer, but I understand that it may be illegal to blacklist people if it stops them working, so blacklists per se are not an option.

Stephen Metcalfe

There is no central list of those who have had funding withdrawn.

Dr Hill

No.

Stephen Metcalfe

On the basis that you assume it may be illegal.

Dr Peatfield

We have never actually done it within Research Councils, but, looking to the future, if we did we would have a record of the fact we had done it.

Stephen Metcalfe

You are not keeping a blacklist. You have the ability to withdraw funding. I think you gave us one case. In 2011 the previous Committee was told that there were no cases where funding had been withdrawn on the grounds of fraud or misconduct in research. Has that changed somewhat?

Dr Peatfield

It has not changed. We have not withdrawn funding; we have worked with institutions to ensure that the funding we have provided has been used properly.

Stephen Metcalfe

Where problems have arisen with the integrity of research are they being reported quickly enough for you to be able to withdraw funding?

Dr Peatfield

That was one of the reasons behind the requirement that universities inform us as soon as they have decided there is a potential for research misconduct, as opposed to some other kind of misconduct, because quite a lot of these things arise from a breakdown in personal relationships. Therefore, once the university has decided that there is a possibility that there is research misconduct that is the stage at which we would like to be informed, so we should be much better able to track the time it takes from that first notification to an outcome. Despite the fact that everyone wants to do something quickly, for all the right reasons, setting up the investigation panels, making them meet and getting all the evidence together takes time.

Stephen Metcalfe

Do you think any of this will change with the creation of UKRI?

Dr Peatfield

On day one, no. UKRI is concentrating on a lot of other things that need to be in place for day one, but I know it will be taking it seriously—Mark Walport himself has said so. Obviously it will want to look at bringing in Innovate UK and Research England under the same umbrella. I cannot speak for it in any detail, but I am sure it will take it very seriously and try to improve things.

Chair

There is a problem, is there not, where somebody departs from a university with a compromise agreement and non‑disclosure clause that keeps everybody quiet about what has happened? That researcher then crops up somewhere else and again applies for public money to do research. How do we address that problem? Should it be acceptable for universities to allow someone to leave with a deal that maintains confidentiality and effectively covers up misconduct in research?

Dr Peatfield

Probably not, but it may be quite difficult to put into practice, in that if there are subsequent retractions they will make the outcome public. Even if a university may think it has reached some sort of compromise agreement and there is a subsequent retraction, it will become evident that that particular researcher has been committing misconduct. It crops up occasionally where somebody just moves from one institution to another. A report by Science Europe last year recommended that universities employing researchers should ask the question at interview, “Have any cases of misconduct been held against you?” If that person lies, that would be a reason for subsequent dismissal if they were then appointed. There is an onus on the new employer, or any employing institution, to ask those who are applying for jobs what their history has been.

Chair

Perhaps that is one way the concordat needs to be tightened up.

Dr Peatfield

Yes.

Dr Hill

I add two comments to that. First, I agree it is an issue. It is also an issue for poor practice and behaviour in many other parts of society.

Chair

It does not mean it is not important.

Dr Hill

Indeed, but I do think we have to keep that in context. Secondly, on retractions one of the weaknesses in the system is how poor we are—when I say “we,” I mean primarily journal publishers—in collating information around retractions. It can be quite hard to find out whether a piece of work has been retracted and who has had papers retracted. The work that Retraction Watch is doing is great, but I still think it is not always 100% transparent. When you do a search and find a journal article, it is not always 100% transparent that there has been a retraction or modification.

Dr Peatfield

Or that the retraction has been due to some kind of misconduct, or even sloppy science. Papers are retracted for quite good reasons—for example, that the wrong regimen was used on the day and there was a mistake in how the research was done, which may be purely accidental. Therefore, one has to be careful about how one connects research misconduct with retractions.

Vicky Ford

We keep focusing on universities as research centres, but a lot of research in areas such as genomics is done in the NHS. The relationship between doctors and pharmaceutical companies is an area where one may be thinking about training doctors in medical research. We have talked about other organisations, not just universities. Is that an area?

Dr Peatfield

Being a medical practitioner, there are a lot of ethical and other issues related to research integrity and misconduct as part of that. Doctors are in a different position for all sorts of reasons. I would refer you to the ABPI, which recently tried to create a register of doctors’ interests and their relationships with pharmaceutical companies. I think that attempt at transparency has been helpful.

Bill Grant

Training has been touched on. A key player in research, as will be well known to you, is the Association of Medical Research Charities. I sense they may be funders and very important players. It has advocated a role for centralised training on research integrity, as opposed to what would appear to be a fragmented system with no centralised training. Do you agree that there is a role for centralised training around integrity? If so, who would be best placed to deliver such centralised training?

Dr Hill

There is probably a core of activity where consistent training would be helpful. It would not necessarily be the best outcome to have a single provider, but potentially UKRIO could develop some guidelines around what that core curriculum might be and other organisations could provide that training. That would be my view.