Oral evidence: Assessing quality in Higher EducationHC 572-iiiTuesday 8 December 2015

Oral evidence, 8 December 2015.

Witnesses

Joseph Johnson MP, Minister for Universities and Science, and Polly Payne, Director of Higher Education, Department for Business, Innovation and Skills, gave evidence.

Chair

Minister, good morning. Thank you for joining us. We aim to finish by about 11.15 am. There is an awful lot to get through, so let us crack on. For the purposes of the record, could you introduce yourself and your colleague and say where you are from?

Joseph Johnson

Yes, of course. I am Jo Johnson, Minister for Universities and Science, and this is Polly Payne from the Department for Business, Innovation and Skills, a director in the higher education bit of the Department; is that correct?

Polly Payne

Yes, Director of Higher Education.

Chair

Minister, we welcome the Green Paper. We welcome the emphasis on a number of things within it, such as improving the quality of teaching, widening participation, and making sure that we retain the status of being world class in higher education. Can I take you back to your speech on 9 September about higher education fulfilling our potential? There is a particular passage in there that is really interesting, talking about a family and varying levels of experience. May I quote you? “This patchiness in the student experience within and between institutions cannot continue. There is extraordinary teaching that deserves greater recognition. And there is lamentable teaching that must be driven out of our system.” Could you tell us where that lamentable teaching is?

Joseph Johnson

Thank you very much for having me, and I will certainly come to that in just one second. What I want to say is that it is a pleasure to be here to give evidence before you, and I am delighted at the interest the Committee is taking in this very important subject. There is extraordinary excellence across our higher education system; that is the first thing to say. We have a great university system in this country, it is one of our national success stories, and it is a terrific calling card for us on the global stage. It is very important to put that frame in context out there, but of course the sector cannot stand still. University systems around the world are becoming more and more competitive. Developing countries are putting in place stronger and stronger frameworks for their own university systems, and in that environment it is incumbent on us to continue to make a great sector greater still. That is the opening frame of how I see the sector. It is continuing and continuous improvement, and that is all the more important for us, as a sector, at a time when we are seeing ever‑increasing numbers of our young people go through university. We are now at a stage of mass higher education in this country, with about 47% of people likely to go through higher education at some point in their lives, and it is vital for us, as a Government, that we ensure that they are getting the best-quality experience for the time and for the money that they are investing in higher education. You referred back to a speech I gave to Universities UK and I used that word; it made a point. It made a point that there is, essentially, patchiness in provision and I am happy, before you, to give evidence of where I see patchiness, if that is helpful.

Chair

Would you use the word “lamentable” again?

Joseph Johnson

I certainly made the point, and the point was made in order to highlight the fact that there is patchiness and variability in provision.

Chair

“Patchiness” is not “lamentable” though.

Joseph Johnson

Patchiness and variability are the features that I want to stress before you today. I am quite happy to give plenty of supporting evidence of that and I think the sector, in its responses to you as a Committee, has also agreed that there is a need to focus on the quality of teaching in our institutions. I am happy to give more evidence on that, if you want.

Chair

I would be very keen for you to give evidence to us, but just to push you on this, “lamentable” is an extraordinarily strong word. Would you use it again?

Joseph Johnson

I think there are patches of poor-quality provision and whether or not we want to use that word—

Chair

Lamentable patches?

Joseph Johnson

Whether we want to use that word, it certainly made a point. It highlighted the point I was trying to make. I do not see the need to repeat it ad nauseam, but I think I made my point.

Chair

Okay, thank you. Could you give us some evidence as to where teaching is patchy?

Joseph Johnson

Yes, certainly. We can see it in a number of different ways. It is obviously a very important issue for the Committee, and it is central to our thinking in terms of our work in introducing the teaching excellence framework. Let us, first of all, look at patchiness in terms of how students themselves see the system. In the NSS 2015 survey, two thirds of providers are performing well below their peers on at least one aspect of the student experience; and 44% of providers are performing well below their peers on at least one aspect of the teaching, assessment and feedback part of the student experience. Let us take another aspect of how students perceive their experience. The Higher Education Academy’s recent survey came out earlier this year, and one third of undergraduates paying tuition fees in England believe their course represents very poor or poor value for money. Let us take another aspect of that: 21% of students found the information given before they started their course vague; 10% said it was misleading. One in three said that knowing what they now know they would have chosen a different course. Over 60% of those felt their course was worse or worse in some ways than expected. Of these, 32% said the course was poorly organised, 30% said they received fewer contact hours than they were expecting and so on. Therefore, from the student perspective, there is certainly some room for improvement. Let us take a look at how this is having an impact on their study. It is resulting in a variation in the amount of study time that students are undertaking. There is evidence that students in this country are not as engaged as those in other countries and are studying less than other European countries in term time. That is from the student perspective. We can also see some evidence of patchiness in terms of outcomes. Dropout rates vary hugely across the system, and you heard earlier from the Director of the Office for Fair Access about the differential attainment rates and progression rates of different groups through university that cannot be explained by prior educational attainment. These dropout rates vary hugely, as I said. Non‑continuation rates for certain black entrants into university are much higher than for white students, for example, and these are differences that cannot be explained by prior educational attainment. They have to do with issues like the degree of teaching support and academic support that students are getting at university. Let us take some aspects of patchiness in outcomes for students. At least 20% of graduates are not finding jobs in high‑skilled graduate employment three and a half years after graduation, according to the Higher Education Statistics Agency statistics, and it might be as much as 58% according to a survey on this subject by the Chartered Institute of Personnel and Development. Therefore, there is a very significant body—somewhere between 20% and 58%—of students who are not finishing up in graduate jobs or going on to further study three and a half years after graduation.

Chair

Is that not something to do with skills mismatch in the economy and the structure of our employment base rather than teaching in higher education?

Joseph Johnson

It is to do with a lot of factors, and I am not saying teaching quality is the sole factor, but the quality of teaching and the right choice of course and the right amount of information available to prospective applicants to universities are all factors that contribute to these poorer outcomes. Can I just finish off? The last aspect of it is on how employers see the quality of teaching and whether employers could support an assertion of patchiness. Almost one quarter of employers, according to the Association of Graduate Recruiters, have open vacancies because they could not find the right skills in the recent graduate cohort. The OECD is producing research that shows that twice the proportion of UK graduates have low levels of basic skills than the OECD average, at about 6.9% versus 3.6% across the OECD. Thus, we are seeing poorer outcomes from our system of higher education in some ways than in other countries, and that is all supporting evidence for the assertion of patchiness.

Chair

Do you put that down to teaching?

Joseph Johnson

If universities are not dealing with low levels of basic skills to that extent, we have to attribute it to a system of admission that is not working or a system of value added that is not working either—one or the other.

Paul Blomfield

I think it is worth saying at the outset that across the Committee and from all the witnesses we have spoken with, there is support for your decision to focus on teaching quality. There is a concern—I am sure you share it—that we need to get it right. It is also welcome—the Chairman has made this point on a number of occasions—that you see the Green Paper very much as a Green Paper, and that ideas can evolve through the discourse that you have kicked off. In that context, one of the issues that has come to us time and again is about the link between fees and the TEF. Clearly, we are going to have to bite the bullet at some stage on the need to increase fees to enable universities to continue to function. However, there is concern about whether this is the appropriate way of doing it. For example, we have heard that within institutions teaching is patchy, and is it right that some subject areas will be able to charge higher fees when they might not be teaching as well and they are on the back of another department. We have heard also that from the way data is collected there could be perverse outcomes; and that fees will go up, reflecting what is happening at one point in time, but the teaching quality might have slipped by the point that people are paying the fees. Do you think that there is a case for thinking about whether we detach the decision to increase fees from the TEF?

Joseph Johnson

As a Committee, you have heard a variety of messages on this theme, and I would say it has not been entirely one-way. There have also been people who have come before your Committee in recent weeks who have made the case for or can at least see a positive way of seeing a link between, tuition fees and teaching quality. Professor Gaskell, for example, said that if there were evidence that minimal standards of teaching were not being increased, he would have no problem in that being linked to a denial of the opportunity to increase fees. Dame Julia King of Aston University, writing in the Times Higher, said, “Aligning fees with a quality assessment based on research evidence…could give more of our graduates a richer, more tailored experience—and a better start in their careers”. There is not a blanket view on this question. It is important that when we think about how we sustainably fund our universities, we have a framework for recognising how we want to provide the underpinning support to universities. It has to be based on some principles and teaching quality, as an organising principle of how we arrange our system of finance for universities, seems to me to be a very sensible one. We want provision to expand where there is high-quality teaching. Universities that offer high-quality teaching will be able to invest more in their teaching and now that we are in an era of limitless student places—we have lifted the cap on student numbers—universities that want to expand their provision and provide more opportunities for people to study at high-quality institutions can do so. Similarly, and in the other direction, institutions that have not received the additional resources from the TEF in this way would find that they would need to refocus their efforts on teaching to get the benefits that higher teaching would bring. I think it is a good organising way of thinking about how we sustainably fund the system in the interests of students.

Paul Blomfield

Do you think it might produce perverse outcomes? For example, coasting departments increasing their fees on the back of an institutional rating or the time-lag issue.

Joseph Johnson

We want to ensure that institutions as a whole are assessed by the panel. That means taking into account the overall framework within which teaching operates in a university, so obviously the experience that students get in a particular discipline is very important. However, so too are the institution‑wide attitudes and status accorded to teaching that universities have and they justify looking at the uplift on an institution basis. Obviously, in time, we want to be able to be more granular and to have the TEF operating on a course‑by‑course and faculty and discipline line, but there is also merit in taking an institutional approach, because that captures the institutional factors that contribute towards the teaching and learning environment.

Paul Blomfield

We could pursue this for some time, but I am conscious of the time.

Chair

Before you come back in, Paul, do you anticipate, Minister, that in the future some iteration of TEF—TEF3, TEF4; whatever—could be having tuition fees based on a course-by-course basis?

Joseph Johnson

At the moment, we are looking at an institutional framework for fees, and that is what we are proposing in the Green Paper and that is what we are consulting on. Obviously, there is scope for systems to evolve over time and we will take the responses to the consultation on this point very seriously, but what we are proposing in the early years of the TEF is institution‑based fee structures.

Paul Blomfield

Just pursuing a slightly different issue, we have talked a lot about learning gain and how we measure it and that being the real goal for an effective TEF, if we can get it right. Do you agree that the work that HEFCE is doing in its pilots will help to inform perhaps not TEF1, but certainly TEF2 and beyond?

Joseph Johnson

We certainly hope so, but there is no guarantee of it. Measuring learning gain is a really important objective for any system that seeks to assess teaching quality, but it is difficult and we do not know yet how the HEFCE pilots are going to be doing—what kind of results they will come up with, whether they will be relevant, useable and so on—so we cannot guarantee it, but certainly it is something we are looking at with great interest, and I hope they produce some really valuable results that will inform future variations of the TEF.

Paul Blomfield

You would accept that if we do not get TEF2 and beyond right, there could be some real potential damage for the sector.

Joseph Johnson

No, I do not accept that.

Paul Blomfield

If we do not get it right in terms of ensuring that we have an effective assessment of quality when we begin to differentiate more, because TEF1 is not going to, is it? It is going to be fairly light touch.

Joseph Johnson

Let us just be clear about how we are phasing in the teaching excellence framework, so everyone is totally clear on that. We are phasing it in very, very gradually. For the first year, in 2017-18, the process is looking at the baseline review of quality assurance as the threshold for an institution being eligible to increase its fees in line with inflation. In 2018-19 and thereafter, we will be using the TEF framework to be the basis on which we make these judgments. For the moment, until we have the learning gain pilots—and they will not be ready for some years yet—we are proposing to use the metrics that we are consulting on in the Green Paper, supplemented by any qualitative information that institutions themselves want to put forward. All this information—quantitative and qualitative—will be assessed in the round by a panel of expert reviewers, so that they can take an informed, nuanced judgment. We are not basing it just on metrics, nor are we basing it just on what institutions supply—it is a blend of the two assessed by the panel.

Chris White

In his opening remarks, the Chairman gave you the opportunity to give the Committee some statistics. Could you broadly outline how you think the framework would address the variability of some of the quality of education, and could you also expand on the consultation process that you are having with institutions and explain how genuine it is?

Joseph Johnson

Yes. As the Chair and Mr Blomfield said, this is a Green Paper. It is not a White Paper. It is a genuinely open consultation running until 15 January, and we are holding a lot of stakeholder events in BIS. I have been going around holding roundtables in every bit of the country I get to, with vice-chancellors and others in universities with views on this. We are and have always been completely intent on working with the sector to make sure we are coming up with a framework that does what it intends to, which is to ensure that students consistently get the high-quality teaching and learning environment they have a right to expect, that employers get the skills pipeline they need, and that taxpayers have confidence that they are underwriting a system that is delivering value for everybody. It is a very consultative process, and we will be consulting again in the new year on the technical details of the metrics and how they would be operationalised, how we would do the benchmarking for all the important contextual factors that we would need—i.e. how many disadvantaged people are studying at particular universities and so on—what is the background, and what is the particular characteristic of that area and region? Those technical details will be fleshed out in a further consultation document in the new year.

Richard Fuller

I will ask Ms Payne, if I may, if this is the first Government to look to try to implement a teaching excellence framework?

Polly Payne

This is the first Government to implement something quite like this. Other Governments have looked at enhancing teaching quality and have looked at things like measuring learning gain. Madeleine talked about the CLA system in the US. The Australians have also done some work on teaching excellence, and we have been discussing those initiatives with those Governments.

Richard Fuller

It just seems to me a normal thing one might expect to have. You go to university to learn; why is there not a framework for showing who is good and who is not? Why has it never been done before?

Polly Payne

Internationally nearly all Governments have a system of quality assurance of universities focusing on teaching, as indeed the UK does, and, as Douglas previously said, the UK is a world leader. The additional and new element, which the UK is at the cutting edge of, is having this teaching excellence framework. Again, as Madeleine said, the UK is at the cutting edge of trying to measure learning gain, so this is new territory.

Richard Fuller

Given that this Government have grasped the nettle about it, I would have anticipated participants from the sector to have been all over the Minister for a love‑in about how excited they were about this, yet in the earlier sessions there was quite a bit of grumpiness from some of the participants. From what you are hearing from the sector, having been given this great opportunity, are they a bit grumpy about saying how hard it is going to be? There is a word we hear time and time again, which has a certain truth to it, but it also sounds like a word thrown up to be a roadblock, which is “diversity”.

Joseph Johnson

Can I come in on that?

Richard Fuller

Please do.

Joseph Johnson

You are right. We are the first Government who are taking action as clearly to address this issue, but that is not to say we are the first Government to recognise that there is a problem. My predecessor David Willetts said very clearly that universities have primarily focused on research, because that is where the funding and prestige come from and where the competition was strongest. However, this is not just a Conservative focus. Before him, we have had leading figures from the Labour years in office make similar points. Lord Mandelson, your predecessor as MP for Hartlepool, Chair, said, “We also need to look, in my view, for ways of incentivising excellence in academic teaching. We have to face up to the challenge of paying for excellence.” Before him, Gordon Marsden, my current shadow in this role, said, “We must reassess the balance between teaching and research. HEFCE should seriously consider incorporating a teaching quality assessment exercise in the Research Assessment Exercise”, which is the parallel exercise for research. David Lammy has made similar comments. I think the sector has not been hostile en bloc. There are many supportive voices in the sector, and before your Committee you have had a number of witnesses come before you who have said things that are supportive of what the Government are trying to do. Steven Courtney at the University of Manchester said, “Teaching in UK education institutions has, since the emergence of the Research Assessment Exercise, lacked parity of esteem with research in terms of its contribution to an institution’s reputation, funding, position in key international league tables and importance in attracting staff”. Professor Colin Riordan of Cardiff said, “It is clear that there is a culture in research‑intensive universities and perhaps in all universities that does value research more highly than teaching”.

Chair

Forgive me, Minister. With the greatest respect, we can read those transcripts.

Joseph Johnson

Good. I just did not want it to be misrepresented.

Chair

I can understand there is broad support for what you are doing, but we are trying to interrogate in terms of—

Richard Fuller

Indeed, and also, Minister, I think there is support. It is the passivity of the responses that struck me.

Joseph Johnson

It is an opportunity for the sector.

Richard Fuller

I have one other question on widening access. Minister, you are trying to achieve two things, which are to improve the educational excellence at higher education, and also to meet the Prime Minister’s goal on widening participation. We have heard, from the panel before and others, about some of the tensions between those. Indeed, my colleague Mr Blomfield just mentioned those. Can you assure the Committee that in your deliberations beyond the Green Paper you will put front and centre maintaining the progress on widening participation as you think about the teaching excellence framework?

Joseph Johnson

Yes, absolutely. I see teaching quality, widening access and improving outcomes for people from disadvantaged backgrounds, as the Director of the Office for Fair Access was saying earlier, to be intimately related. That is why the teaching excellence framework looks very closely at how well institutions are doing at supporting people from disadvantaged backgrounds into university, through university and then on into good-quality outcomes for them in the labour market, whether that is in a graduate job or in the decision to go on to further study. Access and improving outcomes for people from disadvantaged backgrounds is very central to how we are thinking about teaching quality and, yes, we are continuing to be very ambitious on our widening participation goals. As a Committee, you know that the Prime Minister has himself put out a goal for us, as a Department, to make sure we achieve and that is to double the proportion of people from disadvantaged backgrounds going to university by 2020 from 2009 levels.

Chair

You have mentioned a number of times that widening participation is central; your commitment to this is commendable, and what the Prime Minister is trying to do is commendable also. In respect of the link between teaching excellence and the raising of tuition fees, why do you have that? Why do you not have a link between the raising of tuition fees and how an institution widens participation?

Joseph Johnson

There is a number of different ways of how you can organise the sector. We want to make sure that we are achieving our widening participation goals and we have a system that involves the Director of the Office for Fair Access linking an institution’s ability to charge the maximum amount of fees, the £9,000 threshold, to their progress in bringing on and into their institution people from disadvantaged backgrounds. That is the link. It is the link there, mediated by Les Ebdon behind me. That is the way we are operating the fee system with respect to widening participation, so it is already hard‑wired into our system.

Amanda Milling

I am going to pick up on a term that Richard used, which was that there has been quite a lot of grumpiness. There has, but I think a lot of it has been about detail and mechanics. There has been a lot of sessions, as you have seen, where there is a lot of support and recognition that there is a need to enhance teaching excellence. I have a couple of questions, though. You will also have heard there has been a lot of discussion about diversity and that is in terms of the diversity of institutions—some of them are quite specialist—and also the student characteristics. We keep touching on this, but one question is about how the TEF can account for this. The other question is we talked a lot about metrics and you also identified there are qualitative aspects. I would like you to talk to us a little more about how this is going to work, so could you give us a bit more of an explanation?

Joseph Johnson

If it is okay, I am going to ask Ms Payne to do that, but I am happy to come back if you have any further questions for me directly.

Polly Payne

As the Green Paper sets out, we are initially proposing three basic metrics, which are the metrics we have across all institutions—they have been going for some time and are compatible. We are going to have those benchmarked, which is incredibly important both for the diversity of the institution and the diversity of students. The proposal then is for each institution to put forward evidence for why their particular form of excellence is indeed excellent and that will include any evidence they have. At the moment, there are some metrics that are not collected across the sector. For example, the UK student engagement survey, which not all institutions participate in, might add good evidence where institutions do participate. Institutions can also add any other information that they think is relevant. There is obviously a balance for how prescriptive you are and you do not want to be inundated with huge volumes of evidence, so there will be a question of how prescriptive that is or is not, but we hope to give institutions choice. We are consulting on that and will be taking advice also from our delivery panel.

Amanda Milling

Are you suggesting that there will almost be some kind of principles they might need to demonstrate to frame the qualitative aspects?

Polly Payne

Yes. The Green Paper sets out the aspects of teaching excellence that we will be focusing on, which are teaching quality, the learning environment, student outcome and learning gain. Those are the aspects we will be focusing on as TEF develops—TEF level 1, 2 and 3. As the Minister said, we see TEF very much as something that we will be evolving—we will be not running before we can walk. We will be listening to the sector and to other experts, and learning and evaluating the progress of TEF as it goes forward. We have started with the three core metrics, which are mostly based on what data is available and that we think is robust. The ONS is reviewing the robustness of that data. As TEF develops, we will incorporate more core metrics, which might be, for instance, learning gain, as the pilots develop, and also the institutional evidence.

Amanda Milling

The reason why I ask the question is, as you will have seen from the evidence sessions and the discussions you have had, there has been a lot of emphasis simply on the quantitative metrics, and it is just weighing that up with the qualitative aspects as well in terms of being able to demonstrate excellence. I just want to get a sense of the balance between the two.

Polly Payne

We have not yet. That is one of the things that we will be consulting on further and setting out our proposals in more detail when we do our technical consultation. I think the main thing to remember is that, ultimately, these will be human assessments by a panel of experts. There is no black box where you put in various numbers and out comes the magic answer. It will be very much human expert judgment.

Jo Stevens

Minister, can I just take you back to some of your earlier evidence, just to clarify it? You talked about assessing, under TEF, institutions as a whole and then you said, “In time, we would move to a course-by-course and faculty-line assessment.” Can I just ask you whether you envisage a TEF applying within individual institutions to a course‑by‑course or a faculty‑line assessment?

Joseph Johnson

Yes, in time, but not in the opening year.

Jo Stevens

By when do you think that will happen?

Joseph Johnson

We are consulting on this at the moment, but we anticipate this happening as the metrics develop in year two and onwards.

Jo Stevens

You do not have any date in your mind about when you would like it to happen.

Joseph Johnson

No, we are not fixing ourselves to a particular date at this time.

Jo Stevens

If the link then, as you envisage, between TEF and the level of fees applies, can you explain how that would work if you end up with a course‑by‑course assessment linked to TEF linked to fees?

Joseph Johnson

Again, that is not something that we are immediately embarking upon. It is some years into the future, but in the first instance what we are proposing is that any institution that, in time for the 2017-18 academic year, has met baseline standards for teaching and quality generally, as determined by whether they have had a successful quality assurance review, would be able to increase their fees only up to the amount of inflation. In subsequent years, when we are applying the TEF framework as the means of awarding inflationary fee uplifts, we will see how we can best allocate that on a course-by-course basis.

Jo Stevens

Do you think there is a tipping point, essentially, which affects widening participation? If fees go up, obviously you have performed well in terms of widening participation so far, do you think there is a tipping point at which that will then start to narrow?

Joseph Johnson

Obviously, we are sensitive to those issues. We are pleased that, as you said, widening participation has been a strong success story and a positive feature of the reforms that the last Government, from 2011 onwards, put into place. As you said, we now see the very positive story of record numbers of students from disadvantaged backgrounds going to universities. It is up at about 18.2% this year of students from disadvantaged backgrounds going to university—up substantially from where it was in 2010—and we want to see that progress continue. As I said earlier, the Prime Minister has set us a target of doubling it from the 2009 level by 2020, and we have every intention of achieving that goal. Of course, we are very sensitive to the fears you outline, but thus far we have not seen any evidence to suggest that people from disadvantaged backgrounds are being deterred from applying to university.

Jo Stevens

You do accept that there might come a tipping point.

Joseph Johnson

What people are seeing is that higher education is a transformational experience for many people. When they get great teaching and when they are properly supported at university, and when they get on and get good outcomes in the labour market, university can be a fantastic thing for people from disadvantaged backgrounds. That is why more and more people from these backgrounds want to go to university, and we entirely support their aspirations in that respect.

Jo Stevens

However, some of your evidence today has been about graduate‑level jobs and people not attaining those until three years after graduation. Therefore, they pay more fees, they do not get a graduate job at graduate levels so that they can pay their loans back, and we have the highest ever rate of debt repayment default. That is why I want to press you on this tipping point. Will the system collapse at some point? Do you accept there is a risk of that?

Joseph Johnson

On average, higher education is still a great investment. Lifetime earnings are £100,000 greater than if the same person with the equivalent A-levels did not go to university so, on average, it is a great investment. However, that is an average and I want to make sure that the system is delivering consistently good returns for everybody. Part of that is about the kind of teaching experience and learning environment that they have at university, and I want to make sure that everybody who goes to university has the best possible teaching and learning environment.

Jo Stevens

Just going back to some of the points that my colleagues have made about some of the witnesses we have heard being a bit grumpy about the proposal, do you think that might have anything to do with your comment about there being lamentable teaching in the sector?

Joseph Johnson

I think the sector, rightly, wants credit for the progress that has been made in recognising that teaching is an important issue. Since the arrival of the national student survey a few years ago, the sector has begun to redouble efforts on the teaching front and maybe there was a sense that their efforts were not being recognised. They are recognised, but I still think there is room for more rapid progress.

Chair

I have two very brief questions, Minister, if I may, before I bring in Amanda. Jo mentioned about that link on a course-by-course basis with TEF and with the ability to raise fees. Do you anticipate that that will happen in this Parliament?

Joseph Johnson

I cannot give any guarantee of that.

Chair

Would you like to?

Joseph Johnson

I cannot give any guarantee. We obviously want to make sure we are phasing in the TEF in a careful, thought-through way. That is why, as I said to Mr Blomfield, we have a three‑stage process of phasing it in over a number of years. We want to make sure that we are getting it right, that we are not doing anything that has unintended consequences, and we are consulting very deliberately at every stage of this process. This is an important thing to get right. Raising the quality of teaching to the best in the world across our sector is an important objective for us all and we want to work very carefully to get it right.

Chair

What is a graduate job?

Joseph Johnson

HESA has a definition; the ONS has a definition. There are widely available definitions of this out there. There is also a debate about it. The CIPD has a different definition from the HESA definition, which is why I cited that broad range of outcomes from 20% of graduates not getting a graduate job, according to the HESA definition, and as many as 58% of graduates not getting a graduate job, according to the CIPD definition. Somewhere on the spectrum lies an important number for Government policy. We need to focus on this issue and look at why so many people seemingly are not getting jobs that correspond to what they would expect, given their investment in qualifications and in skills.

Amanda Solloway

That was exactly what I was going to ask: how the TEF will help with the skills mismatch, as an example, because if the TEF is about content, how will it lead on to getting a job in that particular field?

Joseph Johnson

It will address squarely the skills mismatch, because a big part of it is going to be about making sure students are properly informed about what is on offer in a particular course at a particular institution: how it will be taught; what the mix will be between lectures, self‑study and other forms of teaching; what the outcomes have been of people who have previously done that course; how they have tended to go on in their careers; and so on. There will be a lot more information that will enable prospective students to make really well-informed choices. In that sense, better information will be a very important way that the TEF contributes towards addressing the mismatch that we see in the graduate labour market, but it is not the only way. We also want to see very strong employer engagement with universities coming through as a feature that the TEF will reward and recognise. We want to see a stronger role for accrediting bodies working in universities, so that more courses are clearly endorsed by accrediting employer and industry groups, so that there is demand for them from the providers of jobs in the labour market. We see that link as also being helpful in terms of reducing the labour market mismatch.

Amanda Solloway

As an example, my background is manufacturing, and one of the things that we used to see was a lot of students opting into a very good design course background. However, we had very few designer posts, but we would have things like merchandisers or possibly buyers, so you are saying that could be addressed at the outset. In other words, it is managing the expectation of the students, saying, “You will be doing this. This will be part of the learning experience. However, at the end of it you will not be just getting to be a designer.” Is that what you are saying?

Joseph Johnson

We want to see more employer engagement with universities in helping them to think about curriculum design, helping them think about how they can get their courses accredited so that there is more confidence in the qualifications that people are coming out with—that employers really recognise them as being a true signal of future productivity. In that sense, we think that will be helpful to students. They will be emerging from university with degrees—pieces of paper to which employers attribute real and lasting value. In that sense, the closer engagement between universities and employers will be really helpful to students.

Polly Payne

Where the information helps us identify particular areas where there is, possibly, a greater skills mismatch or employment outcomes, we can take other action. For instance, at the moment Sir Nigel Shadbolt is reviewing computer science graduate employability, because that is an area where there seems to be a skills mismatch and poorer employment outcomes than you would expect. Therefore, he is reviewing that particular area and working with employers and universities to see what can be done.

Chair

Minister, how will the TEF take into account local economies and the distinctiveness of the regions? Two LSE graduates, one staying in central London, one going to Hartlepool: they will have different salary expectations and different experiences. How will the TEF take that into account?

Joseph Johnson

On the salary point, the employment outcome is not going to look at the salary level per se. We just want to see whether an institution is doing well or not at getting people into graduate jobs. We are not trying to measure a university’s teaching quality by whether it has a very high proportion of people who go on to earn very high salaries. That would not be very helpful. In that sense, we are just focusing on whether they are getting into a graduate job or not or going on to further study. As my colleague Ms Payne was saying earlier, all these factors will be contextualised, and background factors will be taken into account and benchmarked in the TEF exercise, so the regional variations in employment, for example, would be considered in whether an institution has achieved success or not against that particular benchmark.

Paul Blomfield

In a very powerful section of his speech to the Conservative party conference in September, the Prime Minister made the point that too often, for too many people, the ability to get a certain job was influenced by the fact that they might have a foreign name, or the colour of their skin. Was he wrong?

Joseph Johnson

I think we have taken steps to address subconscious bias lately. It is certainly a possible factor and you will have seen that we have asked UCAS and the sector generally to work closely on introducing a system of name‑blind admission.

Paul Blomfield

With respect, Minister, my point is about graduate outcomes. Was the Prime Minister wrong to say—

Joseph Johnson

No, he was not wrong. That is why I was giving the example of what we have done in the university sector, but I could have given a much broader example of what we are doing across the public sector and across the civil service, and also a number of companies have come on board as well behind that initiative, to address issues of subconscious bias.

Paul Blomfield

My point is that if we all agree with the Prime Minister that your race, your social class or other factors are significant in determining job outcomes, how useful is it to measure teaching quality by job outcomes?

Joseph Johnson

There are many drivers of outcomes. Teaching quality and the quality of the higher education you have experienced is going to be an important factor. Obviously, you are then going to encounter in life many other factors that determine your success, and encountering other people’s subconscious bias will be among them. However, it does not mean that we should not ensure that people at university are getting the highest quality teaching that they possibly can and we want them to.

Paul Blomfield

I think we are all agreed that people should be getting the highest possible quality teaching, but the question is: is that an effective measure of teaching quality or is it a measure of those wider factors that you were talking about and that the Prime Minister was highlighting?

Joseph Johnson

As we have said in answer to other questions, we will be taking into account background factors, such as if you are from a BME background and likely, according to the concerns you are raising, to encounter some of these subconscious biases, we would be taking them into account with the background and contextualisation, and controlling for external factors that we will be doing systematically when we look at these metrics. However, again it is worth stressing that the metrics are just one part of the TEF exercise. They are just the quantitative part and they give you a first cut of information. They are then supplemented by the qualitative information. All those pieces of information are then assessed by a panel of expert human beings who can take a rounded judgment. Therefore, we are not robotically relying on one metric to assess teaching quality.

Paul Blomfield

As the discourse around the metrics has gone on, there does seem to be an increasing consensus about the importance of the qualitative aspect. Is that your view as well?

Joseph Johnson

Yes, it has certainly been a strong feature of the responses that we have had in the Department from all our stakeholder engagement meetings that you cannot do this by metrics alone. We are in agreement with that and we have never proposed a system that is solely metrics‑based. We have always accepted that it would have a qualitative component and we recognise the importance of that.

Chair

Following on from Paul’s point, Minister, how would you respond to the letter that I think went to HEFCE from 169 academics, which said it would be “completely inappropriate” to use student outcomes? Do you agree, or are they wrong?

Joseph Johnson

No, I do not agree. As a basket of indicators that are correlated with teaching quality, I think it has its place. It is not the sole metric, as I was saying. It is one of a basket of metrics that itself will be supplemented by qualitative information considered by an expert panel of reviewers. It has its place. It does not have a dominant place, but it makes a contribution to our ability to make an assessment.

Chris White

In the discussion, we appreciate that the TEF is going to evolve, that there will be a number of phases, and that consultation is taking place. However, I just wondered, if you were back in front of the Committee in five years’ time, what would your metrics be?

Joseph Johnson

I think you are right; it will evolve over time, just as the research excellence framework, previously the research assessment exercise, has evolved since Mrs Thatcher created it under Keith Joseph in the early 1980s. It has gone through many iterations over the last 30‑odd years, and we are now in I think REF 6.0 or 7.0, and each one is different from the last. There is a process of continuous improvement and evolution, but the result we can see on the research side has been that the quality of our publicly funded research in this country has increased tremendously, and we now have one of the most effective and efficient research bases in the world. We are world‑leading in science and our system of funding excellence very rigorously is a part of that. We want to see and expect to see a similar evolution of the teaching excellence framework in the years ahead. We can already anticipate some changes that will come in years to come. We will have a greater ability to use the matched data from other Government Departments in future years that will give us more granular information about the relationship between the choices people make at secondary school, and their outcomes later in life and their passage through university. As has already been mentioned, in a few years’ time we will also be able to learn from the learning gain pilots that HEFCE is now running at 70 institutions around the country. We can already see, in the future, ways in which this might evolve and those look to be positive, but that does not mean we are going to stand still and wait for them to arrive. We need to crack on and we should not let the best be the enemy of the good.

Chair

Minister, are you trying to do too much?

Joseph Johnson

An important thing to do as a Government is to make sure that students are getting the best-quality teaching. It is a very important part of our higher education system and we have to ensure that universities are fulfilling their potential as drivers of productivity and engines of social mobility. They are doing well, but they can do even better.

Chair

The reason I ask is, on the one hand, you are changing quite radically the higher education architecture: changes to HEFCE, establishment of the Office for Students. You are changing quality assurance and establishing a new teaching excellence framework. You are clever—you are the nice and clever Johnson, if I may say so—but are you that clever?

Joseph Johnson

First of all, we are consulting. Nothing is being rushed and we will proceed when we are confident we have the right answers.

Chair

Would it not be better to choose one, whether you change the institutional architecture or you put in place the priority regarding teaching excellence? Why do all this all at once?

Joseph Johnson

The system is a coherent whole and it needs to be looked at as a whole. It is difficult sensibly to do bits and pieces without thinking of the interactions between the different parts.

Chair

How do you stop the whole teaching excellence and quality assurance regime from becoming big, bossy and bureaucratic? How do you make sure that it adds value but remains light touch and risk‑based?

Joseph Johnson

That has always been our goal right from the start: to ensure that we look at this not as a mirror image of the research excellence framework exercise, which has encountered some criticisms for the institutional and individual burdens it imposes on universities. To that extent, we do not look at the TEF as a mirror image of the REF. We always have said we want it to be light-touch in spirit and that means the outcomes‑focused metrics is one aspect of it, with universities having the opportunity to supplement them with additional information, and then a panel that reviews it. We think it is proportionate to the objectives of the whole exercise and so, to that extent, I do not think it is going to be bureaucratic—i.e. disproportionate relative to the gains it is seeking to achieve.

Chair

Minister, Ms Payne, thank you very much. We are very keen to help and to add value by making meaningful recommendations, because I think we have a shared objective, which is that while we have an incredible asset for Britain in the higher education sector, with really excellent teaching, we want to make it even better, so thank you for what you are doing. We hope to help you as much as possible.

Joseph Johnson

Thank you very much. Oral evidence: Assessing quality in Higher Education, HC 572-iii 2