Oral evidence: Robotics and artificial intelligence, HC 145Tuesday 24 May 2016
Oral evidence, 24 May 2016.
Witnesses
Richard Moyes, Managing Partner, Article 36, and Dr Owen Cotton-Barratt, Research Fellow, Future of Humanity Institute, gave evidence.
Chair
I welcome both of you to the table. You sat through our first panel and heard the questions asked, some of which were earthbound and some not. Can I start by asking both of you the same question I put to the previous panel? How close are we to seeing intelligent general purpose machines that can handle complex problems, and what would be the implications of that for society?
Dr Cotton-Barratt
That is a great question. It is extremely hard to have confidence on this question. It seems that over the next five years we will see significant advances—I agree with the previous panel—that will very likely bring large benefits to society. Over the long term there is the hypothesis that we may at some point see artificial intelligence get up to human level in almost all domains. It is hard to say when that will happen. There have been a number of surveys of people who work on AI to try to get their estimates on this. The estimates they give are surprisingly close. The average expert, across a number of surveys, tends to give the date by which they think it is as likely as not as between 2040 and 2050, and says there may be a 10% chance of seeing it within the 2020s. Maybe there are elements of optimism because they are the people choosing to work in this field, but they know the technology better than outsiders and we should take their judgment seriously.
Chair
Richard Moyes, the potential for artificial general intelligence in the 2020s is 10%. Stephen Hawking and other scientists wrote an article in which they said that AI was potentially the best or worst thing that could ever happen to humanity. That is quite a polarised position to take. Do you think it is a helpful way for the debate to continue? If not, how should we be thinking about it?
Richard Moyes
For a start, I have no idea of the future of general intelligence development; it is not my area of expertise. It is very important that people are debating and considering that slightly longer-term trajectory of where the technology is taking us, but in our involvements in these issues we are focused much more on the role of specific artificial intelligence and algorithmic systems within weapons systems as we see them developing today. We are focused much more on how computers, sensors and algorithms are being incorporated into weapons systems more and more, and the implications that has for the development of weapons in the comparatively near term. In many respects, that is quite a separate issue from the implications of broader generalised intelligence in society.
Chair
The principles that stem from it are the same, whether we are talking about artificial general intelligence and its implications for society or about specific applications that have ethical concerns. Is there enough serious research going on into the ethical implications and how legislators and regulators should be responding?
Richard Moyes
I do not know about the general framework. Our work is focused substantially on the UN convention on conventional weapons, which meets in Geneva under the framework of a treaty that basically has the power to regulate and prohibit certain weapons. Under that framework, delegations from different countries come to the table and bring their perspectives to bear on what controls, or lack of them, they think should be enacted in relation to weapons systems with developing autonomous capacities. Looking specifically at the UK’s engagement in this area, there would be value in seeing whether the specific types of policy position that the UK has been taking in that context with respect to weapons could in some way be integrated more, perhaps with broader consideration around how these capacities—computer autonomy, AI and algorithms—are going to be managed in other areas. From our perspective, we have some concerns that we might take a rather more permissive attitude in the context of discussions on weapons than we would want to take in other areas of society where perhaps we imagine the risks bear more directly upon populations to which we are accountable. From what we see in relation to weapons, there is certainly scope for more integration of thinking and perhaps broader policy oversight than we are seeing in relation to the Ministry of Defence on its own.
Dr Cotton-Barratt
On the question of more general and powerful artificial intelligence and its impacts on society, I agree with the previous panel that in decades to come we may see that everywhere in life. If it happens it could be very transformative, and that is the thought behind Stephen Hawking saying this could be the best or the worst thing ever to happen to humanity. It could also be somewhere in the middle, but if it is to transform the way our society works, it is extremely important. Even if it is still some decades off, it deserves attention and our trying to understand what could be good and what could be bad about it. Are there any ways to make it more likely that we get the good rather than the bad outcomes? There is a small but growing research community looking into these questions. I think the UK is world-leading in this at the moment. The Future of Humanity Institute, where I work, and more recently the Cambridge Centre for the Study of Existential Risk are looking at these questions. There is also the Leverhulme-funded Centre for the Future of Intelligence, which I think will involve people from Oxford, Cambridge, Imperial College and Berkeley. I do not know of any other academic centres elsewhere in the world looking at this. Right now we have the intellectual leadership, and I think that over the coming years and decades that will more and more become an extremely important area, and it will have to tie up with the practitioners of research into artificial intelligence.
Chair
We are talking now about research-level ethical questions and long-term questions, but in practical terms what are the immediate questions for legislators and regulators that you think we should be thinking about?
Dr Cotton-Barratt
It is too early to have a definitive answer for many of those long-term questions, and as legislators you should not be trying to create one. However, when we are making rules and setting up systems today to deal with the incoming improving technologies that we are going to see over the next few years, we should try to make sure that they remain robust to possibly quite radical improvements in capabilities in the systems. We should not just look for a solution that will work now, but try to imagine whether, if we have much more powerful systems, that is the way we want it to work in our society. There can be a bit of stickiness in the way things are set up. If we have good direction when we are setting out on that, it might help to make sure we remain well placed in the future, and may also serve as a lead that other countries can follow.
Dr Mathias
The Ministry of Defence says that the operation of UK weapons systems would always be under “human control”, and Article 36 says that this should be “meaningful human control”. Can you explain what that is, and why you think you need to make that clarification?
Richard Moyes
Thank you for the question. The word “meaningful” is there to indicate that we need to debate and define the form of human control we are talking about. When we talk about complicated technological systems, we cannot just take human control as being self-evident in some way. There are a number of areas where we can imagine different examples. If a weapons system is operating in a situation where a human defines the specific object it is going to strike against, and if it is activated and strikes that object, we can be fairly clear that there is human control. If there is a range of objects of a similar type and a human stipulates a category of object types within a certain area, and the weapon is activated and strikes at objects of that type within that area, there is a form of human control but it is perhaps less specific than in the first example. Similarly, you can step further back and say that the weapons system has a broad category of possible target types programmed into it; it is set out across a broad area to operate over a longer period of time. Now, all of a sudden, where the actual physical force is going to be applied in that situation is rather unpredictable in the context of the commander who has initiated the attack. In that situation we are seeing a different form of control from that in the first example. All those examples are relatively proximate to certain types of existing systems. We see different levels of control being enacted, so within the international community the issue of autonomous weapons is being debated in Geneva. We are trying to build a structure to that conversation that is focused on thinking about the human elements of control and human judgment in control that we want protected in the development of these technologies. The term “meaningful human control” is a political policy device to try to get a number of actors on the same page and build up a conversation about where the threshold of control should be. We have written some materials on the types of factors that bear upon control, some of which are in the technology itself in terms of its reliability, predictability of operation and perhaps transparency—whether you can see how it works—and some of which are information based and are about what the commander knows about the area where the system is operating. That bears on predictability and constraints in the area of operation, and the systems of accountability within which those uses of force are taking place. We think you could lay out a fairly clear framework of key elements of human control. We would like to see the international community doing that together as a collaborative process of policy development. We would like to see the UK encouraging that and participating in it, but as yet it has been a little reluctant in that respect. We are hoping that next year, when perhaps a more formalised group of governmental experts discuss the issues, we will get the conversation focused more on laying out what we think are the elements of human control that we want to protect as technology develops in the future.
Dr Mathias
The UK Government also say that humanitarian law is sufficient for the legality of weapons systems and we do not need separate legislation. Why do you think a separate ban is needed?
Richard Moyes
Separate but perhaps still within the framework of international humanitarian law. We are certainly not looking to reject the utility of the humanitarian law framework. Of course, technically it is applicable only in situations of armed conflict, and we need to be aware of that as a legal specificity. Our particular concerns are that, as it stands at present, international humanitarian law regulates the use of force in the framework of attacks, where attacks are particular applications of force for military objectives. It may be that more than one weapon is used; it is not a single bullet fired at a single person, but it is a relatively contained application of force. Military commanders have legal obligations around attacks to make certain judgments as to whether or not an attack is proportionate, and other such rules. One of the things we are concerned about is that, as these technologies are able to operate over wider areas and perhaps longer periods of time, the concept of what is an attack gets bigger and bigger. Coming back to the objects on the table example, we might understand that attacking a relatively small group of vehicles close together is conceptually a relatively coherent matter. If we then imagine vehicles spread over a much larger area of a country, or across towns and cities, the use of a weapons system can progressively move from one to the other, striking against them. That seems to be a much bigger version of an attack. We are concerned that a military commander in that context does not have the contextual understanding to make informed judgments about where specifically force will be applied and what the likely implications of that are going to be. By stretching out the concept of attack we are stretching the structural fabric of the law itself, and that is something we should be very wary of. We should be pulling more towards regular human/legal judgments about the application of force in conflict situations, essentially to ensure that we are applying that human judgment on a more rigorous and regular basis.
Dr Mathias
It implies a lack of confidence in the military.
Richard Moyes
Not so much in the military. Unless we develop common understandings of how these things are to be interpreted, the whole international community has the capacity to interpret the legal frameworks differently. Already under international humanitarian law you get quite different interpretations from one person to another and from one country to another. It is quite an open, flexible framework. We feel that developing specific rules in relation to the role of autonomous weapons systems would allow us to ensure that all countries are approaching this from more or less the same perspective. There are still likely to be some fuzzy lines in these areas, but we think that a legal framework would get states more clearly on the same footing. New legal rules have a communicative function, which will help to guide where research, technological developments and commercial interests go, as well as bringing a wider sense of what the public and others consider acceptable behaviour in conflict situations—a sort of stigmatising standard-setting function. We think that all those things can come from new law rather than just relying on the existing legal framework.
Dr Mathias
Acceptability is not going to change, whether it is autonomous or non-autonomous.
Richard Moyes
When you start to think about machines making decisions about who should be killed or where force should be applied, quite a lot of people feel slightly instinctive revulsion or rejection of that notion. There are all sorts of issues in considering what are decisions and the like, but we often find a general sense that having machines making decisions about who gets killed and where force is applied in context makes people queasy about the direction we are taking in relation to technology and society. When I talk about acceptability around those sorts of issues, I mean that we want to strengthen, from our perspective, the sense that having machines making decisions about life and death matters for people is not something we want to encourage; we want to try as much as possible to keep that as essentially a human function.
Dr Mathias
Do you see any benefits from an autonomous weapons system? If we talk about friendly fire, collateral damage and more targeted offence, do you see that as an advantage?
Richard Moyes
There are potentially significant advantages in autonomy and sensors in computers in relation to weapons systems. I do not generally think of myself as a proponent of weapons development and the like, but the key issues we are concerned about are the identification, selection and application of force to targets and machines making that kind of choice, but, at the same time, we can see in weapons systems a capacity for more accurate application of force more narrowly to specific objects. As long as a human is in the background applying it, potentially it has very positive impacts for the surrounding population.
Dr Cotton-Barratt
This might be a place where you try to take a long-term perspective as well and decide how we want our society to be organised around this. The question of whether or not we allow machines to make decisions about taking lives may be a natural bright line; it could be a natural stopping point and we should think carefully before crossing it, rather than saying that in the short term it looks as if it might be beneficial, because from the longer-term perspective it might be the better place to stop. It is a complicated issue. I am not sure.
Graham Stringer
When you are dealing with systems that can adapt, learn, plan and change what was intended, how can you verify that those robots are doing what was originally intended?
Dr Cotton-Barratt
That is a great question. There should be more research on exactly that question. This is one of the potential challenges to the future deployment of these systems. At the moment, the systems tend to be reasonably simple, relative to what they might be in the future. We can understand the underlying algorithms but not how they are making decisions. One of the previous panellists said we do not really know how the machine was better than the best human Go player. At a high level we can say that is how it learns, but we do not know how it thinks about the game of Go. If we can build better tools for understanding that as it proceeds, it is more likely that we will be happy deploying even very advanced systems in the world and giving them autonomy.
Graham Stringer
I am not sure I completely understand either the questions or the answers. Can the workings be transparent if it is not possible for humans to determine why the system is reaching that decision? How can you have accountability and control of a system if you do not know why and how it is coming to its conclusions?
Dr Cotton-Barratt
It is a real obstacle. There are degrees of transparency in terms of who it is transparent to. Is it transparent to the people who wrote the computer program? Is it transparent to the people who are using it? Is it very publicly transparent so that anybody who wants to understand the system can look it up? There are also degrees of resolution and how fine-grained is the understanding of its thinking. To the extent that we can improve and push up our ability to understand that at a fine-grained level, we can have more confidence in these systems. That was the direction Professor Muggleton said was good, and one we should be exploring further. I absolutely agree with that.
Richard Moyes
From our perspective, when we are pushing back a little on technology developing in a certain way with respect to weapons, it is sometimes presented to us as if technology is on a relentless track in a particular direction and we have no power to move it either way. Of course, it is important not to buy into that approach, but it makes me wonder a little bit whether some of these factors may produce something of a natural break or a natural steering in certain respects. If within a military framework we are going to ensure that somebody is accountable for the use of a system, how they are accountable—the way accountability bears on them if the system is not transparent—may become a problem for them as an individual. The key issue is that, as long as we enforce accountability, maybe people themselves will say they do not feel comfortable being held accountable for a system when they cannot quite understand its functioning and cannot be completely sure what it is going to do. This links a bit to issues about building trust in technologies. Military actors like to trust their equipment; they are in situations where they depend on it, and systems that are perhaps too complicated to understand and are slightly unpredictable in their behaviour may not engender trust within those institutional frameworks. I am not suggesting in any way that we should be relying on those forces to put in place the best policy responses, but there are some factors that will perhaps make these systems problematic in an operational context.
Graham Stringer
If somebody sets off a weapons system and it kills a lot of people and it was the weapon that did it, not the person, that is a very clear example of the real problem in terms of the law and accountability. Do you think there is a similar problem when it comes to intelligent learning machines carrying out surgery or medical procedures? The machine may make a decision, but the patient cannot give their permission for that decision because they cannot have an interaction with the machine. They can have an interaction with the person who sets off the machine, but if there is no transparency do you think it will limit the practical applications?
Dr Cotton-Barratt
It could.
Graham Stringer
That is a political answer.
Dr Cotton-Barratt
People may be unhappy with it in some cases. People may be willing to make trade-offs if the statistics show that it is safer to have opaque systems that are automated than not to have that automation, but there is a cost. People may be unwilling to take it, and we should perhaps be unhappy with them taking it if they do. This supports a push towards developing meaningful transparency of the decision-making processes, as well as just developing better ones.
Chair
It is not just about transparency. Presumably, not many of us will understand all the algorithmic processes and decision-making functions, but we might learn to trust an active verification process that stands between us and a machine learning system that gives us confidence that the system is going to do what it says it will do and it has not learned to do something different. How far off realistically are we from having anything like that, and who is working on it? Are we world leaders in that as well as in ethics?
Dr Cotton-Barratt
In my mind, that is another form of transparency. It seems like a useful kind of transparency tool that does the interpretation and turns an opaque system into one you can understand. Maybe you do not understand all the details but you understand some of the salient high-level ones, and that it will do what it says it is going to do. There are a few people researching this at a broad level. One can try to study the question in two ways. One can look at what we understand about automation as a whole and what properties are going to be needed. One can also look at what machine learning systems people are building today, and how we could implement this on top of that. I do not know enough about how much research of the latter type is being done. Of the former type, I think there is a little but not much and there could be scope for a lot more.
Chair
Is it too early, or is it just that not enough people are doing it?
Dr Cotton-Barratt
It is mostly that not enough people are doing it. It has an odd property. For most technologies I think it would be too early. The closer you get to having the technology, the easier it is to do the research, because you can work with things that have been built and understand how they work. It is only because of the potentially extreme transformational nature of this technology that it seems important enough to try to design ahead for how to interact with more powerful systems.
Chris Green
What levels of autonomy should be permitted in robotics and AI systems? I think we have covered bits and pieces, but what level of autonomy should be permitted?
Richard Moyes
Characterising and structuring it in terms of levels could probably be done in various ways. I am not sure I am in a position to lay that out. From our perspective, the key issue of concern is the selection of targets against which force is applied. On autonomy of movement of a system to a target, I feel relatively comfortable. I am sure some issues can arise from that, but the specific focus of our concern is the identification and application of force to the target being in the hands of the weapons system. I am not quite sure how that is characterised as a level.
Chris Green
If you are going to deploy a weapon, should there always be a human in the loop?
Richard Moyes
Yes. From our perspective, we think a human should be specifying the target against which force is to be applied. To give a bit of breadth, that target may be a group of objects; it may not have to be a completely unitary object. We accept that in the use of force you can apply weapons systems against targets comprised of various objects. There is some potential for breadth, and within that process a machine may apply itself most efficiently to those objects, but there have to be some boundaries to the concept of a target.
Chris Green
Would we be more comfortable saying that artificial intelligence could restrict what we do—a version of the dead man’s handle in a train? If we can apply that, would we be far more comfortable about it, or does it have the same ethical or moral considerations?
Richard Moyes
Those are interesting questions. It is possible to feel more comfortable with that. From a personal internal sense, if a weapons system was being applied to a target, and on seeing a red cross on the roof of a vehicle in the immediate vicinity of its detonation it deactivated itself, I would struggle in moral terms to see that as a problematic aspect.
Chris Green
In that kind of instantaneous decision, an automated system would be able to identify it and switch off faster than a human could react, so you give control in that sense.
Richard Moyes
But there are slippery slope issues in these areas. You would need to frame it carefully, but there is potential.
Dr Cotton-Barratt
More broadly, looking beyond weapons systems, when considering what level of autonomy should be required, this is definitely not a question we are going to be able to settle now. I see two main forces that affect the level that we think is appropriate. One is what is safe with the current levels of technology. We are coming to the point where it seems that we can have autonomous cars, that it is safe and it is a reasonable thing to allow. Ten years ago that would not have been a reasonable thing to allow, because the technology was not there and it would have been damaging. There is also a kind of limit. If we imagine much more capable systems, do we want full autonomy? There might be a reasonable principle that, whenever an important enough decision was being made, the human should be meaningfully in the loop. You can make autonomous decisions for small things, but for larger plans you should meaningfully consult a human.
Chris Green
I do not know how a car would in any way have the morality of a human, but in years to come we would want a value alignment in some way between the system and our moral codes. How would you go about formulating and designing that?
Dr Cotton-Barratt
That is another big question. A lot of the people who to some extent are concerned about the longer-term impacts of artificial intelligence focus on researching exactly that key question. When systems are powerful enough that we cannot necessarily understand and monitor everything, and they may have large impacts on the world, we still want to make sure that the actions they take are aligned with our interests. People are currently at the stage of building research agendas to try to address that question. There are a number of different ways to try to break it up and say, “If we answer this question and this question, it looks like we’ll be in a better place.” I think it is still early days for that area.
Chris Green
We need to make sure that computer scientists and engineers and social scientists increasingly work together so that as these technologies are being developed the moral perspective is there, but also that the moral characteristics of artificial intelligence are developed at the same pace. We do not want what the social sciences can bring to be one, two or three steps behind.
Dr Cotton-Barratt
I agree.
Chris Green
There was reference earlier to “Terminator”, so I will throw in “Demolition Man”. There you had artificial intelligence, which was activated by the Government of the day and had certain constraints. If people swore or behaved in a certain way it could restrict their behaviour; they could be fined or punished in one way or another. If you constrain people’s activity with artificial intelligence, monitoring what they are saying and doing, at some stage you get to 1984 and limit what people say and, therefore, what they think. We are a long way from anything like that, but it must be one of the questions you are considering.
Dr Cotton-Barratt
It is a real concern. We have a robust democracy in the UK and I am not too concerned about that happening here, but in some countries if leaders had access to technology that would enable them to do that maybe they would use it in that way.
Chris Green
Surely, you can see such an application at the moment. Not wanting to go into the Regulation of Investigatory Powers Act and that kind of thing, you can see how a Government would want to be concerned about certain words and activities, and for the state to clamp down on them and constrain them. Once it has started in one area and it can deal with that group, what about other groups? It is easy to see that slippery slope.
Dr Cotton-Barratt
It is a big question. It is a political question and one on which I do not have that much expertise. There may be technical tools people can develop that allow the monitoring of things while maintaining privacy in a meaningful way.
Chris Green
We always have to be sceptical of politicians and ensure that the Government do not have too much power in this regard.
Richard Moyes
Linked to that, again in relation to weapons stuff, one of the things you see in those systems is necessarily an encoding of objects in the outside world into categories of target or not target. Obviously, it seems less problematic if we are talking about armoured fighting vehicles, but if we start to move to a situation where we are encoding people with certain appearances or behavioural patterns, all of those things represent quite a pernicious process of reducing people to data points and then bureaucratically administering force around them. We are certainly not there yet, but that underpins some of the anxiety about how we start to encode ourselves and society in particular fixed ways. We have seen in the past that, where bureaucratic structures begin to encode people in those kinds of ways, it is potentially highly problematic.
Matt Warman
This question is related, but different. With the recent headlines in the Daily Mail about Google getting access to the Royal Free’s data, there is a sudden sense that AI is a mainstream concern but perhaps not as well understood as it should be. Do you think that AI in principle poses new challenges to things like patient confidentiality that we have previously taken for granted, or is it the same rules but just with a different set of tools?
Dr Cotton-Barratt
It may increase challenges. I am not sure whether in the case of this particular data there is any AI involved, but in the future there may be challenges coming from it. There may also be large benefits. If it can automate the processes and increase consistency in judgments and reduce the workload for doctors, it could improve health outcomes. To the extent that there are challenges, essentially it means there is less privacy from the same amount of shared data, in that people can get more information out of a limited amount of data. We will want to find ways to handle those challenges, and that may be about making sure that the data is held in the right places and is properly handled and controlled. I do not know the details. My understanding from the outside is that they are very much aware of it and want to make sure that the data is held securely.
Matt Warman
Is it in a sense less worrying if only artificial intelligence has looked at, say, patient data rather than human intelligence?
Dr Cotton-Barratt
It may be. It depends on how the artificial intelligence has been set up and what it will do with the data at the end of it, but if it is just kept internal and never reports to any humans the confidential data that is useful for the algorithms, there is no scope for anybody to abuse the data.
Matt Warman
Who should be responsible for drawing up the rules on whether it should be kept private?
Dr Cotton-Barratt
I do not know enough about good ways to reach a robust conclusion on governance. You will want to get a set of rules that industry and the NHS have signed up to and that the public are happy with, but I do not know what the best process is for producing such a set of rules.
Matt Warman
The Government are going to set up a council for data science ethics. Do you think in that sense that we need a council for AI ethics to try to work out those sorts of problems?
Dr Cotton-Barratt
Data science is a broad category, but it involves a lot of machine-learning algorithms that are the same kind of algorithms as today’s AI algorithms, so the ethics for data science in this case may be about the right level of foundation for it now. Maybe in a few years, if AI becomes more sophisticated, we will need to revisit that question.
Chair
The message we have received quite clearly from both panels is that we are at quite an early stage of development with a lot of these technologies and we do not know where this revolution is going. If I can update some of our cultural references in the Committee, there is obviously quite a lot of misunderstanding among the public and low levels of public trust in AI and robotics, some stemming, for example, from “Captain America: Civil War”—just to show that I am up to date. What do you think the Government, or other bodies, can do to try to help prepare society for the changes that are very likely to be coming as a result of these technologies?
Dr Cotton-Barratt
When establishing guidance or legislation for the nearer term, the Government can try to look out to the longer-term impacts and improved capabilities and bear them in mind, and design with them in mind. The Government can try to support more work in understanding the eventual impacts. There is quite a lot of uncertainty. Maybe making machines act in a way that is aligned with human interests will turn out to be easy; maybe it will turn out to be hard. Some reasonable people think both, but I do not think we should just assume that, therefore, it is easy. The Government could do that by supporting the academic research community or perhaps by internal studies. The academic community is going to include social scientists as well as AI scientists, and there will be legal questions as well. There are a lot of questions. The more we can resolve them in advance of having to have answers, the more we can check that the solutions we have are robust.
Richard Moyes
In very concrete terms, we would like to see the Government over a longer period of time supporting more formal discussions in the convention on conventional weapons and lethal autonomous weapons systems. We would like to see the Government using that as an opportunity to articulate and develop thinking around the forms of human control that should be considered necessary in the use of force. It would need a little bit of preparatory work at national level before we were in a position to do that, but there is an opportunity for the UK in a diplomatic landscape to have an influential position on how we orientate the role of computers in life and death decisions. We should take it, rather than watch the debate drift away from us.
Chair
I thank both of you for the time you have taken to give evidence today. This is a complex and challenging area, and we are grateful for the time you have taken to take us through it. We may well come back to you with questions to clarify different points. I hope you will respond to us. Our next evidence session will take place during UK Robotics Week, very appropriately, but for now I draw this session to a close. Oral evidence: Robotics and artificial intelligence, HC 145 10 [1] The witness later clarified that “The RAS Strategy was developed by the RAS Special Interest Group coordinated by the KTN with support from Innovate UK and EPSRC. The letter from the Rt Hon Greg Clark MP to Professors David Lane and Rob Buckingham (March 2015) provides a detailed response and makes a number of recommendations.”