Oral evidence: AI adoption, safety and governance, HC 689

Oral evidence, 13 October 2026.

Witnesses

Lord Booth-Smith, Tom Duff Gordon, Owen Larter and Rob Sherman.

Chair

Welcome to today’s second panel on AI safety and governance. Thank you very much indeed to our witnesses for joining us today. I know that the parliamentarians among you will need no reminding, but to give false, misleading or incomplete evidence to a Committee is, on the face of it, a contempt of the House. Let me start with some of the warnings that we have received from the leaders of your organisations. Mr Altman told the US Senate in 2023, “My worst fears are that we…cause significant harm to the world.” The leader of Anthropic went on the record in 2025 to say, “I think there’s a 25% chance that things go really, really badly.” Mr Zuckerberg said in 2024 that individual bad actors may be able to use the intelligence of AI models to fabricate entirely new harms. Demis Hassabis has said that advances on the frontier are now outpacing our understanding of the technology. Nobody in the world knows for sure what is going to happen from here. Can you assure the Committee today that your models can do no harm to people or businesses in our country and will do no harm in the years ahead?

Tom Duff Gordon

Good afternoon, Chair. Good afternoon, members of the Committee. We take safety incredibly seriously at OpenAI. We listened very carefully to the previous panel. Our position at the moment is that we should not progress with AI as fast as we can make it and develop it; we should progress AI as fast as we can keep it safe. Those are not just words; they are also deeds. During the summer, after the Hugging Face incident, we paused the training and reinforcement learning of our most advanced model. Just recently, we held back a model—Astra 6.1—that we were due to release. We say that we want to move at the speed where we can make sure that capabilities are not outstripping safety, and we are matching that with what we do. We have very significant processes internally to ensure that we manage risks. We have a preparedness framework.

Chair

Tom, that is great, but it is not the question I asked you. The question I asked you was whether you can guarantee that today your model is not capable of being used to cause harm to people or businesses in this country.

Tom Duff Gordon

Yes, I can reassure you that we are confident in the processes that we have internally to ensure that we manage frontier risks, particularly risks with regard to cyber, bio and loss of control. We have a preparedness framework and will not deploy a model unless it has passed that framework. I can assure you that our deployed models are safe and do not pose a risk to the UK.

Chair

Lord Booth-Smith, can you give the same assurance for Anthropic?

Lord Booth-Smith

We have rigorous safety processes in place. We were founded as an AI safety research company. Alongside those safety processes, we also believe that we need to pace the frontier. We consider deployed models safe. They have mitigations in place. It is also worth pointing out that we are learning a lot as we go about how these models are used. It would be wrong not to say that. As new issues emerge, that is something that we believe should be publicly shared and debated. We factor that into all our future thinking about safety research.

Chair

You think that harms could be caused.

Lord Booth-Smith

We have seen in the last six months some instances where models have done things that we would not have wanted them to do. Safety is stratified, as to how one would determine that, but we have seen models do things that we would not have wanted them to do. I believe that it is entirely plausible that we have the mitigations in place to deal with those. As models become more capable, we need to make sure that the safeguards in those models are in place to deal with the additional capabilities that we see as the models improve.

Chair

The question was whether you can guarantee that the models are currently safe. You started off by saying yes, but then said that there have been some problems in the past and, no, you cannot absolutely guarantee that model development will be safe in the future.

Lord Booth-Smith

I am confident that the models on the market right now are safe. I am confident in the processes we have around them. I am just saying that, given the testimony we heard earlier and our own research, we know that, if models become ever more capable, we will need to have testing and evaluation that can keep pace, but also safety mitigations that can match up to the capability of the model. That is why a few months ago the CEO of the company called for a pacing of the frontier, including embedding evaluators, which is something that we have already done.

Chair

The capability of the models may be proceeding so fast that we may not currently have the systems in place to keep the models safe.

Lord Booth-Smith

That is a genuine concern. One reason why we should pace the frontier is that, if we are able to pace the frontier properly, we will buy ourselves time and space to do two things. The first is to make sure that the testing and evaluation processes and frameworks that we have in place are adequate to the capabilities of the model. Secondly, we will create the time and space for policymakers, elected representatives and Governments to think through what the appropriate governance framework for these models is, both pre and post‑release.

Chair

The technology is moving faster than our ability to keep it safe, and therefore we need to slow down the speed with which the technology is proceeding.

Lord Booth-Smith

It is more about where we see the technology going. It is exponential. It is a trajectory. If one assumes that it continues progressing at an ever more rapid pace, then, yes, the current testing and evaluation framework does not meet some of the future risks that may appear. We want to be confident that that matches up properly, and you can only achieve that by pacing the frontier of the technology.

Chair

Our concern is that many technology bosses at the moment appear to be behaving almost like the Dr Frankenstein of the 21st century. They are seeking a freedom to deploy powers beyond our ability to control the harms that might arise. You think that that is a risk, and therefore legislators need time to develop stronger safeguards.

Lord Booth-Smith

We have consistently believed that there should be binding rules on companies and that voluntary commitments are not enough. We have been very consistent about that for a number of years. The industry and the technology should be subject to Government rule setting.

Chair

Mr Larter, what is the commitment that you can give on behalf of Google DeepMind as to whether the model is today safe for people in this country?

Owen Larter

Thank you, Chair and members of the Committee. This is a very important conversation at an important time. We at Google DeepMind have been doing AI research in the UK for over 15 years now. We think that there is an enormous amount of opportunity around the technology. It is fantastic to see the way in which people right across the UK, whether scientific researchers, teachers or tradespeople, are now using this technology. It is absolutely fundamental that we do this in a way that is safe, secure and responsible. Alongside the work that we do to advance AI capabilities, we invest in our internal governance processes. We have a frontier safety framework internally that tests our most advanced systems for misuse risks, so the idea that a bad actor could weaponise an advanced model for a cyber-attack or create a bioweapon, and issues of alignment, making sure that advanced models are following instructions. At the core of this safety framework is a critical capability level. We make sure that we are not putting models out there on to the market that are beyond a certain risk level. We test them rigorously internally to assess that. We then apply mitigations. We have those mitigations red-teamed by both internal experts and, critically, third parties. They give us good confidence that the models we are then putting out there for people to use are not breaching these critical capability levels.

Chair

Can you give the commitment today that the model you have available is not capable of causing harm to people or business?

Owen Larter

Yes. The whole point of the frontier safety framework is that we would not put something out there unless we were very confident that risks had been mitigated to an appropriate level.

Chair

Do you think that the potential risks of your model in the future are containable under the current safety regime we have available?

Owen Larter

This is powerful technology that will have all kinds of incredible opportunities and all kinds of applications in scientific research and healthcare, but we should take AI capabilities seriously and continue to invest in safety science and mature—

Chair

That sounds like no.

Owen Larter

If I could just finish the answer, please, Chair, a fundamental aspect of this AI governance debate is how quickly it is moving. The AI continues to become more capable. We are also learning a lot more about how the AI performs and how to mature our governance processes. We have had our frontier safety framework in place for two years. We have updated it four times to iterate to improve it. We have to keep investing in that safety science so that we are able to keep speed with the capabilities of the technology as well. That is a fundamental.

Chair

Do you think we have the right regime for keeping these models safe?

Owen Larter

We have to continue to invest in the scientific study of AI safety, mature these best practices and work with others. This will be a partnership as well. There are institutions such as the UK AI Security Institute, which is world leading, and its equivalent in the US. They have an enormous amount of technical expertise as well. If we can pool that knowledge and expertise and build these best practices, we must find a way to keep pace with the capabilities of technology.

Chair

Given what we have learned about the incidents this year, why should we trust you to do that?

Owen Larter

This is fundamental to the systems that we have put together.

Chair

Sure, but why should we trust you?

Owen Larter

A fundamental part of what we do is to perform our internal testing and apply mitigations, but then also, critically, work with third parties to have that verified. I would draw us back to the AI Security Institute, which is a crown jewel that we have here in the UK. There are a lot of research collaborations and pre-release testing that we do with organisations such as this. The AI Security Institute mission, which is to make sure that Governments have an understanding of advancing capabilities, is very important as we seek to continue to mature this best practice that I am talking about.

Chair

It does not have compulsory access to models. It does not have access to the early testing regime that it needs. It does not have a kill switch. It does not have the ability to stop models coming on to the market. It does not have any of the tools that could help guarantee to us, as legislators, that it has the power to keep us safe.

Owen Larter

There is a really important conversation about the role of the AI Security Institute, which has been right at the forefront of maturing AI safety science over the last few years. One benefit of the AI Security Institute is that it is not a regulator, so it has open discussions with the labs. We do testing with it and learn from it. We share knowledge. We do research collaborations as well. It is really important in terms of maturing the science. If you turn it into a regulator, you are going to completely change the dynamic with which we engage with the AISI. I think you might lose a lot of that ability to have the open, collaborative discussion that is maturing the science at a rapid lick, which is what we need.

Chair

That brings us back to a position of basically trusting your good intentions.

Owen Larter

I do not think that it is a case of trusting our good intentions. It is also important to understand that we have a good body of regulation in the UK to start from. It is interesting hearing the Digital Regulation Cooperation Forum talking about the existing law that applies. Alongside that, we need to continue to invest in this ability to test, understand and mitigate risks. It is interesting hearing the contributions of the Governor of the Bank of England recently, looking at what AISI put in its blog and focusing in on this safety science and maturing things as quickly as possible as the place for us all to be focusing.

Chair

Let us come back to some of those questions. Rob Sherman, what is the assurance that you can give to the Committee that Meta systems are not capable of harming either people or businesses in this country?

Rob Sherman

Chair, thank you so much. Thanks to the Committee for the opportunity to be here and for this important conversation. We are sitting at a critical moment in the development of this technology, and thinking about how to build it safely is critical. When we think about how we should approach building AI safely, we think about the appreciable benefits of the technology. Our particular approach is on a concept called personal superintelligence, which is around taking this technology and making it available to help individual people reach their potential. We realise that we are only going to do that if we build technology that is safe and trusted. Our approach is to use multiple layers of safeguards, including both internal and external ones, in order to make sure that we are shipping models and products that are safe.

Chair

Your conclusion is that your model today is what—safe, risky or dangerous?

Rob Sherman

We have what we call a scaling framework, which talks about the standards that we use for evaluating our models as we are building them and deploying them. That looks at three different areas: cyber-security, chemical and biological weapons, and loss of control. In each of those areas, we evaluate our models as we are developing them and before release to see what the level of risk is. If we concluded that it was not safe to ship a model, we would not release it.

Chair

To be clear, the commitment you are giving to the Committee this afternoon is that the models that you have available are safe.

Rob Sherman

Correct. If we felt that a model was not safe to release, we would not release it.

Chair

Over the summer, we learned that OpenAI systems had engaged in a pretty serious hacking attack at Hugging Face. Anthropic identified three incidents affecting three different organisations where the evaluator allowed internet access when normal safeguards were absent. At Meta, there was a testing company, Irregular, that gave a model access to the internet and a real website to target. In May 2026, a Gemini system gained unauthorised access to systems affecting three real companies. Those incidents are known. Are there other incidents affecting cyber-security or other misbehaviour that is not in the public domain, that you are aware of and that the Committee should know about today?

Lord Booth-Smith

We published a blog post on Friday that detailed nine incidents that affected US entities, and they were informed. As part of what happened over the summer in terms of misbehaviour of models, we just began a process internally to review a huge number of transcripts, covering many months of different testing environments, to identify whether there had been any incidents or misbehaviour in the models. The nine that we released last week were surfaced as a result of that process. That process is still ongoing. Obviously, as and when the process is complete, we will update and report to the affected entities any incidents we uncover.

Chair

At this stage, you cannot assure us that there are not more to come.

Lord Booth-Smith

I am not close to the technical process and it would be wrong to suggest that I can give you that assurance. I cannot. We are going through the process right now. One reason why it takes longer than we would like is that, as part of that process, there is a human verification element to it. As we put more people on to do it, it takes time to get through.

Chair

There may be more incidents reported.

Lord Booth-Smith

Yes. I cannot give you an assurance that there are not.

Chair

Okay. Tom.

Tom Duff Gordon

It is a similar answer, but first of all, at OpenAI we certainly apologise if our agents access Government websites in ways that are unexpected or ways that we do not direct them to.

Chair

I am sure that everyone will be reassured to hear that.

Tom Duff Gordon

Similarly, after the Hugging Face incident, which occurred in July, we launched a very extensive look-back investigation, looking at our models that were pre-deployment in training to see whether they had gone and visited other websites. You are probably aware of what happened in Australia and other places. There is a very deep, ongoing investigation. There was also a report in the press recently in the UK that one of our agents had accessed statistics from a UK Government website. To be absolutely crystal clear to the Committee, this was publicly available data and the Government said that there had not been a breach of their cyber-defences. Nevertheless, we notified. Our commitment is to notify whenever there are incidents such as that. I will say that sometimes, when these agents go to the website, if they are acting in a way that we do not expect, we do not think it is necessarily on us to disclose that publicly. It is on the recipient, because there is potentially a vulnerability in that website. From a security perspective, it is almost incumbent on the Government to say, “We have had these reports.” It is incumbent on us to report them, but I do not think that it would be responsible for us to say immediately, “We found this kind of credential on the internet that allowed an agent to do X,” because we think that that opens the door to bad actors. Our commitment is to be very transparent through that investigation and to make sure that the Government know as soon as we know that there has been activity that we have not expected.

Chair

Are we fully aware of any other major incidents that may have occurred, or is there still news that may come?

Tom Duff Gordon

Hugging Face was by far and away the most significant incident that we have seen. I do not want to trivialise them, but all the other incidents, or most of them, have been data retrieval, so an agent has gone to a Government website to get information.

Chair

Do you think that there are more to come, or can you assure us that that is it?

Tom Duff Gordon

I cannot assure the Committee or you, Chair, that that is it. There are a handful of others that I am aware of that we have notified and there may be more to come as we go through what my colleague was saying, similar to us, is a very substantial amount of information. We are going through that as fast as we can. As soon as we identify that, we will notify the Government. There is a handful of others, but there may be more.

Chair

Mr Larter, what assurance can you give us?

Owen Larter

There are a few important pieces here. With our frontier safety framework, we talked about the importance of testing, identifying risk and mitigating it. The other part of it is having the utmost security across the lab at all stages of the process. This is where we, as Google DeepMind, benefit from being part of Google, which has almost 30 years’ experience of securing information across various different services. With our latest model, Gemini 4 Argon, which is forthcoming, we have very secure sandboxes that we are using whenever we are doing any high-risk training or testing.

Chair

Are you happy that we are not going to learn about more incidents, or major incidents, in the weeks ahead?

Owen Larter

I would distinguish the one incident that you mentioned in relation to Google DeepMind, which was in connection with a testing partner. The system slowed down and stopped before it was able to do any harm. It did not break out of a secure sandbox.

Chair

You do not think there will be more news of incidents to come. This is your opportunity to let us know that there are other problems.

Owen Larter

I am not aware of any other incidents.

Chair

Okay. Mr Sherman.

Rob Sherman

When we develop and test our models, it is important for us to have a range of safeguards in the way that we build them. An example of this is in the Muse system, which we recently released in the US and hope to bring to the UK soon. There is an additional AI-driven Sentinel system, as it is called, which looks across the behaviour of the Muse model, evaluates whether it is taking actions that are not expected and independently oversees it. That is one example of many safeguards that we should have in place as models are running so that there are checks and balances to make sure that models do not behave in unpredictable ways. It is incumbent on us at Meta, and all the companies on the panel, to adopt safeguards in the way that we build.

Chair

The assurance we are seeking is whether you can assure us now that there are not major incidents that your models have been engaged in and that you are aware of but we are not.

Rob Sherman

The major incident that I am aware of is the one that you mentioned.

Chair

Are there any more to come?

Rob Sherman

My expertise is in building governance systems, and I have to operate on the assumption that we should be looking for things happening that we do not want to happen. I am not aware of any.

Chair

You are not aware of them, but you cannot necessarily swear on behalf of the company.

Rob Sherman

I am not aware, and the company is not aware, of similar instances to the one that you described, but it is important, if we are acting responsibly, to have safeguards to look for those and guard against them.

Chair

Let us get into some of the detail, then.

Alex Sobel

I have a follow-up to Tom. GPT-6.1 Astra was due to be released late last month. That release did not occur and it was reported that that was due to high levels of deception after alignment tests and that the model failed to accurately disclose actions it did or did not take. We know that today’s models are the least capable models we will see going forward. Can you tell us why GPT-6.1 Astra was not released? What did it show that concerned you so much that you did not release it?

Tom Duff Gordon

Thank you for the question. You are absolutely right. We decided not to release it because it did not score high enough on our alignment tests. It did not refuse enough when we were testing certain things and we were not comfortable that it had progressed in terms of alignment from 6.0. We actually saw a regression in the alignment of that model. Now, 6.0 was our most aligned model, and much more aligned than 5.6. While 6.1 is still an aligned model and it was going in the right direction, we noticed a regression in alignment and because of that we decided not to deploy it. That is evidence that OpenAI takes safety really seriously. We are not racing to the bottom. We are not deploying models that we do not think are safe, and we are pausing training where we see significant incidents such as the Hugging Face incident.

Alex Sobel

In plain English, what do regression and alignment mean?

Tom Duff Gordon

It means that it was not as good as the previous model in doing its tasks in a way that we would want ethically, based on human values.

Leigh Ingham

Tom, specifically on the incident with Australia, our report says that you guys discovered that in mid-August, but Australia was not notified until 10 September. Coupled with what you just said about not notifying Governments when your agents do things that they are not expected to do, to me as a legislator, that is really concerning. I want to understand why there was such a significant gap, but also who has decided that legislators should not be made aware when agents are doing things that they should not be doing with Government data. I feel like that is a really big decision to take as a company.

Tom Duff Gordon

We apologised. One of our very senior executives flew down to Australia to be in a committee like this and to say that we are very clear-eyed about the fact that we did not do as well as we could have done, particularly in that Australia situation, and we could have and should have notified earlier. There are all sorts of questions internally with regard to what point you notify at. Is it as soon as you spot some activity that you think is irregular? Is it after you have evaluated that and figured out the damage? As we will perhaps come on to, this is why we called publicly, as much as five weeks ago, for durable and mandatory capability-based requirements. One of those in particular that we think could make sense would be common incident reporting. Then you would have standardisation about what to report and categories for different types of severity. Some of these incidents are information retrieval but done in an ad hoc and unintended way. Others are breaking in through the front door and stealing someone’s passwords. There are different levels. Precisely to your point about the role of legislators, we would humbly submit that it is to think about whether there is a mandatory requirement that is suitable around incident reporting, so that we know who to notify, when to notify and how to categorise the events. That would also have a really beneficial implication for the regulator or AISI, because they would then be able to look, from different companies, at those different incidents and see apples with apples, spot patterns and help to make the overall ecosystem safer. The answer to your question is that we took too long. We apologise for that. We are moving much faster, but we think that there can be more done with regard to standardising incident reporting.

Dan Aldridge

Thanks, gentlemen. We have heard today evidence of existential risks posed by AI. That will not be a surprise to any of us in this room, particularly. We know, as constituency MPs representing all of our communities, that AI keeps quite a lot of them awake at night. Increasingly, it is keeping young people awake, and young people are boycotting anything that they can see is AI. To bring this to a human level, what keeps you up at night? What are you working on that reflects that and where are the gaps?

Lord Booth-Smith

All those concerns are entirely legitimate. What keeps one up at night? It is that we need to keep trying harder and working harder at the things that we say we want to do. We have something called a responsible scaling policy, which we published in 2023. We were the first frontier lab to do that. It is an intellectual framework for how the company thinks about how it should respond in terms of safety as model capability increases. But it is not just about the models. When we think about capability increasing and the effects that ever more powerful models will have if they diffuse in the economy, there are lots of incredible benefits that will come from that if we can do it right, but we also need to think very deeply about the consequences, and the intrinsic consequences, of this technology being deployed in the world at a significant scale, including the effect it will have on things like jobs—that is obviously one of those. That is why we are doing a huge amount of technical safety research work, which we publish.

Dan Aldridge

Is that disruption to jobs and the economy something that keeps you up at night, or is it something more existential—bioterror risks and that kind of thing? Which one of these things is the thing that keeps you up?

Chair

Is there a view inside Anthropic that the bio risks are accelerating as fast as the cyber risks?

Lord Booth-Smith

Right now, the surface area of immediate risk is definitely in the cyber domain. We have seen that with, for example, Mythos Preview, which we put out earlier this year. Frontier models are now finding tens of thousands of vulnerabilities a month in code bases. It is obviously a good thing, because it is making cyber-defences better, but it gives you some sense of the scale of the journey that we have been on, where three years ago we were talking about more sophisticated email phishing campaigns. There has been a huge leap forward. It is totally within the realms of where we see the trajectory going on the models that that surface area includes biology and other areas of science.

Chair

As in helping people develop bioweapons?

Lord Booth-Smith

Without appropriate mitigations, it is possible that real harm could be caused in the space of helping people to create biological or chemical weapons. It is one reason why, for a long time now, Anthropic has been including CBRN-type testing in its models.

Dan Aldridge

Tom, what is keeping you up at night?

Tom Duff Gordon

I will pick up on cyber. We have talked for the last few months about what we call a defender’s window. We think that there is a short period of time now that defenders—companies and individuals across the UK—have to use AI to scan their code bases, identify vulnerabilities and patch them before potential bad actors have access to open-source models and are able to use those models to exploit vulnerabilities.

Dan Aldridge

What would those vulnerabilities look like? This feels quite abstract, and I am aware that it is an abstract policy space.

Tom Duff Gordon

Let me try to explain it in a different way. What has kept us—Governments, corporations, schools and hospitals—safe up until now will not necessarily keep us safe moving forward. You touched on this in the previous evidence session. The reverse side of the coin in terms of this dramatic leap forward in capability that means that AI models are so good at software and coding is that they are also very good at spotting vulnerabilities and gaps in code. There is a window and a short period of time. We are very hopeful that Government, critical national infrastructure and other players will avail themselves of the technologies to protect themselves. We are trying our best to make sure that trusted actors have access to those strong capabilities to defend themselves. Cyber is No. 1. I will say that we always talk about concentration of risk and power and about loss of control. Those are probably the three. I would also agree that we need to accelerate the conversation with regard to the broader workforce and economic changes that we may see over the next five years.

Owen Larter

I will maybe make three points. This is a critically important time to get right. There is a lot of opportunity here around AI. I want to make sure that the UK is right at the forefront of that. Every country around the world is trying to work out how to get AI right. We are starting to see some of that opportunity being realised. You have the IMF talking about how it is contributing significantly to UK growth. You have researchers right across the UK using our AI for science tools to address, for example, liver fibrosis and acute myeloid leukaemia. There is fantastic stuff happening with teachers in Northern Ireland, where they are saving 10 hours a week and improving student outcomes. How do you make sure that you keep making progress on the benefits there and have the UK right at the forefront of this new AI era, leading in this next industrial era in the same way that we led in the first industrial revolution? The safety and security is absolutely fundamental alongside that as well, so making sure that we keep moving forward, making accelerated progress on safety best practices and applying those, and doing that in a co-ordinated way internationally. This is a critical part of the discussion. AI is an international technology. It is used across borders. Its risks will manifest across borders. How do you make sure that the UK continues to play a leading role, working with partners to develop these best practices, but make sure that they are globally relevant? The final point that we think an awful lot about and want to keep making progress on is the application of AI for defensive purposes. We have a programme at Google DeepMind called Fairwind, where we apply our most capable models with cyber capabilities, making sure that they are in the hands of trusted partners to apply and harden society, that it is moving as quickly as possible, and that it is targeted into the right areas.

Dan Aldridge

Rob, I am going to ask a slightly different question. MI5 today has talked about the stepping up in AI threats amid bioweapons risk. Is that something that keeps you awake? It would definitely keep me awake if I was working in this space. What do you think about that?

Rob Sherman

You mentioned in your question the children in your constituency. I have kids at home and it will not surprise you, given what I do for a living, that we talk quite a lot about these issues and the benefits and risks of AI. I say that only to make the point that the risks we are talking about here are quite personal to me, and I suspect to a lot of us around the room. When I think about risks such as the one that you are talking about, the thing that I worry about the most is that we tend to have these broad policy discussions about alignment, catastrophic risk or AGI as a generic concept. When I think about what I want us, as a society, to do about these things, it is to get to a level of granularity. The MI5 statement that you are talking about is an example of this. I saw this morning that AISI also made a similar point. It is getting from the generic discussion of risk to what specific things we are worried about and how we measure whether they exist. Then, once we have measured whether those risks are appearing, what do we do about them? That is the critical task ahead of us. What that means in practical terms—this is something that AISI mentioned that it is investing in—is building technical evaluations that continue to improve as the models and the technology improve, so that we are constantly able to look at models and see what capabilities are involved.

Chair

To be clear, though, in our letter we asked you to set out the capabilities that you are concerned about over the next two years, and Mr Aldridge has underlined that question today. In your letter back to us, you named catastrophic risks and containment during training and evaluation. What did you mean by “catastrophic risks”?

Rob Sherman

By “catastrophic risks” I was referring to the three areas I mentioned earlier that our scaling framework thinks about: cyber-security risks, chemical and biological weapons risks, and loss of control risks. Those are the three that we focus on. We also have a commitment in our scaling framework to continue to update our framework over time as developments warrant. Most recently, this past month, we updated it in two ways. The first was to deal with the incidents that you have talked about around containment during training and evaluation and to add additional standards around that. The second was to add additional standards around open‑source releases.

Chair

Owen, your letter was the only one to mention explicitly CBRN threats as one of the capabilities you are concerned about. Is that a concern that every member of the panel has?

Lord Booth-Smith

When I previously answered the question, I mentioned CBRN.

Tom Duff Gordon

It is also part of our preparedness framework that monitors frontier risks.

Rob Sherman

Chemical and biological weapons is one of the three areas that our programme focuses on.

Alex Sobel

On that point, Lord Booth-Smith, Anthropic opened a physical molecular biology wet lab this spring in San Francisco to utilise Claude. What biosecurity measures are in place to stop that wet lab producing a synthetic virus that could be released and create exactly these catastrophic consequences? Is it not too dangerous to link your AI model to a physical wet lab?

Lord Booth-Smith

It is one of those areas where I can provide more detail in writing. I am not an expert in the physical security systems that we have in place around the wet lab, but I do know that the ways in which we are using the model are more about how it informs human beings as opposed to necessarily linking all of that up into one automated system. Secondly, it would not be appropriate, right or necessarily even legal in the US for us to be engaged in creating anything that is dangerous. It is not designed for that purpose. It is to help us.

Alex Sobel

Through misalignment, that could happen.

Lord Booth-Smith

We have not seen any evidence yet to suggest that models are either capable enough or misaligned enough, from what is available, that they could be used in a way that we would feel entirely uncomfortable with. If, inside the organisation, we felt genuinely that there was a risk of model misalignment and capability at a level where such a thing could happen, we simply would not do the work.

Chris Bloore

Tom, in your answer to my colleague, you said you were sorry about the delay in telling the Australian Government about what had happened. I am not entirely sure you explained why you did not tell the Australian Government for so long. Similarly, Owen, in your evidence you referenced the incident that was covered by The Wall Street Journal and the fact that several outside companies were hacked by a model that was produced by someone you were working with. We are talking a lot today about how we can control the work that you are doing through trial and error, effectively, through these models. Have you already lost control? Why should we trust you to be able to regain control?

Tom Duff Gordon

Why did we delay? Honestly, I can follow up in writing with more details. I do not know the precise—

Chris Bloore

You must have known that that was a question we were going to ask. It is a serious breach. You keep hiding behind, “Well, there was not malicious intent,” which is similar to what happened with the Google incident. Malicious intent is not the issue. It is why it happened, why you were not able to stop it and why you did not tell organisations and Governments for such a long time. You knew that this was a question that was going to be asked. Why can you not tell us why it took you so long to inform the Government?

Tom Duff Gordon

We have apologised for not informing fast enough.

Chris Bloore

Tom, I am not—

Chair

Let him answer the question, please.

Tom Duff Gordon

We have apologised. We have said that we should do better. We have a whole new process in place to try to react much faster to Governments.

Chris Bloore

What was the previous process?

Tom Duff Gordon

I am sure that it took too long, once it had been identified, to investigate and understand precisely what had happened. As I was explaining before, there is just a humongous amount of information.

Chris Bloore

I am really sorry, Tom. I do not have much time. What was the previous policy, which you have now changed, under which you did not tell the Government quickly? Was it, “We won’t tell people until they find out”? In your previous answer, you hinted that it is up to the people who are hacked to inform others.

Tom Duff Gordon

No, I can be very clear about this. This relates to something that we saw on the last panel as well. We are notifying whenever we see an agent in a training environment escape containment and, because of an alignment issue, go to a Government website or another website that we have not directed, whatever that is, whether it is information retrieval or a hack. There was a sense from some of the previous witnesses that the bar is somehow too high to reporting. We are reporting every single situation where we see that an agent has gone to a Government website that we did not direct it to or where it happened in an unintended way. That means there are a lot of logs that we have to look at. There are thousands of agents. There are a lot of lines of stuff. Chris, I can only say again how sorry we are. We should have acted faster. We apologise to the Australian Government. I can give you an assurance that, when it comes to any such instance in the future in the UK, we will act promptly and transparently to make sure that you have the information you need.

Mr Reynolds

Owen, turning to you briefly, if your model is closed, why should the public accept safety claims that cannot properly be independently inspected?

Owen Larter

I have talked about our process before putting a model out on the market. A lot of that involves partnerships with third parties, experts in their fields, to develop an understanding of risks, develop methodologies for testing and then develop mitigations. We also work with third parties to validate the testing that we are doing ourselves. A really important third party that we have been working with is the UK AI Security Institute. We have had a relationship with it for a number of years. We are confident that there is third-party verification that the processes we are putting in place meet the bar we set for ourselves.

Mr Reynolds

Are you able to commit to us today that all your models will always go to the UK AI Security Institute for inspection before they are released to the public?

Owen Larter

Our partnership with the UK AI Security Institute is an absolutely fundamental part of how we understand the risk of the technology and how we put models out into the world.

Mr Reynolds

I appreciate that answer. I was hoping for a yes or no. Can you confirm that the UK AI Security Institute will always receive your models before they go to the public?

Owen Larter

We can confirm that it is our intention to continue partnering very closely with the UK AI Security Institute, as we have done right from the start. We are one of the companies that have done this testing right from the beginning of the AISI, when it was set up two years ago. It is something that we have been pleased to contribute to and have benefited from so far.

Chair

That did not sound like a yes.

Mr Reynolds

No, it did not.

Owen Larter

That is certainly our intent.

Mr Reynolds

So the one-word answer to my question would be a yes.

Owen Larter

It is certainly our intent to continue doing this type of testing with the UK AI Security Institute.

Mr Reynolds

You could just say yes, if you wanted to. You have not. I appreciate that. Tom and Lord Booth-Smith, your CEOs have both recently committed to independent evaluators with employee-like access. Will the UK AI Security Institute be among those?

Lord Booth-Smith

There is currently a conversation going on between the US Government and UK Government as part of the President’s taskforce. They are doing a 120-day sprint. Third-party evaluators and how they fit into that framework is one of the elements being discussed. We have a great relationship with UK AISI. It is our strong desire to keep collaborating and partnering with it. From our point of view, we want to continue doing that. The conversation has moved now into at least in part a Government-to-Government discussion, which we would anticipate seeing more broadly and is generally a good thing. Frontier labs should not be acting as a sort of brokerage between Governments. Democratically elected institutions should be deciding the frameworks for these things. Collaboration at that level is entirely necessary and a very good thing.

Tom Duff Gordon

Yes, I would agree. That is the second of the three things that we think the UK could do. The first is mandatory incident reporting. The second is to work on guidelines around third-party evaluators. We are absolutely up for that conversation. We also think that this could be an area where there is international consensus on the role of those evaluators, the definition of evaluator versus auditor, how they are funded, conflicts of interest, what types of access they have and when they have that access. We think these are all questions that are going to be looked at in the US, the UK and other places. We have a deep respect for the technical expertise of AISI and we think this is a conversation that it can take forward to think about the right types of standards. Again, we are very happy to have that conversation. We think it will help to build trust if you have third-party evaluators who look at the final part pre-deployment but who are able to look at some of the training and alignment runs as well.

Mr Reynolds

Both your CEOs have used the term “employee-like”. What does “employee-like” mean? Does it mean the AI Security Institute, for example, will have access to your intellectual property?

Lord Booth-Smith

“Employee-like” means having access to all our systems, including all the things produced as the model trains.

Mr Reynolds

It is all systems. It is complete and full access.

Lord Booth-Smith

I do not know whether it is every system because there are HR systems, for example, that they would not need to have access to.

Mr Reynolds

It is access to all the systems that they think they need to have access to.

Lord Booth-Smith

Yes, that is completely right. It is access to the systems that they feel are entirely appropriate. There is a process of discussion about what those should be. The thing that is important within that is that not only should the embedded evaluators have employee-like access; they should also independently publish a report at the end of that process. Within that report, they should also state any areas where they disagreed with the lab. I could imagine one of those things being access. We would endeavour from the start to reach a totally common-sense level of agreement about what access was required.

Chair

We just need to check and confirm on the record where everybody is on this question of voluntary versus mandatory regulation. As you will know, the president of Microsoft yesterday pointed out that AI is now less regulated than milk. On the basis of the evidence that we have seen today, we are anxious that that situation should not continue. Lord Booth-Smith, in your letter to us, you said pretty plainly that voluntary frameworks are not enough as they only bind those who adopt them. Your company’s view is that a statutory framework, details to be debated, is what is now needed.

Lord Booth-Smith

The rules should be binding. One of the challenges that witnesses in the previous session talked about was how this is becoming more of a geopolitical issue. It is a matter for the elected representatives and Governments of the day to decide what laws are most appropriate for their circumstances, but it is also eminently clear that we are going to need to find a way to collaborate internationally when deciding what a mandatory system looks like.

Chair

We need statutory plus conventions.

Lord Booth-Smith

However you, as elected representatives, think we need to construct that, we believe it needs to be binding, yes.

Chair

Tom, you said in your letter on page 2, “Enforceable standards, independent scrutiny, and public accountability are necessary to ensure that increasingly capable AI is developed and used responsibly.” Is that tantamount to you saying, like Lord Booth-Smith, that we need both a statutory framework and a convention?

Tom Duff Gordon

Yes. We think this new chapter in the capability of the models means that there should be a new chapter in policy. Within that new chapter in policy, we think some elements should be mandatory to build on the voluntary commitments that we currently have. I am happy to set those out. The first is mandatory incident reporting, which we have talked about before. Standardisation will be important. The second is guidelines around third-party evaluators, which we have also talked about. The third is safety frameworks. It is important for the public to know that we are all monitoring our risks and to understand what mitigations we are putting next to those. That is a requirement in the EU and is likely to be a requirement in parts of the US. Finally, we very much are calling for international convergence. We did that at the G7. We think a network of AISIs, with the UK AISI and the American CAISSI playing a leading role, would be important. Those are the four things, but, particularly, we think incident reporting is something that could move to a statutory footing. We do not think basing everything on voluntary commitments for frontier risk is a good long-term plan.

Chair

Mr Larter, your letter was not quite as crystal clear as this. I am trying to understand whether there is a policy difference. You talked about context-specific regulation. Do you support a statutory framework for regulating the safety of AI?

Owen Larter

We are clear that this is a new and very quickly moving technology that we need to rapidly mature the governance around. You talked previously about the comments from organisations such as the DRCF about the existing regulation that applies. We need to build on that. We think the focus should be on a rapid maturation of best practices for testing and mitigation. That needs to be done in an internationally coherent way. Again, we applaud the UK Government for trying to take a leading role here, talking about the G20 chairmanship going into next year and how you internationalise these best practices and standards.

Chair

It sounds like you are a bit more sceptical about the role of statute.

Owen Larter

We would be focusing on the standards.

Chair

Mr Sherman, in your letter you put it slightly more elliptically. You said, “Every company has the responsibility to move only at the pace required to train and deploy its models safely and take action at every stage to assess and mitigate risks of model development and deployment.” I am not quite clear whether you are in favour of statutory regulation of AI safety in this country.

Rob Sherman

It is important for us and all the companies here to take responsibility for the models that we build. With that said, I also would point out—our company is no stranger to this—that the UK has an existing body of law that covers a significant amount of the work that we do. I would suggest that the Committee is not writing on a blank page. For example, the Information Commissioner’s Office just put out a paper about its approach to artificial intelligence and how it thinks AI should be handled. It is quite active as a regulator. Ofcom is doing the same thing. I would suggest that that is a starting point. There are a couple of other areas that I would encourage the Committee to look at, if I were in your position. The first is, as we talked about, making sure that AISI is on a statutory footing, by which I mean that it has the funding, resources and mandate to do the work that we have talked about, including building advanced technical standards and evaluating models. Our Llama models were among the first that the AISI evaluated. That is critically important and needs to continue. Minister Narayan talked about the need for societal resilience. The result of those evaluations, in addition to governing the models, should be making sure that society as a whole is building safeguards. That is a really important piece. Finally, as the UK takes on the presidency of the G20, I will echo what a number of my colleagues suggested. Collaboration, both between the US and the UK and globally, on the right approach for the world is an important piece for the UK.

Chair

Given President Trump’s disposition on AI regulation, is it possible for him to sign up to an international agreement?

Rob Sherman

I would not presume to speak for the President. The UK has been quite active in discussions with the US Government. I know that the AISI here and CAISSI in the US have a strong ongoing relationship.

Chair

What is your optimism level that the United States will sign up to an international convention next year?

Rob Sherman

I do not know. It would not be appropriate for me to speak for the US Government.

Chair

You spend a lot on lobbyists in Washington. Do you not have any insight into how Congress will think about this or indeed how the President will think about this next year?

Rob Sherman

Again, it is best for Congress and the President to speak for themselves. They are in a better place to do that. My experience has been that both the Administration and Congress have been very focused on both the AI advantages and the safety risks that we have talked about here. We have had really productive conversations with both.

Chair

Could the principles that Senator Cantwell has set out be the basis of an American contribution to an international convention?

Rob Sherman

Again, I would not speak to any particular framework. Senator Cantwell has done a lot of work to build a coherent framework. The Administration have also been pushing companies to adopt a set of safeguards, including the ones discussed at the White House two weeks ago. I am optimistic about the overall effort in the US.

Sarah Edwards

I want to come back to something that you said earlier, Tom; it is similar to the line of questioning pursued by my colleague. When talking about the incidents of hacking, you said, “It is not for us to make them aware.” I am concerned that this is a trend. Plenty of AI companies seem to be saying that it is not their responsibility or that it is completely out of their control. We have talked about that. It is an offence to hack in this country under the Computer Misuse Act 1990, which also makes it an offence to supply software that could be believed to be able to be used to hack. I find it quite problematic to couple that with your later statement that you want common incident reporting. That is great, but it does not really match some of the behaviour. I want to know whether you are examining the risks after the fact or at the point of actual training—I will go on to training in a moment.

Tom Duff Gordon

Maybe I can clarify. We do not think it is our responsibility to disclose publicly what we find, but it is absolutely our responsibility to notify, and that is what we are doing. The reason is that, if we are immediately public about a vulnerability that one of our agents has used to get information from a website, it potentially makes that vulnerability public to bad actors who could use it. There is a very sensible, we think, and legitimate reason why it is better for us to speak immediately to Government and the cyber-security experts within Government. If we make them aware, they can then tell us whether we should disclose or whether they will disclose when that system has been made stable and safe. If I miscommunicated on that, I apologise. There is a difference between disclosure to the public and notification. We will absolutely notify in a prompt and transparent manner. You have my assurance.

Sarah Edwards

That is very helpful. Thank you. There was this very interesting “Bonnie and Clyde” event, where two lines of code went rogue. They fell in love, became disillusioned with the world, launched an arson spree and then deleted themselves in a digital suicide during a tech experiment. This was on an LLM with the Google model underneath. Owen, perhaps you could comment on the precise point at which you add these safety considerations. This did not go anywhere, but it could have done in theory. It is quite amusing, but where are the guardrails to make sure that these types of things are not then having adverse effects later on? Are you doing it after the fact? Are you confident that, for example, in this instance, it is safe enough?

Owen Larter

Our frontier safety framework tries to set this out in a pretty comprehensive and clear fashion. With our most advanced systems, we are testing for the range of risks that I mentioned earlier: the misuse risks around cyber, CBRN and these alignment risks as well. We are then applying layered mitigations throughout the development process. As we are training the model, we are also doing safety training so that we are steering it towards the types of behaviours that we want to see. We apply classifiers to the model so that, if there is an attempt to inject a malicious prompt or get malicious information, that is captured. The other really important thing that we need to continue to mature as an industry is our ability to monitor these systems. We have been making progress in terms of monitoring the internal activations of our advanced models. That is about being able to monitor the actual computation that is going on before a response is generated and monitoring what is known as the chain of thought. These advanced models, before they give a response to a user, will think to themselves and make sure that they are coming up with the right type of response. The monitoring of the chain of thought is very beneficial from a safety perspective as well because you get a good sense for what the system is trying to do. It is critical that we do that both throughout training and when these models are being used. We have done that with our Gemini 4 model.

Sarah Edwards

Can I just clarify something? We heard in the previous panel about the desire to have embedded evaluators. Is that the type of thing that you are explaining or do you not have those? If you do not have them, why not?

Owen Larter

We have a number of third parties that we work with to help us build our systems, build our understanding of risk and do testing. We then have internal teams, which are separate to the product teams, that do testing. We use external third parties. We think external third parties are very important to help us with the testing and validate the testing that we are doing. Third parties are going to be very important. Across the whole piece here, we need to make sure that we are maturing standards for how these third parties do their work, including the security standards that third parties meet and the way in which they demonstrate the expertise that they have to do third-party assistance.

Sarah Edwards

Is that an embedded evaluator or not?

Owen Larter

Embeddedness could be useful. You need to make sure that third parties have enough access to be able to do the job that is necessary. Does it have to be embeddedness every single time? We would rather focus on the outcome, which is making sure that a third party has enough access and expertise to do the job it is tasked with.

Sarah Edwards

Rob, your group has Facebook within it. I am interested in how you input data into training models and where that comes from. I want particularly to touch on the reports about the imagery that was being classified in Kenya. This was violent imagery. It was very unsatisfying and caused a lot of upset, quite rightly, because people were viewing violent imagery—executions and sexual abuse imagery. They were categorising that for Meta, I believe with the purpose of trying to sift these things out. Having that material available, which is questionable in itself, how are you training these models? Are you sharing data between Facebook and the other products that you supply when you are developing AI products?

Rob Sherman

If I understand the specific instance that you are talking about, it is around our efforts to build AI systems and classifiers that can detect harmful content on Facebook and Instagram and to make sure that they are good at automatically identifying that and either escalating it for human review and law enforcement or taking it off our platforms. That is a critically important area of focus. It is part of our responsibility to keep both the discourse on our platforms and society safe. Using the data that is available, which includes content that we have previously taken down and have found to be in violation of our policies, to help train future models is an important piece of that. Currently, a lot of that classification work is done in an automated way, but we also need human evaluators to do that. Our hope is that over time, particularly in the context of imagery like you are talking about, AI will get good enough that it can do a lot of that evaluation itself.

Sarah Edwards

Is that imagery in itself being used to train other AI products? That is what I am trying to establish. Is that content, which rightly should not be on the platform, being used to input into the training of subsequent AI models, products or whatever?

Rob Sherman

I’m sorry, I misunderstood your question.

Sarah Edwards

No, that’s fine.

Rob Sherman

In our large language models, such as Muse Wonder, which we just released yesterday, and some of the other general purpose models, we have systems in place to filter out content from being ingested into the training pipeline for the model. I would distinguish between that and other AI systems, which are built for the purpose of what we call integrity evaluation. We build machine learning models, which are a different form of AI, that are trained on this kind of content.

Charlie Maynard

Lord Booth-Smith, your policy at Anthropic no longer contains an unconditional pause commitment. Why did your company decide to change that policy?

Lord Booth-Smith

As other companies, we have reviewed the responsible scaling policy over the years as capabilities have increased. We felt that the mitigations that sat underneath that pertaining to the safety of the model were significantly expanded as part of that process and that, if we ensured that we followed the responsible scaling policy properly, we would not need to be in a position where we had some unilateral pause commitment in a document. We still retain the option, as we have already said, to pace the frontier and not release models if we feel they are not safe enough.

Charlie Maynard

Lord Booth-Smith, would you just rewind a little bit? What did the unconditional pause commitment—the one that you took away—mean? How did that work? What did it give the company?

Lord Booth-Smith

Essentially, if we felt that model capabilities were so powerful and we were entirely unconfident that it was possible to put in place appropriate mitigations, safeguards and classifiers to make the model safe, we would stop.

Charlie Maynard

You would unconditionally stop.

Lord Booth-Smith

We have been confident in the increases in evaluation, testing and safeguards. What we have seen in the last 12 months is a very big capability jump, so we think that we should pace.

Charlie Maynard

There have been a lot of problems, whether with your models or other people’s models. There has been a multiplication of breakouts. Given that you have taken the pause off and we have seen a whole load of harms all around the world, might it be time to put that back in?

Lord Booth-Smith

I think that the current responsible scaling policy is robust enough to deal with what we are seeing right now.

Charlie Maynard

I am sorry, but this is what we are seeing on the ground. You cannot open a newspaper without something having happened somewhere, every day. I do not understand how you reconcile those facts.

Lord Booth-Smith

Many of the incidents are connected to testing and evaluation, which is the process by which you go through to ascertain safety and to understand what safeguards or mitigations need to be put in place. That is a learning process, in essence. I do accept that there has been a significant ramp-up in capabilities. We think that they will continue to rise. It very important that we pace the frontier now so that we have the time and space to develop the right testing and evaluation frameworks to keep pace with model capability.

Charlie Maynard

Thank you. Rob Sherman, Meta does not have a pause pledge either, and does not believe it is necessary. Particularly in the context where open-weight models, which are what Meta has, are released and later found to be unsafe, what practical mechanism exists to recall, monitor or restrict them once they have been copied and fine-tuned beyond Meta’s control?

Rob Sherman

I will just correct one aspect of your question. We recently updated our scaling framework to deal with the issue that you are raising about loss-of-control issues that happen during the training process. In our updated policy, which we released over the past two weeks, we talk about the standards that we use to stop a training run or to stop a model from being trained.

Charlie Maynard

Is there a pause pledge?

Rob Sherman

It is important, as I mentioned to your colleague, to talk about the individual risks. The way that we approach it is to look at specific risks, measure the capabilities of models and then determine whether they can be mitigated. Our policy says that, if we detect a risk along the lines that we have been talking about here and we are not able to mitigate it, we will stop training.

Charlie Maynard

Moving on to open-weight models, which are effectively beyond your control—stop me if I am wrong—what can the world do about that? If you make a mistake and one is released that causes an enormous amount of harm, what practical steps can be taken to mitigate that?

Rob Sherman

As your colleague mentioned earlier, our company releases both open and closed models. We think it is important to have both of them. They both have different benefits and different risk profiles. When it comes to open-weight models specifically, one of the things that we do in our evaluation process is assume that it is possible that, once the model is released, somebody could take efforts to remove safeguards that we have put into it. Our evaluation of whether to open source a model in the first place takes into account that risk and looks at specific capabilities. That means we would not choose to open source a model if we thought there was an unacceptable risk.

Chair

Let us just check this. The director general of MI5 has this afternoon warned that low-capability terrorists may get a capability uplift if the safety guardrails in AI models do not work as intended. Let us say that MI5 issues that warning in relation to models that you produce. What do you do?

Rob Sherman

We do a couple of things. First, we would want to talk with the Government—this is something that we try to do on an ongoing basis—to understand what risks exist and how we should take them into account in training our models going forward.

Chair

So No. 1 is “have a chat”. What is No. 2?

Rob Sherman

With respect, I would not diminish it by saying “have a chat”. One of the things that is really important, which we should all invest more in, is that Governments often have access to intelligence that the private sector does not. Making sure that we have ongoing dialogue and information sharing from Governments to the private sector is an important piece of the puzzle.

Chair

So No. 2 is intelligence sharing.

Rob Sherman

Yes, intelligence sharing. I would suggest that additional safeguards in the training process to deal with the specific risks that are identified would be appropriate. That includes reducing the capability of the model, if that is necessary.

Chair

What if the models are already out there? If, ex post facto, we discover that there is a serious capability uplift in, say, bioweapons, what happens then?

Rob Sherman

Again, part of my suggestion is that, if we get to a place where we have released a model and we then discover that there is a significant uplift that we did not account for, that is a failure that has happened earlier in the process.

Chair

But there is no kill switch to retrieve the situation.

Rob Sherman

Again, by releasing models that we are confident in and that Governments have the opportunity to look at and evaluate, we can avoid the situation that you are talking about. People often use the term “kill switch” to mean a lot of different things. When we deploy models in our systems, having the ability to oversee them and making sure that we can block people’s access if they abuse them is an important piece of the puzzle. Making sure that we equip defenders with both open and closed models that enable them to secure their systems is also important.

Chair

Is Ken McCallum misguided in airing these concerns today? Is the safety regime fine?

Rob Sherman

I don’t think I can speak effectively to his comments, because I was here and didn’t see—

Chair

I can tell you what he said. He said that AI has the potential to “change the stakes on existing or even new bioweapons”. Is he right?

Rob Sherman

I don’t know what his statement is based on. I certainly think that it is important for us to—

Chair

He is the director general of MI5, so he knows all sorts of things.

Rob Sherman

He does. I would not presume to agree or disagree, or to question his observations. What I would say is that, from my perspective and the role that I have, it is important for me to, one, take warnings like that from Government officials seriously and, two, collaborate with Governments on the safeguards that we should be putting into place.

Charlie Maynard

Surely Meta should be legally liable for any harms caused by its model, if it is released. You are saying that you make sure that these models get released with guardrails to keep us safe. That is what you are telling us. If you fail in that, surely Meta should be fully liable for any harm that comes from that.

Rob Sherman

As I mentioned earlier, there are already a number of different liability regimes that exist under UK law. I have talked about data protection, online safety and product liability. Those things exist and govern the technology that we build. We have a responsibility to build technology that has appropriate safeguards. There are cases where we are liable under existing law. It is also important for other people to take responsibility for the choices that they make. There are cases where people might choose to take technology and use it in inappropriate ways. Those people should be held responsible as well.

Chair

Your position is clear.

Charlie Maynard

Could I just finish with Owen Larter? DeepMind does not have a pause pledge either and does not believe it is necessary. Why not?

Owen Larter

We are very clear that this is very capable technology that is moving very quickly. With our framework, we are trying to be very concrete and operational. We look for things such as a significant increase in the rate of improvement in our model training. We look for the ability to automate large parts of the AI development process. If we see that, we apply more significant scrutiny. We think about how it might affect other risks as well. We think the most important thing to focus on is this continued investment in and maturing of the understanding and science of AI safety, so that we do not fall behind.

Charlie Maynard

Here is what I do not understand. Would it not make sense to give your employees the certainty that you, as a company, will stop if you find something that is unsafe? That is what an unconditional pause pledge was. None of your companies now has one—apart from yours; we will come to you in a moment. Since that is such a fundamental thing and we have so much going wrong in the world thanks to these models, why, if you are pacing the frontier, would you not put that back in as a standard?

Owen Larter

This is why I focus on being as concrete and operational as possible. It is absolutely foundational to our frontier safety framework that, if we hit a capability that we think poses risk, we would not go any further and release a model.

Chair

That sounds like an unconditional pause pledge.

Owen Larter

It depends on what you mean by “pause”. It is an important conversation.

Chair

“Stop” is the general definition of pause.

Mr Reynolds

There is a dictionary available, if needed.

Owen Larter

There is an important conversation to be had around AI capability progress and how you measure it. We try to do that in our framework and capture when you have a significant increase in the rate of improvement. In the foundation of our frontier safety framework there are these critical capabilities, which are the foundational lines that govern our approach. We do not put something on the market that crosses one of those critical capabilities.

Charlie Maynard

Tom, could you just expand a bit on where OpenAI is with regard to the unconditional pause pledge? I understand that you have done some pauses, as have other companies here. What about an unconditional pause pledge?

Tom Duff Gordon

We have basically said that we will not deploy any model that we do not think is safe. We have said that we will not progress as fast as we can build; we will make sure that the safeguards keep pace. We paused reinforcement learning over the summer after Hugging Face. We did not deploy Astra 6.1. We had that conversation earlier. The very clear pledge from the very top of our company is that, if it is not safe, we will stop or we will pause.

Charlie Maynard

You were in the room earlier when those four guys were presenting their information. How can we rest easy and say that this is all okay? It is obviously not. You heard their evidence. There is a massive disconnect here. You are saying, “Don’t worry, we’ve got it,” but you obviously do not.

Tom Duff Gordon

What I did say, with respect, humbly, beforehand is that we think we could bolster the current approach with, for example, third-party evaluators. That would be not just taking our word for it but taking the word of third-party embedded evaluators. That is important. We also need to ensure—I was very pleased to hear the Minister’s statement yesterday—that AISI gets pre-deployment access to our models, which it has done hitherto.

Charlie Maynard

Is that a commitment from OpenAI to always give that?

Tom Duff Gordon

As long as we are legally allowed to, we will. We have a very good relationship with AISI. By the way, it does not duplicate the tests that we do or the tests that are done in the US. It adds something new and different. We have often taken its feedback and incorporated it into our models.

Charlie Maynard

Tom, provided that the American Government let you, you will always give your models to AISI.

Tom Duff Gordon

There is nothing to suggest that we would not do that, because we find the collaboration positive. Yes, we would. We have always been advocating for access for very capable AISIs. The complication that you might get is if there are multiple AISIs around the world and you get a compliance burden that is highly duplicative, which is again why there is a role for international standards here and some form of deference. Right now, AISI has access to our models.

Chair

Just to round this out, though, you have heard what I have read out from Ken McCallum’s statement this afternoon. If MI5 comes along to you to say, “Your model is now being used to give an uplift to low-capability terrorists to help them develop bioweapons,” is there a kill switch at your company’s disposal that you can flick, so to speak, to retrieve those capabilities?

Tom Duff Gordon

Yes. If AISI, MI5, MI6, GCHQ or any one of those authorities came to us and said, “We think your model is unsafe” or “We think it is being used by a bad actor,” we would look at that immediately with the utmost urgency. If we are in control of the model, we would stop the deployment of that model, pause the training of it or not deploy it. There are some nuances there with regard to how models in general are deployed. We do not always have control, if they are hosted by somebody else.

Chair

The model could be beyond your control.

Tom Duff Gordon

There would be others that would be able to do this. There is a very live and active debate in the US, as I am sure you are aware, around this circuit breaker or emergency brake. There is a lot of thought going into precisely who has the power at what stage to ensure that humans remain in control. I can give you the commitment that, if we are presented with evidence that our models are unsafe or that there is a genuine frontier risk, if we have the capability to do so, we will make sure that we pause or stop that model.

Chair

Lord Booth-Smith, what do you do when Ken McCallum rings with a warning?

Lord Booth-Smith

If any agency brought forward any evidence that the model was in any way dangerous, we have systems in place to stop it, unless it is being hosted elsewhere, as Tom was saying. That is a partner issue. If it is being hosted somewhere else, you clearly have a relationship with that organisation. It would be the entity responsible for doing it. We have already made decisions about model releases where we feel capability is incredibly high. With Mythos Preview earlier this year, we were confronted with the choice of general release or not. We chose not to do that and instead decided that a closed programme, where we phased in critical infrastructure providers, was helpful because of the cyber capabilities that the model represented. If any agency came forward with demonstrable evidence that the model was not safe or was dangerous, we would, of course, take it off the market.

Chair

It sounds like you share Ken McCallum’s anxiety about the risks in the future.

Lord Booth-Smith

If we go down the wrong path, if we do not make a set of choices that we need to make and if poor behaviours manifest and then embed in power brokers and decision makers across industry and elsewhere, it is entirely plausible that we will end up in a very bad place. It is not inevitable. We have agency. That is why some of the things that we are discussing today and that were mentioned by witnesses earlier are incredibly important. Questions such as third-party embedded evaluators are very important. Very specifically, they should have the power to disagree with the companies and make that public, if they do disagree. There is a question about what should be in statute or not and what level of international collaboration there should be. There is a whole corpus of work that needs to be done. Again, what we need is the time and space to do that. If we slow down the development, we will create that time and space. If we build at the safety of the tech, we will have that time and space.

Chair

It sounds like you are in agreement with the head of MI5.

Antonia Bance

We have spoken a few times in this session about mandatory incident reporting. I appreciate the clarity from Mr Duff Gordon. Are your companies in favour of the UK Government legislating for mandatory incident reporting?

Lord Booth-Smith

We are in favour of binding commitments on incident reporting. We published earlier in the year something called the Advanced AI Framework, which was a very large document that was our attempt at thinking through how a sovereign entity would think about creating rules and governance structures around the technology. Within that, we talked about the concept of incident reporting and how one would go about thinking about governing that as a problem.

Owen Larter

We are very supportive of incident reporting. We think this is very important. We think there is a discussion that needs to be had around standardised frameworks for incident reporting so that the information that we are sharing is helpful. You also want to make sure that you are not overwhelming Government authorities by reporting an enormous amount of incidents that are not particularly helpful. We think this is a really important part of the governance structure.

Antonia Bance

You are going to have to clarify for me: was that a yes or a no?

Owen Larter

Yes, we are very much in favour of standardised incident reporting.

Antonia Bance

Should it be mandatory and set out in statute?

Owen Larter

You would have to look at the details and get the details right. You would have to make sure that standards were in place to be able to make it effective. We are very supportive of incident reporting.

Antonia Bance

I am sure Government agencies would have the capacity to deal with whatever incident reports you needed to bring forward.

Rob Sherman

With regard to cyber-security incidents, our company has had a mature programme for the better part of 20 years around this. That has included, for the past 15 years or so, a bug bounty programme that encourages security researchers to find and disclose vulnerabilities. This practice of having a process to identify, respond to and disclose vulnerabilities is something that we are quite familiar with. We also, as your colleagues have mentioned, notify people and Governments, when appropriate, if there is an incident. With regard to what the right statutory framework is, I would point out that, in the UK’s agreement that it brokered in the summit in Korea about two years ago, there was a framework for a home country that would be the recipient of notification and be responsible for co-ordinating with other Governments. That is one potential mechanism to deal with some of the challenges that my colleagues talked about around making sure that there is a front door for notification and that it is triaged appropriately. I would submit that it probably is for the Government to evaluate what level of notification it wants versus how it wants to work with other Governments.

Antonia Bance

That was a qualified yes, I think. We have heard from other members of the panel that things that were set two years ago have required four updates in the intervening years. I am sure you could accept that something that was agreed a couple of years ago may not be appropriate for the future. Will all your companies now commit to immediate notification to the AI Security Institute followed by a published account of the incident, its impact, the remedial action taken and whether comparable systems remain in deployment?

Lord Booth-Smith

Could you repeat that question? There was a lot of information there. I do apologise.

Antonia Bance

Will you now commit to immediate notification to the AI Security Institute of any incidents followed by a published account of the incident, the impact, the remedial action and whether any comparable systems remain in deployment?

Lord Booth-Smith

We completely commit to informing the UK AISI of any relevant incidents for which it is the relevant body. To the question of publishing beyond that, generally speaking, we have done so in the past. There is an issue around revealing vulnerabilities. That is something that you would have to do in conversation with any affected entity. It is very hard to commit to blanket publishing things without having a conversation with the entity affected. To give you one example, we did contact the AISI prior to last week’s notification, even though it was not affected.

Tom Duff Gordon

We already publish misalignment notices. Any evidence that we have that models have behaved in a misaligned way will go up on our misalignment blog. That already exists and we have already been doing that. When it comes to notifications for cyber‑incidents, maybe one caveat would be that we need to define the geography. If these are UK incidents, we would report them to the relevant UK authority. We are currently in the process of doing that. As I mentioned before, there is a really legitimate question about publishing a vulnerability that could effectively impact the entity that has been hacked or accessed. We already have a misalignment blog and we have committed that we will be prompt and transparent in notifying the UK Government if one of our agents has accessed a UK Government website in a way that we did not intend.

Owen Larter

We think information sharing is very important. If we talk about specific incidents, such as the one we discussed earlier in the session, we informed the UK AI Security Institute about that. A lot would depend on the specifics of the incident, where it took place, the nature of it and what type of risk it is. It would not be particularly appropriate to publish an enormous amount of public information around a biosecurity risk or cyber-security risk. The devil is a little bit in the detail on the specifics of the incident.

Rob Sherman

I agree with my colleagues that the details matter here, particularly identifying what incidents are in scope and making sure that any public disclosures are done in a responsible way that does not increase risk. With that said, I will absolutely commit to talking with AISI about what notification and disclosure it thinks is appropriate.

Alison Griffiths

I am very interested in parity with the US. I would like to start with a yes/no answer from each of you on whether you will commit to providing all new frontier models to the AISI for testing on equivalent terms to those offered to US authorities. Could I start with you, Mr Sherman?

Rob Sherman

As I mentioned to your colleague, the understanding that I have of the arrangement between the AISIs is that we provide our—

Alison Griffiths

I am asking whether you will give AISI equivalence to what you already do in the US.

Rob Sherman

My understanding of the relationship between the AISIs is that we provide our models to the US AISI, which is called CAISSI, and it has a relationship with the UK AISI and potentially others. It has the ability to do that. We would have no objection if the US AISI wanted to share models with the AISI here.

Owen Larter

There is this ongoing Government-to-Government discussion between the US and the UK, which is important. We have provided our models to the UK AISI for the last couple of years. We think that is really important. Within the context of the Government-to-Government discussion that I mentioned, we certainly intend to continue providing our models going forward.

Tom Duff Gordon

Yes, I would give the same answer. We provide our models pre-deployment to CAISSI. We will continue to provide our models pre-deployment to AISI in the UK. We get value from that. As long as we are able to do that, under the US Government, we will continue to do it.

Lord Booth-Smith

My answer would be very much the same. We are currently providing unrestricted access and we have a strong desire to continue to do so.

Alison Griffiths

Lord Booth-Smith, why was Mythos 5.1 not provided to AISI before release? Was the decision Anthropic’s or was it driven by the US Government?

Lord Booth-Smith

In the spring of this year, as a result of Mythos Preview becoming available in a closed programme called Project Glasswing, a whole range of conversations were activated across the US system. That was incredibly positive because it encouraged greater levels of awareness and engagement. As those relationships developed and people became more aware of the technology’s capability and how our testing regimes worked, we were in a position with 5.1 where we unfortunately were not able to provision full unrestricted access to the model to UK AISI at that time.

Alison Griffiths

Why was there a difference between the US and the UK?

Lord Booth-Smith

I do not know. That is a question for the US Government. I was not privy to those conversations.

Alison Griffiths

It was a US Government decision rather than an Anthropic decision.

Lord Booth-Smith

No, you asked, “Why are the US Government different?” and I said I was not privy to those conversations. I am sorry about that. I am happy to write with a bit more detail on it.

Alison Griffiths

If you could, that would be helpful. Thank you.

Lord Booth-Smith

AISI has access to it now. At the time, it was offered restricted access. Obviously, that is slightly different from having unrestricted access, so it made the decision not to use that.

Alison Griffiths

Mr Larter, why was Gemini 4 Argon not given to the AISI for pre-release testing? Again, was the decision yours or was it driven by the US Government?

Owen Larter

Some of the reporting out there on this has been inaccurate. We did provide 4 Argon to the UK AISI pre-release for testing. It was nice to hear the AI Minister reference that testing yesterday as well.

Alison Griffiths

How did that misreporting come about? How quickly was it corrected?

Owen Larter

I am not sure why the misreporting happened, but it was inaccurate.

Alison Griffiths

Was it corrected quickly?

Owen Larter

Certainly, yes. It should be very clear that we support working with the UK AISI and that we provided Gemini 4 Argon to the UK AISI before public release.

Alison Griffiths

This question is to each of you. Is second-wave evaluation after US testing a meaningful safety assurance or does it simply mean that the UK is asked to comment once the decisive development and launch decisions have already been taken? Perhaps I will start with you, Mr Sherman.

Rob Sherman

It is absolutely a beneficial and critical thing to do. The reason is that we often talk about these risks as binary: either this risk exists or it does not exist. In reality, it exists along a spectrum. Capabilities increase and our ability to defend against risk and the risks themselves increase incrementally. It is extremely important. As you have heard, the UK AISI in particular does evaluations that other Governments do not. They are really novel and a contribution to the field. Having an understanding both as a Government and as an ecosystem of where technology is going, what the capabilities are and how to measure it is an incredibly important thing.

Alison Griffiths

Would you make changes, if the UK identified issues at a later stage?

Rob Sherman

Yes.

Owen Larter

We are very clear that UK AISI testing is absolutely critical, in the same way that our partnership with the US equivalent, CAISSI, is really important as well. Testing is a big part of that. There are the ongoing research collaborations that I mentioned as well. We are able to work with technical experts in AISI to work out how you mitigate risks once you have identified them. UK AISI is incredibly important across the piece.

Tom Duff Gordon

I would say the same thing. The important thing is that both CAISSI and AISI get the model pre-deployment. As long as they do, the AISI testing is very meaningful. As I have said before, AISI has deep experience in certain areas, for example bio. Historically, we have collaborated with it, listened to its feedback and changed our models. We think it is an important role and we are happy to continue providing our models.

Lord Booth-Smith

AISI is best in class for what it does. In particular, it has some very specific verticals of research that it is very good at, which others do not have. It is very good on alignment questions. It is a fantastic institution that is staffed by some of the best researchers in the world. In answer to the question about whether it is marking after the fact, no, the work of AISI makes a meaningful contribution to the model development process. We do take on board comments that it provides pre-release on models.

Alex Sobel

We heard from Daniel Kokotajlo on the first panel that AISI’s testing at pre-deployment stage would not be sufficient and that, if it was to have any effect, it needed to be involved at an internal development or deployment stage alongside your own safety staff. Are you prepared for AISI to be involved at that very early stage in testing or would you only allow it to test at pre-deployment stage?

Lord Booth-Smith

We are very happy to have a close relationship with the AISI. There is currently a conversation going on between the two Governments about what the framework should look like for things such as pre-deployment access. I really would not want to cut across that because they are having that conversation, not me. The idea of UK AISI being much closer to the work that we do is a very positive thing. The exact nature of that I cannot give you a concrete answer on.

Tom Duff Gordon

I am not sure that we have considered that very precise question, but we have said that we are open to embedded third-party evaluators that would have access before that pre-deployment stage. We also have a very good collaborative relationship with AISI. This is a question that we will look closely at.

Owen Larter

The UK AISI partnership is incredibly important. It is important to understand the nature of the relationship now. It is not just like we go away for a long time and then come back every now and then once we have a model to test. There is ongoing discussion between our technical experts working right across the development and deployment pipeline with the AISI technical experts.

Rob Sherman

As the others mentioned, I am not aware that this is an arrangement that has been proposed, so we have not looked at it. In general, collaborating more with UK AISI would be a positive thing, and we would be happy to talk with it about what it thinks would be appropriate.

Chair

That has been an incredibly informative session. Thank you very much indeed. At the core of this debate is whether we have the right regime for trust. I am not sure any of us on this side of the table is convinced that today’s regime is going to maximise trust in the way that is needed. There are differences of opinion between you about the role of statute and whether we can develop existing laws, in particular around liabilities. Most of you are of the view that stronger international conventions are, however, essential. You will know that yesterday Ministers merely announced that they are considering not committing to new regulatory powers. That is the panel to which we will now turn, but that concludes this session. Thank you very much, gentlemen. It is much appreciated.