Oral evidence: Immersive and addictive technologies, HC 1846

Oral evidence, 15 May 2019.

Chair

Good afternoon. If I could call this meeting to order of the Digital, Culture, Media and Sport Select Committee. This afternoon is a further evidence session in our inquiry into immersive and addictive technologies. We are delighted to welcome representatives from YouTube to give evidence to the Committee this afternoon. You may be familiar with the White Paper the UK Government have published on online harms. At the heart of that is the principle that social media platforms have a duty of care to their users to make sure they are protected as far as possible from engaging with harmful content. Do you accept that principle?

Marco Pancini

Thank you very much for the question. Let me introduce myself. My name is Marco Pancini. I am Director for Public Policy for YouTube for Europe, Middle East and Africa. The White Paper is a very important piece of policy-making. For the first time, a Government came with a comprehensive overview on all the different online issues and came also with some interesting solutions. We believe, for example, that the idea of a regulator could be a great opportunity to build the trust between the institutions and the industry in discussing and tackling the specific issues related to the internet. We have seen that in other sectors, and it was a successful way to find concrete solutions. We are really looking forward to participating in the public consultation with our ideas. In answer to your question about us taking responsibility—we want to take our responsibilities. It is in our business interests. It is a priority for us, as our CEO stated recently. He repeated it at several opportunities. Creating a safe platform, creating a safe experience for our users, is our responsibility.

Chair

You believe that it is your responsibility to ensure that harmful content, content that should not be there, is removed in a timely fashion?

Marco Pancini

Indeed, and let me describe our approach to that. We have a comprehensive approach to tackle harmful content online. We have invested in people. We now have more than 10,000 people working on safety on our platforms. We invested in the best of our technology to make sure that together with people, technologies can identify at scale violation of our policies. If we look, for example, at the first quarter of this year, of the 8.3 million videos that we took down, 6 million were taken down thanks to the machines that we are using in order to send us a referral of violation of our policies. That is very important. Thirdly, we need to keep our policies updated. We need to keep on keeping track of the development of online harms, and that is what we are doing. In the last six months we updated something like 30 of our policies, and we constantly do that.

Claire Lilley

My name is Claire Lilley. I am Public Policy Manager for Google, with particular responsibility for child safety. In terms of the White Paper, we were really pleased to see the focus on one of the key codes of conduct that the Home Office is driving forward and pushing through, which is on child sexual abuse and exploitation. We really feel that this is an area where we have tried to push the boundaries and lead the industry towards providing some solutions free of charge, and some technological solutions to ensure that that sort of content is removed as quickly as possible. It is true to say that because that content is clearly illegal, that is relatively straightforward in some ways because the definitions are very clear. The point I want to make is that we have not waited for regulation in order to create those technological solutions. We have been really, as a company, striving to drive forward and to push the boundaries of what is possible and to make that as available as possible.

Chair

I know we have some questions on child safety coming up later on, which probably lets us explore that in a bit more detail. I just wanted to stay on this principle of duty of care. Do you believe that duty of care relates to not just removing content that has been reported to you for being in breach of your platform policies, but also to proactively identifying that content for yourself, not waiting for referral?

Marco Pancini

That is a very important development. We made important progress in this area in the last two years. We have now an internal team that is proactively looking for new trends of online harms, studying continuously the challenges that are put forward by people that want to exploit our platforms. The head of this team is a former deputy intelligence expert from the FBI who is reporting directly to our CEO. We have a team, part of this broader team, also here in London, and that is a way for us to work in a proactive way and study how online harms are developing. That is an example. Just to give you another example, when we identify a video that is against our policies—some of the ideas that are of your concern, of course—we take these down. We make sure that if the same video is uploaded again on our platform, we proactively block the video. We do not let the video go up online. I think that being proactive is part of the responsibility that the company likes to take.

Chair

Looking at the Christchurch terror attack and the way in which platforms like YouTube were exploited to spread footage of that, do you feel the systems you have in place are adequate?

Marco Pancini

First of all, it was a horrific moment for us. It was a shock. When we woke up in the morning, we took over from our colleagues in the region to work on this terrorist attack. We were talking with the New Zealand Ambassador a few days ago in Brussels, and exactly this was the question: what kind of measures can companies like YouTube put in place in order to make sure that we can be more prompt in making sure that these kinds of content are not spread? To answer your question, of course what happened on the day was very specific, but still there are some areas where we were able to be on top of the challenge. First and foremost is working together with the authorities and the law enforcement. In the past years we developed a relationship and contacts, so now, when something like this happens, in a matter of minutes, law enforcement can get in touch with us and alert us that something is happening. For example, we identified the original video in a matter of minutes. That is working. Another thing that also is working is the use of technology to identify the video itself but also part of the video, and that is what happened. At the end of the day we identified something like 20,000 pieces of the same video that were uploaded on our platform and we took them down. The third area where we have good collaboration in place is between the different companies of the sector. We exchange information, but of course there are areas where we have to improve. One is our policies. We decided, because of the experience of Christchurch, that if events of this kind happen, instead of letting our machines identify pieces of a video that is depicting an attack and sending those videos for review to our team, while at the same time leaving the content to be uploaded online—now, if something like this happens and the machine identifies pieces of the video, we block these videos. We review the videos. Only after we check that it is an authoritative source that is reporting the facts do we let the content go up online.

Chair

I know we will ask some more specific questions on Christchurch later on, but I just asked at the beginning in context to this. Often when we have had evidence sessions with companies like yours, we are given lots of statistics about how much harmful content is identified and removed. Then something like Christchurch happens, and it appears that those systems, when they are required to operate at scale and at speed, are not operating at a level that we would find satisfactory. Do you feel that that has demonstrated that more needs to be done, and that, as they operate at the moment, that level of adequacy is not high enough?

Marco Pancini

One piece of evidence to show that we think something more has to be done is that today we are signing, together with the President of France and the Prime Minister of New Zealand, a pledge, a call to action, to make sure that we, as an industry, come up with solutions in order to be ready in cases like Christchurch. To answer your question, yes, we have a plan and we have measures that we want to implement, including making it a privilege to livestream content on our platforms, because of this interaction.

Chair

I suppose the question a lot of people ask is: why does it take a terrorist atrocity like that to make companies like yours sign a pledge?

Marco Pancini

We fully understand the question. What happened on the day for us was a huge learning. We learn from the failure of some of the systems that we put in place, and that is why I am saying that changing our policy is a way to address the risk that something like this can happen in the future, and make sure that the content that is related to a video uploaded of this nature is blocked as soon as possible. To show that that is something that we are working on seriously—for example, when we saw what happened in Sri Lanka, in a situation that had some similarities, we were able to be up to the threat.

Chair

From our point of view, when we last had evidence from YouTube in February last year, we raised questions about harmful content and the removal of harmful content. We were given impressive numbers and told that everything that could be done is being done, and then something like the Christchurch attack happens, and then the response is, “There are new pledges, new commitments, offers to do more”. That leaves us wondering, why does it always take some terrible event to make the companies recognise that they need to do more in this case? I have some more questions on that later so I will move on. We are talking about harmful content and a duty of care towards content itself. Do you think the duty of care should also mean the company has a responsibility to make sure people aren’t watching for excessively long periods, particularly younger users? Do you think there should be a function where people are prompted to take a break from watching?

Rich Waterworth

I can talk to that. To introduce myself first, I am Rich Waterworth. I am the Marketing Director for YouTube. I will come to some specifics about products and tools that we have built to do exactly what you are suggesting, but to set the context, to emphasise, as Marco was saying, we really, truly believe that we have and want to have a responsible duty to users to make sure they all have a really positive and healthy experience on YouTube. The vast majority of users do. As part of that responsibility, we recognise that there may be instances where people need some help in managing their relationship with any media, including YouTube. We have built and now have a broad range of tools across YouTube, across Google properties and across Android, the mobile platform that Google runs. On YouTube specifically, we have a series of wellbeing tools, one of which is exactly what you are suggesting. Users can set a timer to deliver a notification after a set period of time. The video will pause and prompt them to think about whether they need to take a break. That notification can be turned on by users in settings. The default for that is that that would be delivered after 75 minutes of continuous viewing, but the user can adjust that to set it for a higher or lower time. That is one tool that we have. We also offer users a dashboard so that they can see how much time they have spent watching YouTube by day or by week, so you can get full daily visibility of how much time you have spent on the platform. We do absolutely recognise that tools like that may be valued by users, and we have built those and we think that is an important thing to offer.

Chair

I have seen it reported quite widely that 70% of the film time that people watch on YouTube is not films that they have searched for but films that the platform has recommended to them. Is that correct?

Rich Waterworth

Yes, that is correct. That, as you say, has been reported externally. Around 70% of consumption is coming from recommended videos, and the remainder, about 30%, is coming from a combination of people searching for videos, people going directly to videos on channels or on their homepage that they know, or people subscribing to channels—that is an indication of saying that you want to follow a channel, in which case that channel will show up in your feed on YouTube. A user can also choose to be notified when a channel uploads a new piece of content. That is the breakdown of consumption.

Chair

The functionality is that—perhaps you can confirm this is correct—you use artificial intelligence to assess the viewing history of the user against the profile of the new piece of content, or any piece of content on there, and use that to make a match that is then used to make a recommendation to the viewer. Is that how it works?

Rich Waterworth

That is broadly how it works. Up Next is a tool that takes, as you say, information about the video that is being watched and will find a range of suggested next videos to recommend. If a user is logged in, then the algorithm will use information related to prior content that has been watched or that it knows that user might be interested in. We have made significant changes to that algorithm over time. Over the last year or so, that algorithm has evolved from being focused on recommending a piece of content that someone would go on to watch—users gave us feedback that they were finding that there were too many recommendations in the list that were too similar. We do surveys on the site and we look at other signals such as how much of the content that they are being recommended they watch. Do they like that content? We realised that a narrow set of recommendations was not creating as high satisfaction among users as we wanted, so we have evolved that algorithm to recommend a broader range of content. We have also evolved that algorithm to demote certain types of content, such as click-baity type content. There are a number of ways in which that algorithm has evolved and continues to evolve.

Chair

It is quite an astonishing statistic that 70% of what people watch is content that is recommended or prompted to them by YouTube, rather than things they are actively looking for. It would really demonstrate the power of the platform to keep people watching for as long as possible. They maybe go on to look for one thing or they are being recommended one thing, and then they are watching more and more and more as it is recommended to them by the platform. It would suggest that the platform is remarkably successful in keeping people on for as long as possible.

Rich Waterworth

Up Next is a feature that users value really, really highly. There are a number of different ways that it is used. A very common use case on YouTube or type of consumption on YouTube is around music. It is very common for people to listen to music on YouTube, and that can be for a period of time, in a similar way that they may listen to radio. Users find that the ability to have effectively a linear stream, where one track or one piece of content plays after another in a similar way to a radio station, is a very useful thing. In a similar instance, there are lots of examples where people are using YouTube to do cooking, when they have their hands busy, and being able to have a device and watch one programme after another again replicates the experience of a linear television channel. It is a highly valued feature for users.

Chair

Since YouTube has introduced the feature whereby AI is used to select videos on the homepage that people are most likely to want to watch, what has been the impact on viewing patterns since those changes came in in terms of the amount of consumption?

Rich Waterworth

Our main focus has been on how satisfied users are with the experience.

Chair

Yes, but I did not ask that. Are they watching more as a consequence of this feature being introduced?

Rich Waterworth

We would have to take that away and get some detail on specific patterns, because YouTube consumption has grown over time from a combination of factors.

Chair

This is from a new book, Chris Stokel-Walker’s book, “YouTubers”. I don’t know if you are familiar with it. It states that in the three years since Google Brain began making smart recommendations, watch time from the YouTube homepage has grown 20-fold. Is that correct?

Rich Waterworth

I would have to check if that number is correct.

Chair

Does it sound right? You are a marketing director. It is pretty core to the business, I would imagine.

Rich Waterworth

I would have to go back to that specific and we can get details and come back to you. I do not know if that is—

Chair

If that was the case, that is a pretty dramatic shift in viewing patterns as a consequence of using AI to more accurately guess what people want to watch. You might say it is clearly a feature people like. Evidently so, but I just wonder if people are being drawn into ever-lengthier sessions of watching because the machine is so clever at constantly tempting people with new things.

Rich Waterworth

It all comes back to whether users are finding content or being recommended content that they enjoy and having a good experience. There are a number of factors that impact on how much YouTube is consumed. One of the really big factors over the last number of years has been the growing availability of smartphones, and YouTube consumption on mobile devices is now a very large majority of consumption. People have that availability. There are a pretty broad range of factors that have driven and continue to drive YouTube usage and growth, and that is one of them.

Chair

A 20-fold increase in three years is pretty dramatic, even in a fast-moving market.

Rich Waterworth

If that is the case. I don’t know if that number is correct.

Ian C. Lucas

Can I just ask you, Mr Pancini? Does YouTube now accept that it is legally responsible for the content on its platform?

Marco Pancini

If we look at the legal framework in which we are operating, which is the Commerce Directive, there are different responsibilities according to the role that the platform is playing in relation to the content. I will give you two examples. If a user is uploading a video on YouTube, the user is the content provider—they are the author of the content and responsible for the content. We are also carrying a responsibility, which is to make sure—

Ian C. Lucas

That person may be paying YouTube or you are securing income from that process, so you take that responsibility. You are happy to take that responsibility. My question to you is: are you responsible for the content of your platform? I want to be clear on this because I think it is a very important question.

Marco Pancini

Indeed. Our important responsibility in relation to the content that is uploaded on our platform is to make first and foremost available tools that allow every user, everyone that is interested, to report the content. Whenever the content is reported to us we need to take action, but on top of that we have also responsibility, which is sometimes it goes beyond—

Ian C. Lucas

This is why I am asking you the question, because there was a time when platforms were saying, “The content of the platform is not our responsibility”. That was largely the position that we were being told perhaps back in February of last year. That position seems to me to have changed. That is not your position anymore.

Marco Pancini

Let me clarify because it is a nuance, but it is an important nuance.

Ian C. Lucas

It is important, but I want a clear answer to the question.

Marco Pancini

There is a difference between liability and responsibility. Liability means that if I am posting content on a platform online, on every platform, which contains an illegal activity, something that is illegal, I am responsible for that as an individual. The platforms carry some responsibilities as well, so I am liable, just to make it very clear, to stick with my distinction. I, as a user, am liable for the content of my video. The platform is responsible for a different aspect, which is not the content itself, but is making sure that first and foremost we provide a safe experience for users. Secondly—

Ian C. Lucas

No, no, no. Hold on. This is not a pick and mix process. Are you lawfully responsible for the content of what is on the YouTube platform? That is what I want to know. Yes or no?

Marco Pancini

We are not liable for the content, no. We are not liable for the content unless we are producing the content. For example, we have a service called YouTube Premium. We produce the content and we are liable for the content. Users or any individuals that are uploading content online are responsible for the content.

Claire Lilley

I think, perhaps, if I may, the shift that you are referencing is something that I have felt within the company, which is a real shift towards responsible innovation and being a leader in making sure that we are being responsible in the way that we are designing and developing the tools and the way that we are thinking about our users and the need to protect them. If I think about children, for example, we have really tried to be thoughtful and responsible in the development of tools that are specific to children. For example, there is our YouTube Kids app, which I can talk you through, or our Family Link app, which enables parents to manage their children’s use. I would say that responsibility is very much a theme that is running through the company at the minute and it is running from top to bottom.

Ian C. Lucas

That is good to hear, but you used to be a teacher. My wife is a teacher, and she talks to me a lot about social media and the impact on children, and I know your background in the NSPCC. Isn’t it a bit late at this stage to be considering the responsibility towards children? What I see in my constituency surgeries are increasing numbers of parents who are concerned about the impact of social media on their children and on their children’s mental health. It seems to me that an awful lot of people have made an awful lot of money before we started to think seriously about the impact of social media on the mental health of our children. Do you agree?

Claire Lilley

I think that is a very legitimate concern but I do genuinely think it is an area where Google has really tried to push the boundaries and to lead the industry. The creation of YouTube Kids, for example, was a recognition that YouTube main is not appropriate for young people. They need a more contained, safe environment. The investment that has gone into that is not to be underestimated. It is significant, and rightly so. It is also a sign that we want to show what is possible so that other players in the industry can do the same and provide children with good, quality content that is suitable for their age and stage of development. There are a huge range of issues. The Chair asked before about the impact of screen time on young people and so on. The evidence base is very much still developing. We certainly want to take a responsible approach to young people and their mental health. For example, we saw a report earlier this year from the Oxford Internet Institute, which looked at the experiences of 300,000 adolescents and found that the association between their technology use and their wellbeing was less than 0.5%. In fact, wearing glasses had a greater negative impact on their mental health. It is a very nuanced debate. There are a lot of factors at play here. What we as a company try to do is be very evidence-led in our approach, but also really understanding our responsibility to protect young people in particular.

Chair

Just a note for the record. The statistic I read out earlier about the 20-fold increase in use of the platform since Google Brain technology was being used to recommend videos on the homepage was also reported by the Verge in 2017 in an article that was focused on YouTube. It also included the statistic about the fact that 70% of videos are recommended. It is another source for it. It doesn’t seem to have been a fact that was challenged by YouTube either at the time or since.

Simon Hart

We have moved on but a quick reference to Mr Waterworth’s comments earlier on about a timer prompt process, which you referred to. I wondered if you had any indication of how many people, particularly young people, use that. How effective is it?

Rich Waterworth

It is a fairly new feature in the last year, just under a year. We do not have usage data on it.

Simon Hart

You have no idea if it works at all?

Rich Waterworth

As the marketing person, I do know that we have made tens of millions of people aware of that through promotional messages. A couple of things I do know that give an indication of the use of these kinds of tools. Within the Android set of wellbeing tools there is a wind-down feature, which enables Android users to set effectively a time limit when they want to be either asleep or going to sleep. You set that time, and from the time you set that, you do not get any more notifications on your phone, and the phone screen goes to black and white, to greyscale. We have seen just over a quarter—27%—reduction of usage among people who enable that setting. We do see that these settings have usage, and I can say that we get qualitative feedback from users that they really appreciate the fact that we have developed this and are telling them about this.

Simon Hart

As yet, you do not have anything you can give the Committee on the particulars? I maybe got you wrong, but did you say a 75-minute limit?

Rich Waterworth

Exactly.

Simon Hart

You have no idea whether that actually—

Rich Waterworth

I am happy to take that away and see if we can find any more detail on that, but I do not have that with me now.

Simon Hart

That would be useful. Very quickly on that, is the fact that you are introducing it a sort of admission that you think there may be something in the evidence that suggests that long periods of time onscreen may be having a detrimental effect on your customers? What is prompting you to go down this road?

Rich Waterworth

The reason behind creating tools like this is really linked to a very core design principle within the company, which is about giving users transparency and control over their usage across many different products, and it is definitely the case in YouTube. It is one of many ways we give users control over—

Simon Hart

It must be in response to something, and that is what I am trying to get at. I am suspicious that you are responding to a growing awareness perhaps that elongated periods of time onscreen may be doing a certain amount of harm. I wondered if these new innovations were a reaction to that or is it really just a gesture of goodwill on your part?

Rich Waterworth

As Claire was saying, the evidence base of screen time and impacts and so on is growing and developing, and we take a very keen interest in that and we think that is very, very important, but building those tools is not a response to a particular thing. We are very regularly building and releasing new tools and features of all different kinds, and that is one wellbeing tool I mentioned among a number within YouTube and a number within Google and Android. These things are developing both within the company and within the industry at large.

Claire Lilley

I don’t know, I am just surmising, but it is potentially because mobile devices have become in the last five years so ubiquitous in our lives, and we all need better ways of managing that. That has been a theme running through the tech sector generally over the last two years, I would have said. We saw other companies introduce similar tools.

Marco Pancini

One general principle of design of the Google product is to make technology transparent, to make sure that technology is there when we need the technology, not to become a slave of technology. That is something that our CEO, Sundar, has repeated several times. I think these kinds of tools are going in the direction of making sure users have full control over their day-to-day technology. They can activate technology when they need it but they are not hooked on or a slave to any technology.

Jo Stevens

Mr Pancini, can I just ask you a question? Earlier you were asked about whether or not you have systems to proactively take down content, and I think you said that you do. Where you have taken content down previously and it pops up again because somebody has uploaded it again, you have people who will then take it down again. Can I just clarify? Are those the only circumstances in which you proactively take down content?

Marco Pancini

I gave you two examples. One example is the case when we take down a piece of content and the same content is coming up again. That is a proactive use of our detection tools.

Jo Stevens

No, I understand that bit.

Marco Pancini

The second is, for example, the work of the Intelligence Desk. Maybe I can give you—

Jo Stevens

Yes, so you do have another system. That was what I wanted to clarify.

Marco Pancini

Correct.

Jo Stevens

You mentioned that you have that headed up by a former FBI official. How many people do you employ on proactively looking for content that is harmful so that you can take it down?

Marco Pancini

To clarify this point, the people that we have, these experts of the Intelligence Desk, are not just facing the platform looking for the illegal content because that would not be effective. This is work where indeed the machines make scaling a much more effective way. Just to give you an idea, of the 225 minutes of content that we took down in the first quarter of this year, 90% of it was flagged by machines.

Jo Stevens

Yes, but can you answer my question about how many people you employ? They may not be solely employed on identifying content to proactively take it down, but how many people do you employ where that is part of their role?

Marco Pancini

There is no one that is looking actively for violations, not because we do not want to have someone doing that but because it will not be effective. The experts that I mentioned to you before search for information about new trends of online harms using different sources. Something that they use is the observation of our platform. For example, the flags that we are receiving, but also they use other sources. For example, the interaction with law enforcement. The interaction with the experts and the trusted flaggers that are reporting content to us. They are experts in their own domain, so they look in academic papers. They speak with other experts. They collect all this information. They understand, “Okay, this is a new trend of online harm that is happening on our platform. Let’s start to find the solution with our machines to identify these and fight against them”.

Jo Stevens

How many people do you employ who do that work that you have just described?

Marco Pancini

In general, we have 10,000 people working on security. Part of them are working in this team.

Jo Stevens

How many?

Marco Pancini

I do not have this information with me. I am very happy to come back to you to clarify this information.

Jo Stevens

That would be very helpful. Thank you.

Claire Lilley

Could I outline some work that we proactively do in relation to child sexual abuse imagery? Just very briefly, we have three specific types of technology that we use to identify this content. We have one machine learning AI, whatever you want to call it, which identifies images that are already known to authorities, to the IWF and so on, and we run scans and remove those. The second one is a product we pioneered called CSAI Match, which looks for video content, which was industry-leading at the time. The third one is a new piece of technology that we just announced last year, which will look for new images. The reason it is important for me is these are images that have never been seen before, so probably they are less likely to be historic images. They are more likely to be images of children who are being abused today, and therefore offer us a greater chance of rescuing children who are being abused today. Those are the three things in relation to images. Fourthly, we also do proactive work in order to identify in comments any words that paedophiles might be using, and sometimes these can appear to be innocuous terms and sometimes they are very context-laden, but sometimes they might be different in different languages. We work with a network of what we call trusted flaggers across the globe who feed us the on-the-ground intelligence about what the latest terms are that are being used so that we can feed those into the classifiers, proactively identify these comments and remove them as quickly as possible.

Clive Efford

Mr Pancini, can you just explain to me so I understand a little bit better, how do you monetise time on your platform? How does it make money for you?

Marco Pancini

The business model of our platform is linked to advertising. Advertisers that find it interesting to promote their products on the platform can buy the possibility to display their advertising on our platforms. All the revenues that we get from advertisers, we share them. Actually, we give to our creators the vast majority.

Clive Efford

Let’s say I spent an hour on YouTube. Is it that you get paid for every advert that pops up on my screen? How do you make your money out of me seeing shoes or double glazing being advertised?

Rich Waterworth

The amount of money made is not linked to the time that you spent watching something. The way that advertising works is that some videos you watch will have usually a video advert before them—not all of them—and that will depend on whether an advertiser wants to reach a user such as yourself at that point in time with a certain product. A majority of videos will not have one of those, but the most common form of advertising you will see will be a video advert before a video you are going to watch. There are two types of video adverts. There is a very short video advert, which is usually six seconds, and there are longer video adverts. The longer video ads can be skipped by the user after seven seconds. If they are not interested in that, they can skip that and go straight to the content. There are also other advertising opportunities on the site. For example, sometimes there will be what we call a companion ad, which might be below the video that you are watching, which might offer information about a related product or service. Those are the most common types of adverts.

Clive Efford

Yes, but what is it that triggers payment to you?

Rich Waterworth

Every time an advert is served and viewed by a user, the advertiser will pay as part of the package that they have.

Clive Efford

The longer someone spends on your site, the more you are likely to earn?

Rich Waterworth

As I said, ads being served is not linked to time spent on the site, but that is how the advertising payments work.

Clive Efford

Mr Pancini, you told the Home Affairs Committee that your artificial intelligence is very precise, so how can you be confident that it is precise enough to deal with critical events such as the Christchurch shootings? What are you doing to improve it?

Marco Pancini

Indeed, if we look at what happened on the day, the machines that we have in place were able to identify all the different pieces of the original video. First and foremost, the original video was identified together with the police and we were blocking it, so it was not possible to upload the same video on the platform. What also happened is that our machines were identifying when users were uploading different pieces of the video, but some of these pieces of videos were also included in reportage from the BBC, from other sources, authoritative sources. Indeed, for us it is important to have on our platforms also authoritative sources, if a user is coming to YouTube and wants to know what is happening in Christchurch and searches for Christchurch, we want the user to see authoritative sources reporting of the facts. For this specific reason, when the machines were identifying the pieces of the original videos, the videos were going online and were flagged for review to our people. Because of the amount of content that was posted, it was not possible for us to keep up with this. That is why we changed our policies, and now if something happens again these videos will not go online. They will be reviewed by our people and only after approval they will go online.

Clive Efford

Just so I am clear about that, how will you know to delay posting that content? How do you know it is harmful? What is triggering it for you?

Marco Pancini

Let us go back to this example. When a video is reviewed and we see that it is against our policies, we apply a fingerprint, so a technological watermark, to the video. We put this video in our database of videos that are against our policies, that we identified as against our policies. When a user is trying to upload the same video, the system recognises the match between the original video that we took down and the video that somebody is trying to upload, and then the video is blocked. For example, if somebody is just uploading a part of the video, so not exactly the same video but a part of the video, there is a light that comes up and says, “Wait a minute. We need to review this video because it is linked to something that we saw before and we decided it was against our policies”.

Clive Efford

Correct me if I am wrong, that sounds like how you respond once you have taken the initial content down.

Marco Pancini

Correct.

Clive Efford

How do you identify and how quickly can you identify harmful content of that nature, or of any other nature for that matter, to take down the initial posting?

Marco Pancini

In the context of terrorist content, it is a matter of less than one hour usually, in the relationship that we have with law enforcement, with the relationship that we have with experts like the Counter-Terrorism Internet Referral Unit.

Clive Efford

Is that in response to being contacted by authorities or is that your own monitoring? How are you generally alerted to the presence of content on the platform?

Marco Pancini

Usually it is happening in real time. With the kind of relationship that we have with the Internet Referral Unit or Counter-Terrorism—

Clive Efford

You wait to be told to take it down.

Marco Pancini

It is happening in real time, so the content is identified, is sent to us and we take action against the content.

Clive Efford

Just to be clear, there are instances where you act without being asked or required to take harmful content down?

Marco Pancini

For known videos, for example, of terrorist content, we act. If we recognise the video, if the video is already in our database, we block the video. For new content, we work together with law enforcement and take prompt action.

Clive Efford

If an individual has a history of posting, let’s say, racist material on YouTube, is it just the racist content that gets taken down, or does that individual get taken off the platform?

Marco Pancini

First and foremost, if the single violation itself is so bad that we need to take action, we take action directly on the channel of the user, so the user is not able to use YouTube anymore. If it is a normal violation, then after the third violation the user is kicked out from the platform.

Clive Efford

The person who carried out the atrocity at Christchurch urged people to sign up to a site, PewDieWee.

Marco Pancini

PewDiePie.

Clive Efford

Yes. That individual has been accused in the past of posting anti-Semitic and racist material. Is he a suitable person to be on YouTube?

Marco Pancini

Every user has to comply with the same rules. In the context of PewDiePie, as with any other user, we apply the same set of policies. When we identified that PewDiePie was uploading content that was against our policies, we took action against his content. We apply exactly the same rules that we apply to every user also to PewDiePie.

Clive Efford

Against the content but not against the individual. He is still able to post on YouTube.

Marco Pancini

On this specific instance we took action against his content. He never happened to be in the position to have his channel suspended.

Clive Efford

Is there a wider moral issue, if people know the views of an individual that uses YouTube regularly, that you are normalising people who hold those extremist views? Does that cause concern for YouTube?

Marco Pancini

When we look at the content on our platforms, we make an analysis on each and every piece of content that is uploaded. That is the way for us to maintain an objective approach to the behaviour of the users of our platform, which means that we have a very strong set of policies. We apply the law. If we looked at the content that is uploaded on our platform, if the content is against our policies or of course against the law, we take action against the content. We do not look at the views of the individuals that are on our platform, even if in some cases we can disagree with them or they can be controversial. We look at the content.

Clive Efford

Is the viewing and following of extremist content profitable?

Marco Pancini

In all the cases that you made, so a user that has abused our platform—

Clive Efford

Is there a big customer base that uses YouTube that watches extremist material? Is that a disincentive for you to take it down?

Rich Waterworth

I can answer that as well. It is not in our interests and it is not good for our business. It is not good for our brand. It is not something that our advertisers want and it is not something our advertisers are serving ads against. One of the very important principles that guides how we deal with transgressions against community guidelines is that monetisation on YouTube is a privilege. It is not a right to everyone. Not all creators are able to monetise and monetisation is not granted to all. That is not something we profit from or want to, and it is not something that our partners want. It is absolutely not part of our commercial—

Clive Efford

If you do not profit from it, why is it there?

Rich Waterworth

YouTube is fundamentally, first and foremost, a content platform. We are proudly an open content platform. That brings enormous benefits to the world. People are learning things from people they have never met. They are finding new voices. Diverse users and voices are able to access audiences around the world. There are enormous benefits from that, but it is very, very important to qualify that it is not openness without any limits. We do have limits. We have guidelines that Marco has talked about. We introduced in 2017 a category of content called limited state. That is a category of content where someone may not have gone past our community guidelines, so a piece of content may not be violating community guidelines and it may not be taken down, but it might be close. It might be highly controversial. In that case, we would put that piece of content in what we call limited state, which means that it cannot be monetised, there are no comments around it, there are no likes, and it cannot be served up as a recommendation to any users. We recognise that there are lines, and we try and we draw a line where the openness of our platform ends. As Marco described, we work with third parties, with society and with academics to review those policies and make sure we have the lines in the right place, and then we are investing significant amounts to enforce those very aggressively.

Clive Efford

In answer to earlier questions about how many people are monitoring this, you could not answer.

Marco Pancini

Ten thousand people.

Clive Efford

Ten thousand people, but when we asked how many of them are watching the content, enforcing these new policies that you have put in place, you cannot tell us. Why is that?

Marco Pancini

No, that is the 10,000. Just to clarify, the number of 10,000 is about the number of people who work every day to keep the platform safe, including people who are reviewing flags we have received from users and flags that are generated by our machines.

Clive Efford

Can I ask you about your algorithms? YouTube is yet to reveal how its algorithms fully operate. Is there an opportunity to be more transparent and thus improve both the platform’s algorithms and the trust in them?

Marco Pancini

Transparency is absolutely very important. For example, when we look at Google, on Google we publish a website that explains how search works, where we explain in very plain words how the algorithm works. This approach is something that we are considering also for YouTube, taking into consideration also a piece of legislation that was approved recently by the European institutions called platform-to-business, which will imply specific obligations of transparency to platforms that provide services online. That is the direction we are taking. Taking into consideration the information we gave you today, for example, on how recommendations or Up Next works, our intention was to let you understand that is not secret. That is generating why we are suggesting a video, but it is based on previous views or previous videos that the same users or a user that has similar behaviour have seen before.

Clive Efford

How do we ensure that the algorithms are not biased? Without transparency, how can we be sure that that is the case?

Marco Pancini

Transparency is very important, so again—

Clive Efford

Why haven’t you—

Marco Pancini

That is the direction of travel, transparency, of course. Also in terms of algorithms, we made huge improvement in terms of looking at the data collecting implications of AI. I am very happy to come back to you, consulting our internal expert on all the works that are done.

Clive Efford

Do you accept that transparency in this area is absolutely essential for people to have confidence in the datasets that they are trained on and how the algorithms operate? Do you accept that?

Marco Pancini

I accept that transparency is important.

Clive Efford

You have yet to make that data available.

Marco Pancini

I am accepting that transparency is important. I will tell you more that in terms of the dataset, we are pioneering as Google, as the wider family, the work in this direction by making something called TensorFlow. Most of the datasets that we are using for our machine learning analysis are available for researchers to look into them.

Clive Efford

If we are going to regulate in this area, how is it going to be possible to regulate if we do not have that transparency? How do we know what we are dealing with?

Marco Pancini

I think here the key is to find the right balance between transparency and maintaining something that I think is also a fair principle, especially for other companies that do not have our size, industry secrets. Finding the right balance between transparency to final users and to the wider public and protection of important industrial secrets is the future in this.

Clive Efford

One final question. Do you use Google search to inform your algorithms to recommend films or videos on your site?

Rich Waterworth

If you are logged into YouTube, your usage on other Google products, including search, could be taken into account, in particular around the advertising that you might be served. We have a dashboard called myaccount.google.com where you can go and see precisely the data that YouTube or Google has and uses about you in relation to all of our products and services. For each and every one at a granular level you can turn that on or off. For example, you could see that YouTube is saying you are interested in cars because you have watched a piece of content around cars or you have consumed car content somewhere else. You could choose individually and say, “I do not want to be targeted by advertising around cars”, so there is a very, very granular level of control and visibility of what signals are being used.

Chair

Just a couple of quick questions on Christchurch, just to clarify a couple of things. You talked about the removal of content. Have you closed channels or cancelled accounts of anyone who was sharing the footage?

Marco Pancini

Yes, we have done. We closed also a series of channels. I do not have now the numbers. We are very happy to come back to you. As I said, 20,000 pieces of content were taken down by the end of the day, and we closed also a few accounts of people who were spreading misinformation or even glorifying the attacks. We did, and I am happy to come back to you.

Chair

If you could write to us with that information.

Marco Pancini

Yes, of course.

Chair

Thank you. Do you know what the maximum number of views for any one film was that was shared of the Christchurch attacks?

Marco Pancini

You are talking about the original video or other pieces of—

Chair

Any of the films, so any of the films that were shared, that were taken down, what the maximum number of views for one of those films was.

Marco Pancini

No. I do not have this information. I can search for this information.

Chair

Also, would the YouTube algorithm have recommended that film to other users as well? People were using basic search terms to find the footage, but is it possible that some customers would have had a recommendation that prompted to them to watch?

Marco Pancini

The only possibility that some content related to the attack would be recommended is if this was an authoritative source like the BBC or other kinds of broadcasters. Of course, none of the videos that were identified by our machine for review would be eligible to be recommended. That could not happen.

Chair

Have you done an audit looking at the way in which the footage was shared, maybe to identify networks of people who take a particular interest in this sort of footage—the sorts of people who are interested in terror events and may be of interest to the authorities? Have you done a full audit of the platform?

Marco Pancini

We made an analysis of what happened on the day that I am very happy to share with you, including the information that you asked about.

Chair

Thank you very much. Just one final thing on the algorithm. One of the concerns about the algorithm is it seems to direct people towards odd content, and there are big concerns about the way medical conspiracy theories and anti-vaccine campaigns have been promoted through YouTube. Also, on the day of the Notre Dame fire, there were reported instances of YouTube recommending films relating to the 9/11 attacks in New York, which have nothing to do with the Notre Dame fire but would suggest that somehow it might be a terror attack or might be linked. Why would the algorithm be doing that?

Marco Pancini

Let me take a step back. Why are we working specifically on creating some additional information when users are looking for breaking news? We are doing this because we want to make sure that when a user is coming on the platform and is looking for something that is happening or news-related information, we provide to them authoritative sources. Also, if somebody is looking for events that could trigger some conspiracy theories or some questionable results, we want to make sure that still the users can see what authoritative sources, like Britannica Encyclopaedia, are providing in relation to what the user has searched for. Going to what happened in relation to Notre Dame, the algorithm linked the picture of the fire in Notre Dame to September 11. It is a mistake. Machines make mistakes. Humans make mistakes. What is the solution? The solution is to make sure that humans and machines work together. In this case, what happened is that the combination that was triggered—again, the picture of the fire—was solved by humans looking at these mistakes made by the machine and solving it. After one hour this bug was solved by our engineers.

Chair

Yes, I understand that. The concern or the suspicion is that the algorithm has a bias towards conspiracy theories, sensationalist news and, therefore, it makes these connections.

Marco Pancini

No, it is the other way around. Let us say a user was searching for September 11, the idea is for us to provide a panel, a box near the content saying, “Look, this is what happened”, and you get alternative sources. The problem is that this box was triggered. In order to address your concerns, we wish to ensure that if some conspiracy theories are present in the search page—something that we are also trying very hard to avoid—there are also alternative sources that are displayed prominently.

Chair

I understand that. The reason I asked the question that way around was, if you are not just looking for information about 9/11 but you are looking at another news event, and then alongside that subtly YouTube is promoting information about another atrocity, or another thing that took place, there could be an inference or suggestion that somehow there could be a pattern between them.

Marco Pancini

No, no, that was just a mistake on this single case.

Chair

Yes. But the concern is that there seems to be a lot of mistakes where conspiracy-theory footage is promoted through the platform, and there is a slight concern as to whether that is because there is a bit of bias. Because some people might find conspiracy theories more sensational, they might drive bigger viewing figures and, therefore, the platform directs you towards some of that footage.

Rich Waterworth

I completely understand the question in terms of what that looks like. Just to be really, really clear, the watch-next algorithm is not designed to recommend more extreme or extreme content. Also, back to my earlier point, it has evolved and continues to evolve. The way that it is designed is to recommend a range of content. Some may look or feel more extreme, some will look less, some will be closely related to what is being watched and some will be more distant.

Chair

It is quite simple, it seems to me, that the next-up function is, “If you liked that, you will like this. If you take an interest in that, you will probably be interested in this”. If you have an interest in extreme content then you would be recommended more extreme content.

Rich Waterworth

I was going on to also say, we have made changes to the algorithm to reduce what we call click-baiting content, and we have made changes to promote authoritative sources so that it will positively promote authoritative sources. We are very conscious of managing that dynamic, and we are absolutely designing that product and that feature so that it promotes authoritative sources, so that it promotes a range, and so that you do not drive people into a single source of truth. We are very consciously managing, building and evolving that algorithm to ensure that precisely the scenario you are suggesting is not the case.

Marco Pancini

Just to give you an idea, the majority of the views that go to broadcasters on our platform, so coming from Up Next in the recommendation that we are providing, it is in our interests to make sure that quality content and authoritative sources are promoted.

Claire Lilley

Just one more thing from me on the recommendation system. One of the use cases is parents watching YouTube and then co-viewing with their children. If they are watching what we categorise as family-friendly content they will continue to just get family-friendly content. If the parent then later that evening watches a cooking video, or a DIY video, or whatever, while they are watching the family-friendly content it will never go to the DIY content or the cooking content. It will stay within that backing. That is one of our more recent developments to really cater for how people are using the platform in the real world.

Jo Stevens

I am going to go back to removal of content; I have just one question about that. When content is removed and you have identified it as being harmful, is that removed by a person or is it removed by an algorithm?

Marco Pancini

No, no, it is always removed by a person.

Jo Stevens

Always by a person? Always? Okay.

Marco Pancini

Always.

Jo Stevens

Thank you. I want to move on then to the recommendation system that we have talked about. Do you recommend videos with political and current affairs subjects?

Rich Waterworth

Yes. Those videos are included. We have seen over a number of years the consumption of current affairs content has grown on YouTube, alongside uses around music and all sorts of other things. We have seen a growth in that, and yes, that is included. That is one of the main reasons why we have taken some of the steps that we have to really closely manage Up Next and what is being recommended.

Jo Stevens

Yes. Do you profile users politically for those recommendations?

Rich Waterworth

No. The recommendations are based on what is being watched. If you are logged into YouTube it will take and include information on past content you might have watched. The recommendations are designed to recommend a range of videos, not, “You have watched 10 videos like this; we must recommend to you 10 more like that”. The recommendations are designed to provide a range of options.

Jo Stevens

Okay. Because the platform has come under criticism for conspiracy clips relating to politicians. When we spoke to Juniper Downs when we did our evidence hearing in the States at the beginning of 2018, she accepted that there was still work to do and progress to be made in terms of improving authoritative sources of news and moving away from that conspiracy-theory stuff.

Rich Waterworth

Yes, and that has been a big area of investment and focus over that period of time. Not to repeat myself too many times, but we have introduced specific changes to the algorithm to manage that as closely as we can. We think that is a really important area. One of the recent areas that I do not think we have mentioned yet is that conspiracy-type content is demoted by the algorithm now. That is a change that we have specifically made, so that if it is anti-vaccine content, or other sorts of content that have been flagged and are clearly of concern, those are demoted.

Jo Stevens

Okay. Another criticism that you have had is from a former employee, Guillaume Chaslot. He said the algorithm he worked on, “heavily promoted Brexit, because divisiveness was efficient for watch time, and watch time leads to ads.” How does that correspond with what you have described as your broader societal aims and the values behind your company and your responsibilities?

Rich Waterworth

Yes. I think it very much speaks to the journey that we have been on and are on with both Up Next and YouTube. The absolute focus for us as a company from the leadership all down through the company, and the way we design products, is around our responsibility to our users, providing a high-quality and good and positive experience for users. The changes we have made exactly talk to this fact that we do not, in that algorithm, just prioritise watch time. We prioritise users having a good and valuable experience. That means that we take feedback from users, qualitative feedback where they will fill in a survey while they are using a site, telling us whether it has been a good experience for them, as well as other signals relating to their engagement with the content. Those are the things that determine whether we feel that tools and features are working or not, or are they just driving watch time.

Jo Stevens

Okay. Obviously, we have elections a week tomorrow; the European elections. You have talked about the steps you have taken around authoritative news. When did you introduce those steps?

Marco Pancini

We will go back to your question, but for us the European elections are very important, and we found the findings from your report on misinformation extremely interesting and in line with the work that we are doing together also with the UK, participation in the context of the code of practice on this information with the European Commission. Some of the steps that we are taking go in the direction of providing more transparency around political advertising. We are now making available every week information on how much money is spent for political advertising in Europe, with a breakdown country by country. If you go on the record you can see how much money was spent on our platforms in the UK, and even the different advertising that was proposed.

Jo Stevens

When did you introduce that for the UK?

Marco Pancini

I think that was launched three weeks ago. It is for all of Europe, including the UK, the transparency record, for advertising and the authoritative sources. Going back to your question on when it was launched, I can come back to you, but is one month ago, or something like that.

Jo Stevens

One month ago. Okay. Have you done any assessment of whether or not in that month the feature has been successful? Obviously, we are in the regulated election period now. Have you done any assessment?

Marco Pancini

To go back on the process that we are going through with the code of practice, in the code of practice we have to provide every month to the European Commission and to all of the member states a report that is stating all the initiatives that we are taking and all the product that we are launching to fight misinformation and to make sure that the electoral period is going on—there you can see in a very transparent way every month. Yes.

Jo Stevens

Yes, I understand you are reporting that data to the European Electoral Commission.

Marco Pancini

No, no, to everyone. We document it to the Commission but this document is public so everyone can consult this document.

Jo Stevens

However, you are not doing anything internally, as far as you are aware, to assess whether or not the system that you have put in place has worked?

Marco Pancini

Let me go back and see what kind of findings we can already provide to you in relation to our findings.

Jo Stevens

Okay. You mentioned that you had rolled this out in the UK and in other countries across Europe. Did that happen to all the countries at the same time or were they done in a particular order?

Marco Pancini

No, no, the transparency register is for the 27 plus one, 28 countries, and was launched at the same moment, again, a few days before the end of the month in April.

Chair

Just to be clear on political advertising, do you disclose the amount of money being spent on advertising?

Marco Pancini

Correct.

Chair

Do you disclose who is spending it?

Marco Pancini

Yes. Yes, in two dimensions. You can see on the report both how much money and who invested for political advertising. It is also important for you to know that in order to do political advertising at the moment you need to be registered. You need to provide to us evidence that you are doing this as a carrier of interest, or as a political party registered in the member state. Again, all this information is tracked and made public, but also every member who wants to know who is paying for a political advertisement can click the three dots on top of the political advertisement and see who is paying for it.

Chair

What would that say? If I had set up an account to serve these adverts it might tell you which account had done it, but would it tell you who the real person was or who was providing the money? Or is it just a self-declared piece of information from the person who served the adverts?

Marco Pancini

You can see in the report that we provided to the Commission—in order to be registered and to make political advertising on our platform you need to go through our registration process. You need to provide evidence that you have an interest in doing political advertising, including registration of your party or institution that wants to make a political advertisement, and including responsible people for the campaign, and the countries in which you want to make it. There is registration at the beginning of the process, and then the advertising is proposed and, of course, it has to be in line with our policies and transparent to the public and to the institution—

Chair

Is the information that is provided to you when a registration is made the same information available to the public?

Marco Pancini

The same information that is provided to us and then made public to the users, yes.

Chair

It is all of that?

Marco Pancini

Yes.

Chair

On profiling political messages, you have an advertising tool on YouTube that is rather similar or the same to the sort of tools that Facebook have. With Facebook it is called Lookalike Audience, and you have Customer Match. You can supply data to help target adverts and you can have a data set and then match to people who are similar on YouTube. Is that tool available for political advertising as well?

Marco Pancini

We need to look into the tool that the advertisers use in order to provide political advertising. However, let’s just make it very clear, we do not provide those making advertising any information on the political preferences of our users.

Chair

Okay, but if I want to run a political ad on YouTube and I want to use Customer Match to help expand the audience for it, in the literature that you provide online it is quite clear that the idea of Customer Match is to help you to grow your audience. If you are using Customer Match to run political ads you are using data about people to guess whether they are interested in receiving that ad, just as Facebook does.

Marco Pancini

Our method is very different to Facebook. Unless you have an answer on this, I can come back to you with a screenshot.

Rich Waterworth

Yes. I can come back to you with details about whether political advertisers have exactly the same access to Customer Match or if there are any differences. I think we need to come back with details on that.

Chair

Okay. I think this is fairly basic stuff about the way the platform works. We are getting up to Facebook levels of writing back at the moment.

Clive Efford

Are you aware of what the Home Affairs Select Committee did with a fresh laptop with no content on it whatsoever? They went on YouTube and they searched, “News, politics and sport”, and they were recommended a right-wing YouTuber. They clicked on that, and then were recommended a far-right Canadian video. They clicked on that and were recommended a UK channel from someone who uses racist, homophobic and violent content. Within three clicks they got to someone who made remarks about rape in relation to a woman member of this Parliament. That is a fresh piece of kit, no history on it whatsoever, immediately directed to far-right content. What is driving your platform when people are looking at political content?

Marco Pancini

We are looking into this specific example in this moment. We received the letter from the Home Affairs Committee yesterday, so we will come back to them with an analysis of the case that they have brought to our attention. We can go back to the description of how Up Next works, because I think it is addressing this.

Clive Efford

Okay. Just so you get the chance to have the full picture, the Chair of the Home Affairs Committee then put this content into her personal iPad, so these US and Canadian channels that the laptop was directed to. When she put them into her iPad, of the top videos that were recommended to her the first one and the fourth one were both of Stephen Yaxley-Lennon, a far-right extremist. Doesn’t that cause you concern, that your platform is being driven unbeknownst to you? Is there any way that someone is hacking into the system and forcing it in this direction?

Marco Pancini

Again, we are looking into the case that has been brought to our attention by the Home Affairs Committee in these hours, because we received the letter last night. However, on the specific point of Stephen Yaxley-Lennon, we do not promote any video of his channel. His videos are not allowed to be shared, commented on, or promoted by our platforms. In two cases that are in the letter, as far as I can remember—because again, we received it last night—reference is made to one report speaking about this individual, and secondly about a video of a third party, the Canadian media, which is not directly managed by Stephen Yaxley-Lennon.

Brendan O’Hara

We have talked a bit today about almost rogue YouTubers and how you take down content that breaches your rules. YouTube has also been pivotal in the creation of this influencer culture, and there are a handful of people who probably earn more money and have more recognition factor than some Hollywood stars. They are hugely important in driving particularly young people, I would imagine, to your platform. How do you deal with those influencers and those YouTubers who have this mass following, who are creating or propagating disinformation or spreading harmful content?

Rich Waterworth

I can talk a bit about influencers. We call them creators in general. They are a group of people that we work very closely with and we are enormously proud of. I talked earlier about why we are proud that YouTube is an open platform; it enables new and diverse voices to express themselves creatively and to find audiences. The other really important part to keep in mind when thinking about creators is that these are often people whose job is running a YouTube channel. They are creative entrepreneurs. We see, particularly in the UK, enormous growth, and we should be very, very proud of the strength of the creator economy in the UK. These are people from the length and breadth of the UK who are finding ways to express themselves that have not always been easy to do. They are finding audiences around the world. They are often building very, very strong communities around their channels. Those can be communities about learning how to do something, communities of shared interest, or communities of support around their personality or interests. First and foremost, I think we are and should be very, very proud of those creative entrepreneurs, in the spirit of the UK having a proud history across the music industry and television industry. We also take seriously our responsibility to those creators as they build their businesses and manage themselves. We do a lot working closely with them to help educate them as to how to be successful creatively and commercially, but also to help them manage some of the challenges that come with being an individual, creative entrepreneur. Some of them find that very challenging. We work closely with them. We also take very seriously the fact that they are very influential, as you mentioned, often with younger audiences but often with older audiences. We take a strong position in terms of harnessing the positive influence that those creators can have. We have invested over the last few years in a programme called Creators for Change. That is 68 creators and influencers from around the world. There are eight of them from the UK. They are using their channels and their influences to tackle really challenging topics, which could be bullying, could be xenophobia, or could be hate of different kinds. We have created a network to encourage and really lift up that group of creators to use their voices and use their influences to help tackle some of those issues. There is a really important role in the emergence of influencer culture and creator culture. There is a really important role in knowing how to harness that for positive benefit to users and society.

Brendan O’Hara

Do you as a platform feel you have a responsibility for protecting truth?

Rich Waterworth

As I say, we are an open platform, and people express themselves and communicate on the platform. We have community guidelines, and so if a piece of content is contrary to the community guidelines that will be taken down. We also have, as I mentioned, limited state, where something may not be against our guidelines but it may be highly controversial and damaging, in which case that video will not be recommended and all features will be taken away.

Brendan O’Hara

Would this be what is known as borderline? That is your borderline content?

Rich Waterworth

Yes.

Brendan O’Hara

This is all very well and good but 30 million people have watched a show that questions whether 9/11 happened. Thirty million people. There is a channel with 19 million subscribers that has just released a documentary saying that the earth is flat. These are your influencers. These are the big, big players on YouTube. Where is your responsibility to protect people and defend truth?

Marco Pancini

That is a very good example. For this specific reason, now when a user is searching for information on September 11 or information on other important historical facts that are sometimes subject to conspiracy theories, we provide very evident authoritative sources that put what happened in the right perspective; in this case, Encyclopaedia Britannica, or in other cases Wikipedia. It is very important for us to make sure that, of course, users have access to a diversity of views, and sometimes those are incorrect views, but always in the full respect of historical fact. Therefore, we provide a so-called knowledge panel, so some additional information that we proactively provide with context on these facts.

Brendan O’Hara

Surely even by your own evidence you are saying that you teach or you encourage and mentor these people on how to be successful. It would appear that these two channels alone have over 40 million subscribers. The lesson would be, the more outrageous you are, the more controversial you are, the more successful you will become, surely, which is not a good message to send.

Rich Waterworth

Our actual experience in looking at the most successful channels on YouTube is that they do not tend to be extreme. They tend to be more mainstream. They tend to be dominated by music, by comedy, and by vlogging. These are the kind of things that are the most popular content on YouTube. Education is a huge and really growing category. Our experience is that those are the types of content that really succeed on YouTube.

Claire Lilley

You mentioned in particular young people, so if I may, we feel like we have a responsibility to young people, both as creators of the content, but also as viewers of the content. If children are in the content we will do a lot in order to try to protect them. Every time you upload a video to YouTube you will be presented with a question to say, “Is there a child in this?” and then be directed to information that will help you understand what your responsibilities are to any child in that video; complying with local employment law in terms of paying your children, in terms of working hours and all those sorts of things. We have teams of people who reach out to creators, particularly where there are children and where there is a family dynamic, because these are very popular channels, and we do a lot of work to reach out to them. However, it is also true that in terms of the children who are viewing the content, a lot of children—43% of children in the UK—use YouTube to do their homework. That was a stat that came up. We really like to try to promote really good content. There are some creators like Physics Girl and Veratasium who are promoting really good STEM content, a lot of it particularly aimed sometimes at young women, which we really feel proud of as things that young people are engaging with and that are getting really good followings.

Brendan O’Hara

Rightly so. Those are things of which you certainly should be proud. Would I be right in saying that you define borderline content as misinformation and conspiracy theories, as not directly contravening your rules but a bit dodgy? Is that roughly how you would define borderline?

Marco Pancini

If I may I would like to challenge this assumption. We are investing in products, and you are going to hear more from us on this by the end of the year, in order to make sure that when the user comes on our platform they find content that is fulfilling them and is making them feel good, not content that is low quality, like, for example, conspiracy theories or content that is shocking. We do not have any interest, because it does not drive any positive experience, in borderline content.

Brendan O’Hara

The question I wanted to ask though was: how do you police borderline content? You cannot sit and watch every single video that is uploaded, can you, and then make a value judgment on the content of it?

Marco Pancini

Indeed. What we do, and maybe Rich can say more about it, is find ways when the user is searching for something, so in a general search, to provide authoritative sources, or content that is coming from qualitative sources. If a user wants for whatever reason to make a specific search for content that can be controversial, we cannot cross the line too much. We need to provide to the user the content that they are specifically looking for.

Rich Waterworth

To add to that specifically on the policing of borderline content, the model is very much the same as policing of content that goes beyond our community guidelines. That means that we work with experts internally and externally to understand and create a shared and agreed definition of what borderline content should be. We want that to be a collaborative and collective view, not just our view. Then we enforce that view with a combination of humans and machines, using machines to pick up where there might be content that comes into one of these categories, and using human reviewers to confirm or deny and to make the machines more and more effective over time, which is what is happening there.

Brendan O’Hara

Moving on just a bit, Child Wise recently in its under-age social-platform use report said that YouTube was considered the worst offender for advertising, with the majority of children mentioning the video platform and its reputation organically without prompting. Why would Child Wise come to the conclusion that it has?

Claire Lilley

From research that I did when I was at NSPCC, children do not like advertising, and that is fair enough. None of us do, but it is the way in which the platform is funded. That is kind of a reality. One thing I would say is that we do have very restrictive advertising policies with regard to children on our YouTube Kids platform, which is specifically designed for nought to 12s. We have on there a much-restricted range of products that can be advertised, so no political or religious advertising, no beauty advertising, no food and drink advertising. The advertising that is on there must not have any incitement to purchase, for example, so the way in which the ads are presented is very conservative. On that platform children will only be presented with an ad once every 15 minutes. Also, in recognition of the idea that children’s digital literacy is not necessarily as developed as an adult’s might be, what we do is serve them what we call a three-second bumper. It is basically a warning that what they are about to see is an ad, so they can understand when what they are watching is entertainment content and when what they are watching is commercial content. We really try on the YouTube Kids platform to reinforce positive, very conservative, very responsible advertising practices.

Brendan O’Hara

Why, in your opinion, would Child Wise have come to the conclusion that it did?

Claire Lilley

That children do not like advertising?

Brendan O’Hara

No, that YouTube is the worst offender for advertising to young people?

Claire Lilley

I do not think we treat young people aged 13 to 18, or aged 13 to 24, any differently than we treat other users in terms of advertising.

Marco Pancini

Take into consideration that, for example, we want to provide the user with full choice. If a user wants to have an experience of YouTube with no advertising, there is always the possibility to subscribe to YouTube Premium, which is the YouTube platform with no advertising, plus more content.

Claire Lilley

I just want to correct my previous statement; we do treat children differently, because we do not serve interest-based ads to under 16s. It might be their frustration is that what they are seeing is ads that they do not feel are relevant to them because we do not profile them in the same way we do with adults.

Brendan O’Hara

Just finally, what is YouTube’s policy about informing viewers if a video is sponsored or not?

Rich Waterworth

The ASA has very clear and good guidelines and we invest, as part of our resources and support for creators, in making sure that we reinforce and make sure that they are aware of that. That includes at the moment upload. If you are uploading a YouTube video there are reminders that if there is product being promoted it needs to be very transparent and clear. We think that that is a very good and important guideline, and we take steps to make sure that creators are aware, and if they do not adhere to that then they will be dealt with.

Brendan O’Hara

Every video that has a sponsor on YouTube is transparent and it can be seen as such?

Marco Pancini

Correct.

Chair

On the subject of children, I want to ask about some research that was conducted earlier this year by the Cyprus University of Technology, which analysed 130,000 videos targeting toddlers between the age of one and five.

Claire Lilley

What was the age range again?

Chair

It was children between the ages of one and five. It was based on people watching content aimed at children between the ages of one and five, so children’s cartoons, children’s programmes and that sort of thing. They found, based on this sample set of 130,000 videos, there was a 45% chance of a child coming across inappropriate footage within 10 clicks of a child-friendly video. They found that children stand a one in 20 chance of being recommended problematic videos after watching any video. I wonder if you recognise that research and those figures. Certainly, the idea that there is a 45% chance of coming across inappropriate footage within 10 clicks of watching a child-friendly video would seem to be concerning.

Claire Lilley

It certainly would be concerning, yes. I completely agree. Just for me to clarify, is that on YouTube?

Chair

Yes, on YouTube.

Claire Lilley

Okay. What I would say in response to that is this is exactly the reason why we created YouTube Kids, which is a platform designed for nought to 12 year-olds. We recognise that YouTube main is for 13-plus, and there is a lot of content on there that is just not suitable for younger children. There is co-viewing on there, as I said, with parents, but in the main this is why we put in a lot of investment. YouTube Kids has a much-restricted corpus of content and a much higher threshold to be on there. There are lots of systems in place. Parents can choose, for example, to turn search off, they can choose to have only parent-approved mode, where they literally can handpick every video and every channel that their child might want to see. The videos that are served on the home screen have been verified by humans as 100% safe. It is an app designed with built-in features to protect those children at that very, very vulnerable life stage, because we absolutely do not want any child to see content of that nature.

Chair

I understand YouTube Kids, and this research, I believe, was on the YouTube main. However, again, it is just concerning that the way the algorithm is driving viewer behaviour is someone is watching a series of films that are designed to appeal to very young children, and then within a relatively short period of time they end up being directed towards more problematic content. That is a concern.

Rich Waterworth

One of the Up Next algorithm changes that Claire mentioned earlier—and this may be something that has been introduced since the time that you are quoting, I do not know—is that if you start a session and you are watching family-friendly content, even on YouTube, it will only now recommend family-friendly content. We are very conscious and aware, and this is why we have tried to emphasise that there are changes that have been made and are being made to make sure that that algorithm is delivering a safe and a high-quality experience for all of our users.

Chair

When was that change made?

Rich Waterworth

I would have to come back to you on exactly when it was made, but in the last six months, I think. It is quite recent.

Chair

Okay. If you could I would be grateful, because that research is dated for this year. I would be interested to see whether it was done before. We can check with the institution as well. Thank you.

Rebecca Pow

Apologies for being late, but I just have a couple of really quick questions. Just on that note, and listening to everything that is being said, I get the impression that you are just reacting as this whole thing grows like topsy with no concept of the direction it is all going in. It sounds like you have some good measures in place but do you not feel like you are firefighting every day to keep up with this thing that you have created, that it is almost impossible to control what people are seeing?

Rich Waterworth

To step back for a minute, we have just gone past the 50/50 moment, where more than 50% of the world is online. In the UK in many markets it is well beyond that, so very, very high levels of penetration. What happens when everyone is online is you get bad actors as part of that. Digital consumption and penetration is an evolving story and continues to be. Our responsibility is and will continue to be to get ahead of what is happening. What is happening is changing over time, and partly it is changing over time because of our measures through the intelligence desk, through more robust enforcement procedures, and through much, much better algorithms that catch content at scale. As those measures get more effective, bad actors—who, let us face it, are around in any situation where all of humanity is there—have to try new things.

Rebecca Pow

I understand that, but it does sound like a lot of the time you are playing catch up rather than being ahead. Reacting to the Christchurch incident, for example, is catch up rather than being ahead.

Claire Lilley

In relation to kids, I do think we are ahead. I think we are industry-leading in the creation of tools like Family Link and tools like YouTube Kids. We saw that, and that has been a product of a decision five years ago to build those products and to recognise that future need. It is true that we are continually iterating the products, and that is right and fair, because what we see is people changing the way they are using the products all the time as new technology comes online. Particularly for me, the way offenders are manipulating and using systems is changing all the time, and we are always horizon scanning; the intelligence desk is always trying to get ahead of those. One of the key areas for me that is really important in getting ahead of this is about the education of young people and their parents and about how they can use technology in a safe, responsible and enjoyable way, and get the most out of it. That is why we are investing heavily in education programmes, both in primary and in secondary schools.

Rebecca Pow

At the moment I am glad I do not have small children, because it sounds like a minefield of trying to control what your children can and cannot see with all the different methods, switches, names and all of that. To me it sounds complicated for the parent.

Rich Waterworth

I am a parent and it is challenging. This is why, as Claire said, we are investing in education. Our Be Internet Legends programme has been experienced in 50% of UK primary schools and growing, and we are going to get to 100%. By the end of this year that programme will have been experienced by 1.9 million young people in the UK. These things are challenging and we need to continue to invest across a range of measures, education, intelligence, enforcement, that is what we are doing and will continue to do.

Rebecca Pow

One final question, just picking up from Mr O’Hara on the influencers, I see you have your set advertising structures and how you control adverts, but surely the influencers themselves are advertisers, aren’t they? They are really endorsing products, they are valuable to advertisers. How much control do you have over that?

Rich Waterworth

I think this relates to what we were talking about earlier. There is an ASA code that states that if an influencer, if a creator, is promoting a product they have an obligation to be fully transparent about that. That is a code that we fully support and we educate our creators around it, including at the point of upload, and there are penalties if they do not adhere to that.

Marco Pancini

There are also laws in place. There is a recently approved law called AVMS, an audiovisual media services directive, that will be applicable also in the UK. There will be clear rules for promotion. Going to your question, if I may for one second, our CEO described this approach as responsible growth. It is really encompassing all the different product development and it needs a clear vision of making sure that every new product, every new policy that we are implementing, is responding to the need of the platform to grow, but in a responsible way. Indeed, we have a plan. One example I would like to make, and maybe Claire can say more, is Family Link. Because we know how parents are struggling across all the different platforms, we, as Google, wanted to give them one place, it is called Family Link, where they can go, they can log in and make sure that all of the different accounts of their kids are linked there. They can control from one simple page the behaviour of the kids and how the kids are interacting with technology.

Ian C. Lucas

I will come back to our conversation about liability earlier on. Just one short question: in the Online Harms White Paper it says, “The Government proposes the establishment of a new statutory duty of care to make companies take more responsibility for the safety of their users and tackle harm caused by content or activity on their services”. Do you as YouTube accept that as a legal basis for operation or will you be opposing it?

Marco Pancini

As I said, we are absolutely engaged in the discussion, with our chief legal officer meeting with the Government to discuss that and to better define what the duty of care means. Because for us what it means is to act as a responsible platform, to have this approach of responsible growth, and to make sure that when we design a product this product is trying to maximise the experience of the user and keep the user safe.

Ian C. Lucas

I am sorry to interrupt you. I am just going to ask you a straight question here, and I would like a straight answer. Do you agree that you should take responsibility for harm caused by content or activity on your platform?

Marco Pancini

I can appreciate that is a very thin line but the responsibility, yes, the liability, no.

Ian C. Lucas

I do not understand this difference. I am a lawyer. Liability is legal responsibility for something. The only difference I see between the two words is moral responsibility but not having legal liability. What I want you to confirm is that you will have legal liability for the content of your platform.

Marco Pancini

As a lawyer, I am also a lawyer, it has to be well defined.

Ian C. Lucas

That is why I want a straight answer to the question.

Marco Pancini

That is why I cannot give you a straight answer, because we are still in the process of defining what exactly the duty of care means in terms of knowing in advance—

Ian C. Lucas

That is why I have quoted you the Government’s words. I would like you to tell me whether you will be opposing that proposal by the Government or will you be supporting it?

Marco Pancini

We will not oppose the proposal. We will be part of the process in discussing this proposal. However, I have also seen the hearing from your Secretary of State when he came here to discuss that, and indeed there are some areas that still need to be defined, such as the role of the regulators and the code of conduct. It is very important to define exactly what this duty of care will mean for us so we know when we have crossed the line and we become liable. In very practical terms, when we have this conversation with our colleagues from the US, they ask us, “Okay, Marco, in concrete terms what does it mean? In what situation will we, in a sense, break these rules? When will we not have fulfilled our duty of care?” Indeed, we are struggling to provide them with a clear example, because still we lack the code of conduct. We know, of course, the law. If we break the law, or if, as in the example made by Jeremy Wright, a website that you say is illegal should be IP blocked. Giving access to illegal content online would indeed raise a liability or responsibility and would be in breach of the duty of care. I think that is the right direction in designing how this will play out.

Chair

I think that is a good summary of why it is important that Parliament decides, rather than leaving it to companies. Thank you all very much for your evidence this afternoon. I appreciate it, thank you. We will now invite the representatives from Instagram to come and take their places on the witness table. Thank you. Examination of witnesses Karina Newton, Head of Public Policy, Instagram; Vishal Shah, Head of Product, Instagram.

Chair

Good afternoon, and thank you for joining us this afternoon for our evidence session on our inquiry into addictive and immersive technologies. Before we start the questions themselves, I just want to state for the record that obviously the Committee is bound by the House’s rules on sub judice. For the avoidance of doubt for everyone in the room, the sub judice resolution applies to proceedings in the Committee and, therefore, witnesses and members should not make reference to any individual case that is currently before the UK courts or cases that have been listed for a date to appear before the UK courts either. I just wanted to make that clear. Vishal Shah, you are Head of Product at Instagram. You will probably, I am sure, be aware of the tragic news that has been reported today of the death of teenage girl in Malaysia. The incident seems to be that she posted on Instagram asking people whether she should live or die, and 69% of responders voted that she should die. She then took her own life. What can Instagram do to make sure there are no further tragic cases like this?

Vishal Shah

Thank you, Chair. I want to start by saying the news is certainly very shocking and deeply saddening, and our thoughts and prayers go out to the family of the young woman in Malaysia. The tools that we build, in this case the polling sticker in stories, are designed for helping people express themselves and to connect with the friends, family and followers that they have. By and large we believe that is how the platform is mostly used. There are these cases, like the one we have talked about today, where our responsibility for keeping our community safe and supported is tested, and we are constantly looking at our policies. I will pass over to Karina in a second to talk about our policies. We are deeply looking at whether the products on balance are matching the expectations that we created them with. In cases like the polling sticker, we are finding more evidence of cases where it is not matching the expectations or the creative guidelines under which we created them. We are looking at whether we need to make some of those policy changes like we have done with Live more recently. Maybe Karina can talk a little bit more about how we think of suicide specifically from a policy perspective, and some of the product changes we have made as a result.

Karina Newton

Yes, it was absolutely heart breaking, and I have not talked to anyone at Instagram that did not feel the same. The way we have looked at our policies around suicide and self-harm has been formed by expert discussion. We work closely with academics, charities, and people who have lived experience to inform where we draw the line and what we allow and do not allow on our platform. We work with the Samaritans here in the UK. We do not allow the promotion or endorsement of suicide or self-injury content. That polling sticker as a story would violate our guidelines. It promotes the act of suicide. We have allowed—and we think that there is an important role to play especially for Instagram to be a place that can help destigmatise mental health—that you can come on to Instagram under our policies and admit that you are struggling and that you need help. This offers people who are in your life, your friends and family, the opportunity to say, “Can I help you?” We also provide information; so if you post content and it is removed for suicide or self-injury content, we will provide the person who posted it with resources on how to get help and connect them with local organisations, such as the Samaritans. We will also, when we review that content, if we are worried about immediate harm work with local law enforcement to get someone there. We have heard anecdotally that we have saved lives through this programme. We are looking very much holistically at our policies to ensure that they are in the right place. The case in this tragedy is no different. We will be looking at it and making sure that we are doing everything we can to ensure the safety of the people who come to Instagram.

Vishal Shah

If I could add one thing to answer your question specifically, one of the things that we see in cases like this where there is some amount of media attention is that then results in some mimicry on the platform, where other people try to copy this behaviour. We are always looking at trends and new ways in which people are expressing these kinds of mental-health issues or suicidal tendencies, and it is a constantly evolving space. In this case we are paying specific attention to the usage of the polling sticker and cases where there might be other troubled people who are using the same methodology.

Chair

I have seen—of course this was yesterday—the poll was live for about five hours or so. What more can your systems do to identify people using the polling sticker for something that could lead to harm, as this did, sooner? I assume that no one reported it to Instagram while it was live. If they did report it and no action was taken, I think that would be a very serious matter, I am sure you would agree. I would be grateful if, if possible, after the event you could write to confirm that was the case; there is no record of this poll being reported to Instagram. What can the company do to identify people who are using these tools in ways that could cause harm?

Vishal Shah

For this case specifically it is an active area of investigation. Our team on the ground in Malaysia is engaged with the local authorities to make sure we can understand more information about it. Generally, this is a bit of what I was referring to earlier, where sometimes the ways in which these tools are used and how people are expressing these thoughts can evolve. Without talking about the specifics in this case, because it is still active, there is speech that maybe we would recognise as potentially coming from people who are suffering from mental illness and expressing these self-harm type thoughts. It is my understanding that sometimes it is the use of emojis, or the use of things that are not part of standard parlance, around why and how people are expressing these things, especially with young people. Sometimes it is about making sure that when cases like this occur, and we learn of a new way in which people are expressing these thoughts, we can add those to the proactive detection of these things and put up warnings offering the producer of that content, the person expressing those thoughts, a way to get help. Also, for the people who have come across this content, especially again with young people, to make sure they have the opportunity to get help if they come across content that they are discovering on Instagram.

Karina Newton

I would like to add just one thing. We as an industry are really looking at how we can be very proactive in this space. You may be aware that Secretary of State Matt Hancock recently brought together several members of tech companies, including us, and we are going to be working with the Samaritans to invest in research in this area and use this research to inform how we think about things, from policies, to enforcement, to support resources. I do think it is a really positive development to work with the Government and work with charities such as the Samaritans to better understand how we can invest support and prevent instances like this.

Chair

Given that Instagram holds a huge amount of data on its users, is there not more that you could do to identify people who are at risk, maybe because of the content they are engaging with and the profile that is being created by the data based on their engagement on the platform? Can more be done to identify these people and to monitor their activity, or even to reach out to them?

Vishal Shah

This is a constantly evolving space. From a technology perspective, we are trying to get better at predicting and practically removing or identifying this content. Sometimes it is about past behaviour; sometimes it is about the act that they are currently in the process of sharing or expressing. At the same time, we want to be conscious of ensuring that the content either violates our policies, which Karina talked about, or is a case where we want to be offering help more proactively. There is a spectrum there. In general, I agree with you that our responsibility to our community of keeping people safe and supportive means we should be continuing to invest, as we have been, to help keep people safe.

Chair

Instagram uses the same advertising tools as Facebook. That is correct, is it not?

Vishal Shah

We use the same underlying advertising platform as Facebook. That is correct.

Chair

Is it not possible to use some of the tools that are used to profile people to sell them stuff to identify people that may be suffering and, therefore, may be at risk?

Vishal Shah

To be clear on the advertising side, there are many categories of targeting that are not possible through our advertising tools. Things like mental health and depression are in that category of things that are not targetable. Maybe not to jump into jargon, but it is important to think about how we can use all of the signals that we have on the platform, all the ways that people interact with content, and the accounts that they follow, to understand if there is additional risk, and from a product perspective, see if we can get ahead of some of these issues. I do think it is important to make sure that we have the right balance of very clearly defining when something violates our policies and making sure we act on that explicitly, versus cases where we can do a better job of detecting something in advance and working with law enforcement if we believe there is imminent harm for a young person using our platform.

Chair

Do the same sort of Lookalike Audience tools exist on Instagram that exist on Facebook?

Vishal Shah

We support that, because we sit on top of the same advertising platform, yes.

Chair

Yes. So, would it not be possible, where there are cases of people that are known to have been engaged in harmful content and may be at risk, that analysis can be done to see what other users share similar characteristics based on an interest in the same sorts of subjects?

Karina Newton

These are some of the things that we are trying to evaluate and understand. Obviously, we have some privacy concerns of users, and also some GDPR obligations as well. But one of the things that we are trying to see, if you search right now in Instagram we have a couple of different surfaces; there is a home tab, which is where most people spend most of their time. That is where you see what you have followed. Then we have a search and explore tab, which is where you find content. If you search for a term such as “suicide”, we have a pretty good indication that you are interested in that subject, and so we use this as an opportunity to help. As soon as you search for that term you will receive a pop-up message that says, “Can we help? Based on what you are searching for, this is some information”. That is localised in 40 different countries with resources there. We are using the signals that we have from what people are searching for, and we are looking for additional opportunities to do that, perhaps based upon accounts you follow, or perhaps based upon other search terms. We really do want to use the time that people are on Instagram and our platform as a way to provide resources in as many ways as possible.

Chair

Yes. I have seen those prompts that you describe. You mentioned that there are GDPR privacy issues on using data to try to identify people that might be at risk. I suppose the concern we would have is that those privacy and GDPR issues do not exist in terms of using these advertising tools, but inferred data is used all the time in advertising tools to sell people stuff. Instead, could that data be used to identify people who could be at risk of self-harm?

Vishal Shah

Just to be clear, you cannot target people based on their potential likelihood to have mental illness or to have depression. From a technology perspective, there are perhaps more granular signals that can be used, as opposed to ones that are about specific profiles or interests that you might have, can potentially help identify these things in a compliant way from a data perspective. Right now we have that in the search capability where it is a very clear intent. I think the steps are in the right direction, which is offering people help when we think they need it. Now the question is: where else can we highlight that and show it across the product?

Chair

Let’s say I was the Samaritans and I wanted to run a campaign on Instagram, and I said, “I have a load of data on people that are already in our database that have given us their data, can you expand that audience through lookalike audiences to find more people who are like that?” Is that something you would offer?

Vishal Shah

Lookalike audiences are available. You have to give proof that you have permission to use the custom audience in the first place. The way the algorithm would find potentially other people that are interested in the same topic would not necessarily be based on the specific interests that they might have. It might be other accounts that they might follow. Let’s say this happens to be young people; it might index on young people and things that they care about.

Chair

I understand that, but if you are a charity organisation that works with vulnerable people and says, “We have a dataset of vulnerable people”, the purpose of using a tool like lookalike audiences would be to find other people that share similar characteristics. The tools that are available would do that. You have confirmed that would be possible to do. I just wonder whether that is something the platform could look at doing, working with charities, to try to identify proactively people who may be at risk and maybe someone should reach out to them.

Vishal Shah

I do think that the engagement we are in with the Samaritans and other charitable organisations helps us in a couple of ways. One is it allows us to leverage expertise on these topics that we may not have inside the company, and the second is it allows us to engage in research. I know we have been working with some of these organisations on grants to do some of this research to then incorporate best practice into the product. In general, I think this is an important area for us to be investing in, given that young people do use our platform. It is Mental Health Awareness Week here in the UK, and it is something we certainly deeply care about in terms of destigmatising and then being able to play our role in helping combat it.

Chair

Could I just ask a final question on this? Will Instagram be taking action against the accounts that voted in this poll, particularly those that encouraged her to take her life?

Karina Newton

We are currently investigating that right now.

Chair

Would you potentially suspend those accounts or cancel those accounts?

Karina Newton

Literally, as I am sitting here, a team is investigating. We are happy to circle back after that investigation is over.

Chair

You say you are investigating, but it is a matter of fact there was a poll. People voted in it. You have records as to who voted. Is this a point of principle? Will the company be taking action against those accounts?

Karina Newton

I hope you can understand. It is just so soon. Our team is looking into it to see what the content violations are. Again, that poll in and of itself goes against our policies for promotion of suicide.

Ian C. Lucas

Can I just ask some questions about the governance setup within Instagram and the relationship with Facebook? Do you have a separate board that runs Instagram? Are you a separate business to Facebook? How does that work?

Vishal Shah

We are a product group inside Facebook, to use our internal terminology, which means that product decisions that we make around Instagram specifically, how content is displayed, how our algorithms work, how people express themselves and the tools we create are all governed by Instagram. We were part of the same company, and therefore the Facebook Board of Directors is also the Instagram Board of Directors. From a product perspective, we have considerable autonomy in terms of the decisions that we make.

Ian C. Lucas

The buck stops with Mark Zuckerberg with Instagram?

Vishal Shah

Instagram is run by Adam Mosseri. He is the head of Instagram and the majority of decisions will land with Adam. We are a company and Mark Zuckerberg is CEO, and there are some decisions that go to the CEO.

Ian C. Lucas

In terms of the data arrangements between Instagram and Facebook, are Instagram data shared with Facebook?

Vishal Shah

Because Instagram and Facebook are part of the same company, it is clear as part of the terms when you sign up for this service that the data policies are shared between the two platforms.

Ian C. Lucas

Does that include WhatsApp as well?

Vishal Shah

WhatsApp, to the best of my understanding—and Karina, please correct me, I have more familiarity with Instagram—is on a separate data policy, but Instagram and Facebook are together.

Ian C. Lucas

Are the data shared with WhatsApp?

Karina Newton

We have separate data policies that govern them. Facebook and Instagram share a data policy and WhatsApp has separate terms and data policy. The data are not shared between the two companies.

Ian C. Lucas

They are shared between Instagram and Facebook but they are not shared with WhatsApp. Is that right?