Now playing, everywhereThe tricky task of policing YouTube
How to clean up the world’s biggest video-sharing site
SUSAN WOJCICKI, the CEO of YouTube, received the first message about the massacre in New Zealand at around 8pm. Assaults on two mosques in Christchurch had begun minutes earlier. The shooter had live-streamed the killings on Facebook and the footage from the social-media site was being shared on YouTube as the killer had clearly hoped. Ms Wojcicki checked in with her team. Executives and software engineers were looking for different versions of the video so that machine-learning programs could be trained to hunt for them. Thousands of human reviewers were scouring through videos that had been automatically tagged, sorting news reports and the like from the offending footage. The world’s largest video platform—owned by Google, the world’s most powerful search engine—was mobilised to cleanse itself of the horrific clip.
It failed. Before she went to bed at 1am Ms Wojcicki was still able to find the video (she chose not to watch it). In the morning copies of the video continued to be accessible using generic keyword searches. New versions were being uploaded more quickly than they could be identified and taken down. Finally, at 6am, Ms Wojcicki decided to remove all videos flagged as suspect, without waiting for a human review—a first for YouTube. Hours later, the site also blocked users from filtering searches by new uploads, another first. “We don’t want to be the place where people are finding that,” says Ms Wojcicki.
The company is not alone in its interest in policing what people upload to it. An increasing number of governments and regulators around the world think social-media firms must change their ways. Facebook takes most of the flak but YouTube’s problems are particularly tricky because videos are difficult to monitor at such scale—500 hours of new ones are uploaded every minute. Children and teens consume it in their masses. And its videos are increasingly viewed as an important source of news and information as well as entertainment (see chart 1).
Ms Wojcicki is confident that she can sort out policing YouTube. “I actually think I can solve it or at least I think I can provide a blueprint about how to address these issues that no one else has figured out.” Thus far such confidence is difficult to credit. But how YouTube chooses to moderate its content, and how governments compel it to do so, will affect not only the world’s most popular video service. It will also help shape the acceptable contours of free speech online, and the lives of the people who produce, consume or are otherwise affected by digital content.
From its inception in 2005, YouTube has delivered a new kind of entertainment to people almost everywhere: the rest of humanity (and their pets). User-generated videos, uploadable and viewable by all, made it possible for anyone to find an audience online. Since then YouTube has become the free television service for much of the world (like Facebook and Twitter, it is blocked in China). More than 2bn people now visit the site at least once a month. It accounts for 11% of the world’s bandwidth on the internet, second only to Netflix, with its much higher-resolution videos, according to Sandvine, a research firm (see chart 2). The volume of entertainment, education, information and dross on offer is hard to fathom. It would take 100,000 years to watch it all at a single sitting.
Every day tens of millions of fans, many of them children, tune in to watch their favourite stars, who have built huge followings on YouTube. They dispense silliness, confessional tales and practical tips. Gamers in Europe narrate virtual conquests, women in India and Saudi Arabia give make-up tips in Hindi and Arabic, teens in America share their anxieties, an elderly quilter in Missouri teaches her craft.
For YouTube and its most successful stars that has proved lucrative. The firm does not disclose its revenues, but MIDiA Research, a consultancy, estimates them at nearly $17bn in 2018, close to half of which went to content creators on the platform. The top channels for children rake in millions of dollars a year through advertising. The most famous YouTubers—personalities who have built huge followings of young fans—earn millions a year as well.
YouTube’s stars hold sway over their fans. In a survey by the firm, 40% of young subscribers said that YouTubers understood them better than their friends or family, and 60% said YouTubers had changed their lives or worldview.
But there is a dark side to hosting over a billion hours of user-generated content online, algorithmically sorted and recommended to billions of viewers. A series of scandals suggest that YouTube is having difficulty coping with the volume and diversity of the content it is hosting, recommending and monetising. In 2017 adverts were found running alongside violent videos made by Islamic State. That prompted big brands to remove advertising for a while. The same year young boys and girls were found in videos that appeared abusive or salacious and which were recommended millions of times before offending channels were shut down.
Some of the site’s most famous stars have breached the boundaries of decency. Logan Paul, famous for his lowbrow antics, posted a video at the end of 2017 of a dead body he found in a “suicide forest” in Japan. In early 2017 PewDiePie, who had 53m subscribers (then the most of any channel), was reported to have made anti-Semitic references in his videos. PewDiePie, whose real name is Felix Kjellberg, apologised; YouTube dropped him from a lucrative advertiser programme, but he was allowed to remain on the site. Later in 2017 he used a racial slur about black people and apologised again. On April 28th he posted a video asking fans to stop spreading a “Subscribe to PewDiePie” meme, which was referenced by the shooter in Christchurch. He has now amassed 95m subscribers.
Politicians at first paid only passing attention to much of this. After the presidential election in America in 2016 public ire was mostly directed at Facebook over fake news and breaches of privacy, as well as enabling hate groups. That allowed YouTube’s missteps to go by without serious repercussions for the firm. “Thank God for Facebook” became a popular expression in the company’s hallways.
But since last year YouTube itself has come under fire for providing an outlet for hateful figures from the alt-right and for promoting all sorts of conspiracy nuts with its recommendations, including flat Earthers and anti-vaxxers. In February paedophiles were found swapping notes in the comments section of children’s videos, pointing out parts they liked. YouTube has now disabled comments on most videos that feature children. On May 1st YouTube (and Facebook) were also accused of allowing scenes of atrocities committed in Libya’s civil war to circulate unchecked.
As a result, criticism of YouTube has intensified. Like Facebook and Twitter, it is accused of merely reacting when specific problems are exposed by the media or activists, but not before its algorithm has served up offending content millions of times. These scandals, say detractors like Guillaume Chaslot, a former Googler who worked on YouTube’s algorithm, are the bitter fruits of the site’s “manipulative design”. An algorithm and user interface engineered to maximise “watch time” keeps users on the site in part by serving them progressively more extreme videos on whatever subject they happen upon—a “rabbit hole” that can lead those curious about a global tragedy into conspiracy theories or rants by white nationalists. A senior executive said in 2017 that recommendations drive 70% of the site’s viewing.
The site’s engagement-driven model in turn rewards those who provide more outrageous content. Users lap it up with gusto, training the algorithms to serve more of it, and so on. In April a story on Bloomberg, a news service, alleged that some executives discouraged taking into account such risks in the pursuit of a billion hours of user time a day—a goal set in 2012 which Ms Wojcicki embraced, after she became CEO in 2014, as a “north star” for the company and which it achieved in 2016. The site’s engineers have tweaked the algorithm, based in part on user surveys, to account for “satisfaction” in watch time. But the goal remains the same—to keep people on the site as long as possible and maximise profits.
YouTube’s immense popularity makes the question of how best to moderate social-media platforms more urgent, and also more vexing. That is partly because of the view taken in Silicon Valley, inspired by America’s right to free speech guaranteed by the First Amendment, that platforms should be open to all users to express themselves freely and that acting as a censor is invidious. With that as a starting point platforms have nevertheless regulated themselves, recognising that they would otherwise face repercussions for not acting responsibly. They began by setting guidelines for what could not be posted or shared—targeted hate speech, pornography and the like—and punished violators by cutting off ads, not recommending them and, as a last resort, banning them.
As governments and regulators around the world have started to question the platforms’ power and reach, and advertisers have pulled back, the firms have gradually tightened their guidelines. But by doing so they have plunged deeper into thorny debates about censorship. Last year YouTube banned certain kinds of gun-demonstration videos. In January the platform said it would no longer recommend videos that misinform users in harmful ways, like certain conspiracy theories and quack medical cures. It also banned videos of dangerous pranks, some of which have caused children to hurt themselves. On April 29th Sundar Pichai, boss of Google, declared, in an earnings announcement that disappointed investors, that “YouTube’s top priority is responsibility”. He said there would be more changes in the coming weeks.
Governments meanwhile are taking direct action to curb content that they deem inappropriate. On April 21st, after bombings in Sri Lanka killed 250 people, its government took the draconian step of temporarily banning social-media sites, including YouTube, to stop what it called “false news reports”. After the Christchurch massacre, Australia passed a hastily written law requiring platforms to take down “abhorrent violence material” and to do so “expeditiously”. Even in America, where social media has been largely unregulated, members of Congress are drafting measures that would give significant powers of oversight to the Federal Trade Commission and restrict how online platforms supply content to children, an area where YouTube is especially vulnerable.
Ms Wojcicki says she needs no persuading to take further action against unsavoury material. Yet YouTube does not plan to rethink the fundamental tenets that it should be open to free expression, that people around the world should have the right to upload and view content instantly (and live), and that recommendation algorithms are an appropriate way to identify and serve up content. What is needed, she says, is a thoughtful tightening of restrictions, guided by consultation with experts, that can be enforced consistently across YouTube’s vast array of content, backed by the power of artificial intelligence.
YouTube’s record thus far does not inspire much confidence. Children’s programming, one of the most popular sorts of content, is a case in point. Parents routinely use their iPads or smartphones as baby-sitters, putting them in front of children and letting YouTube’s autoplay function recommend and play videos (see chart 3). Children are served up nursery rhymes and Disney, but sometimes also inappropriate content and infomercials.
YouTube executives say that if parents let their children watch videos unsupervised, it should be on YouTube Kids, a separate platform created in 2015. But in reality most children watch the main site and are exposed to the same “manipulative design” as their 40-year-old uncles. Some children’s advocates are furious because they consider this an easy fix. They argue that the site’s algorithm knows when children are using it and could be programmed to switch off autoplay and tightly curate content. When pressed on the subject, executives insist that the site is not meant for children under 13 years old without adult supervision.
YouTube has acted more decisively in other circumstances. Its crack down on terrorist-recruitment and -propaganda videos in early 2017 used machine learning and newly hired specialists. There was an obvious incentive to do it. In what became known as “Adpocalypse”, big firms fled after learning that some of their ads were running with these videos, essentially monetising terrorist groups. There have been a couple of sequels to Adpocalypse, both related to children’s content, and both first uncovered by outsiders. This adds to the impression that YouTube lacks a sense of urgency in identifying its problems, and responds most rapidly when advertisers are aggrieved.
Ms Wojcicki disputes this, saying she began to recognise the increasing risks of abuse of the platform in 2016, as it became clear more people were using YouTube for news, information and commentary on current events. She says that was when she started to focus on “responsibility”. In 2017, as a result of Adpocalypse, she began expanding the firm’s staff and contractors focused on content issues; they now number more than 10,000, most of them content reviewers. Chris Libertelli, the global head of content policy, says that Ms Wojcicki and Neal Mohan, the chief product officer, have told him there are no “sacred cows” in deciding what content should be limited, demonetised or banned. Ms Wojcicki says that with wiser and tighter content policies, and the company’s technology and resources, she and YouTube can solve the problems with toxic content.
This rhetoric will sound familiar to anyone who has heard Mark Zuckerberg, who built a reputation for cutting corners in the pursuit of global dominance, when he talks about the challenges confronting Facebook (see article). His apologies for Facebook’s breaches of trust, and his promises to do better, have rung hollow. Jack Dorsey, boss of Twitter, has also been excoriated for doing too little to control abusive trolls and hate speech.
Ms Wojcicki, in contrast, strikes even some critics as the “good CEO” of the social-media giants, the one with a soul. She sounds utterly convincing when she talks about trying to make YouTube a force for good and seems more sincere than Mr Zuckerberg when it comes to minimising the harm her company causes. But even Mr Zuckerberg has conceded that Facebook needs more government regulation.
Everything in moderation
While the need for regulation might be clear, the details of what should be regulated, and how, are messy and controversial. Few free-speech advocates, even in Silicon Valley, are zealous enough to want to permit beheading videos from Islamic State or the live-streaming of massacres. Yet most of the questions about content moderation that YouTube wrestles with are much less clear-cut. YouTube appears to be weighing whether to ban white nationalists, for example. If it does so, should the site also ban commentators who routinely engage in more subtle conspiracy theories meant to incite hatred? Should it ban popular personalities who invite banned figures to “debate” with them as guests? Ms Wojcicki is conscious of the slippery slope platforms are on, and fears being criticised for censorship and bias.
Another important question will be how to go about enforcing restrictions. When you serve a billion hours of video a day the number of hard calls and “edge cases”, those that are hard to categorise, is enormous. The tech firms hope that AI will be up to the job. History is not reassuring. AI has been trained for straightforward tasks like spotting copyright violations. But even with low error rates the volume of mistakes at scale remains immense. An AI capable of reliably deciding what counts as harassment, let alone “fake news”, is a pipe dream. The big platforms already employ thousands of human moderators. They will have to hire thousands more.
Given the complexities, wise governments will proceed deliberately. They should seek data from platforms to help researchers identify potential harms to users. Regulations should acknowledge that perfection is impossible and that mistakes are inevitable. Firms must invest more in identifying harmful content when it is uploaded so that it can be kept off the platform and—when that fails—hunt for it and remove it as quickly as possible. With the great power wielded by YouTube and other social-media platforms comes a duty to ensure it is used responsibly.