UK regulator orders social media firms to adopt measures to stop viral illegal content

Social media companies have been ordered to have emergency measures in place to stop illegal content going viral, as regulators battle to stop the type of misinformation spiral that circulated after the 2024 summer riots.

Sites such as X, formerly Twitter, and TikTok will have to have a “crisis protocol” in place to intervene when the sharing of dangerous content begins to spike.

Under the measures to be implemented by Ofcom, the UK’s tech regulator, online platforms will also need to reserve a dedicated line of communication channel through which the police can contact them in the midst of a crisis.

It follows concerns at the top of government over the speed with which misinformation spreads at pivotal moments. Ofcom’s announcement also follows the outbreak of rioting in Southampton over the police response to the fatal stabbing of Henry Nowak.

The idea of forcing social media platforms to have emergency measures ready to stop illegal viral content was suggested by the Commons science, innovation and technology committee.

It made the recommendation following an investigation it launched after the riots of summer 2024, when misinformation spread quickly after the murder of three girls at a dance class in Southport.

Its report found that “misleading and hateful messaging proliferated rapidly online, amplified by the recommendation algorithms of social media companies.”

Ofcom said it would define a crisis as an “extraordinary situation in which there is a serious threat to public safety in the United Kingdom”, which is highly likely to have “resulted from a significant increase in relevant content”.

“During a crisis, certain kinds of illegal content and/or content harmful to children can spread rapidly online,” it said. “In some cases, this can create a significant risk to the public safety within the United Kingdom.

“For example, evidence from previous crisis events illustrate how the perpetrators use online services to carry out illegal activity, such as inciting racial or religious hatred, making threats or inciting violence. This can lead to an increase in the amount of illegal content circulating online but also manifest in violence in the real world.

“Such crises are exceptional, and this means that online service providers’ usual content moderation systems and processes may not be sufficient in such circumstances.”

The bar for the implementation of such protocols is high, dealing with outright illegal content. The new measures will come into force after being approved by parliament.