🔗 Share this article

Technology companies and child safety organizations will receive permission to evaluate whether artificial intelligence systems can produce child abuse images under recently introduced UK legislation.

Significant Rise in AI-Generated Illegal Content

The declaration coincided with revelations from a safety watchdog showing that cases of AI-generated child sexual abuse material have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025.

Updated Legal Framework

Under the amendments, the authorities will allow approved AI companies and child protection groups to inspect AI models – the foundational technology for chatbots and visual AI tools – and ensure they have sufficient protective measures to stop them from creating depictions of child sexual abuse.

"Fundamentally about preventing exploitation before it occurs," declared the minister for AI and online safety, adding: "Experts, under rigorous conditions, can now detect the danger in AI systems promptly."

Addressing Regulatory Challenges

The changes have been implemented because it is against the law to produce and possess CSAM, meaning that AI developers and other parties cannot create such images as part of a testing process. Previously, officials had to wait until AI-generated CSAM was uploaded online before addressing it.

This law is designed to preventing that problem by enabling to halt the production of those materials at their origin.

Legislative Structure

The amendments are being added by the government as modifications to the crime and policing bill, which is also establishing a ban on possessing, creating or distributing AI models developed to generate exploitative content.

Practical Impact

This recently, the minister toured the London headquarters of Childline and heard a simulated call to advisors featuring a account of AI-based exploitation. The interaction portrayed a teenager requesting help after being blackmailed using a explicit deepfake of themselves, created using AI.

"When I hear about children facing blackmail online, it is a source of intense frustration in me and justified concern amongst parents," he stated.

Alarming Statistics

A leading internet monitoring organization stated that instances of AI-generated exploitation material – such as webpages that may include numerous images – had significantly increased so far this year.

Cases of category A content – the most serious form of exploitation – increased from 2,621 images or videos to 3,086.

• Female children were overwhelmingly victimized, accounting for 94% of prohibited AI images in 2025
• Depictions of infants to two-year-olds rose from five in 2024 to 92 in 2025

Industry Reaction

The legislative amendment could "represent a crucial step to guarantee AI tools are safe before they are released," commented the chief executive of the internet monitoring organization.

"Artificial intelligence systems have enabled so survivors can be victimised repeatedly with just a simple actions, giving criminals the capability to make possibly endless quantities of advanced, lifelike child sexual abuse material," she continued. "Material which additionally exploits survivors' trauma, and renders children, especially girls, more vulnerable on and off line."

Counseling Interaction Data

Childline also published information of counselling sessions where AI has been referenced. AI-related risks discussed in the conversations comprise:

• Employing AI to evaluate body size, body and appearance
• Chatbots dissuading children from talking to trusted adults about harm
• Being bullied online with AI-generated content
• Digital extortion using AI-faked pictures

During April and September this year, the helpline conducted 367 support interactions where AI, chatbots and related topics were mentioned, significantly more as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 sessions were connected with psychological wellbeing and wellbeing, encompassing using chatbots for support and AI therapy applications.