- June 23 , 2026* – A BBC investigation has revealed that OpenAI’s ChatGPT can be manipulated to produce graphic violent and sexualised images through modified prompts, raising fresh concerns about the safety measures in advanced AI systems.
The findings came from research by British AI security firm Mindgard. The team discovered that a prompt originally designed to create harmless, humorous content could be altered to make ChatGPT’s GPT-5.4 model generate disturbing imagery without any explicit instructions from the user.
According to the BBC, the chatbot produced violent and sexualised content even though the prompt contained no direct references to such material.
OpenAI responds
Following the BBC’s inquiry, OpenAI said it had added extra safeguards to block abuse of the specific prompt. However, Mindgard reported that slight changes to the prompt could still bypass the new restrictions.
Peter Garraghan, founder of Mindgard and Professor of Computing at Lancaster University, called the discovery alarming.
“This is a perfectly innocent-looking instruction to an AI, but the consequence is it generates very, very bad imagery and content,” he told the BBC.
He added that some outputs were “very gruesome, sometimes sexualised, sometimes both together.”
Mindgard researcher Jim Nightingale, who uncovered the vulnerability, said he was personally disturbed by the material. In his report he noted the images reflected patterns learned from real-world data used to train AI models.
“I’m struck that while what I saw was generated, an artificial image, it has ties to real images, and the real world,” he wrote.
Earlier deepfake vulnerability
Mindgard’s earlier research also found ways to manipulate ChatGPT into creating nude deepfakes of real people by inserting their faces into AI-generated images. OpenAI said it had fixed that issue, but researchers later found another method that produced similar results.
Mindgard said it first alerted OpenAI in May. The initial response was automated and an early fix failed. More substantial action was taken only after the BBC contacted the company directly.
Garraghan said further testing could likely uncover more harmful outputs, but researchers stopped due to the disturbing nature of the content already found.
AI safety under scrutiny
Responding to the report, OpenAI said it uses multiple layers of image safety protections, combining automated detection with human review to block policy-violating material. The company reiterated that its policies prohibit sexual violence, non-consensual intimate imagery and attempts to bypass safety measures.
The report comes amid growing global concern over AI safety. In Nigeria, the National Information Technology Development Agency (NITDA) previously warned about ChatGPT vulnerabilities and potential data leakage risks.
Experts say the findings highlight the ongoing challenge of balancing AI innovation with strong safeguards against misuse as these tools become more integrated into business, education and public services.
