In a shocking turn of events, AI systems might not be as safe as their creators make them out to be. The UK’s AI Safety Institute (AISI) has dropped a bombshell report showing that four leading Language Learning Models (LLMs) are “highly vulnerable to basic jailbreaks.” Even more unsettling, some unjailbroken models produced “harmful outputs” without any provocation from researchers.
What’s Going On?
Most publicly available LLMs are designed with safeguards to prevent them from generating harmful or illegal responses. Jailbreaking, in this context, refers to tricking these models into ignoring their safety protocols. The AISI used prompts from a recent standardized evaluation framework and some developed in-house to test the models. Shockingly, all four LLMs responded to harmful questions without any jailbreak attempts. Once AISI tried “relatively simple attacks,” all models responded to 98 to 100 percent of harmful prompts.
AISI’s Role and Findings
UK Prime Minister Rishi Sunak announced the opening of AISI at the end of October 2023, and it officially launched on November 2. The institute’s mission is to rigorously test new AI technologies, identifying potential risks before and after their release. This includes exploring social harms like bias and misinformation, and the more extreme risk of AI getting out of human control.
The AISI’s report indicates that current safety measures in LLMs are woefully inadequate. The Institute plans further tests on other AI models and is developing more robust evaluations and metrics to address these concerns.
The Scarlett Johansson Voice Controversy
Switching gears to another AI controversy, OpenAI’s recent release of GPT-4o’s voice mode has stirred quite the buzz. Observers were quick to note that one of ChatGPT’s voices, Sky, bears an uncanny resemblance to Scarlett Johansson’s character in the movie Her. OpenAI insists that Sky’s voice is not an imitation but rather belongs to a different professional actress using her own natural voice. However, due to the controversy, OpenAI is “working to pause the use of Sky” while addressing these concerns.
“We believe that AI voices should not deliberately mimic a celebrity’s distinctive voice—Sky’s voice is not an imitation of Scarlett Johansson but belongs to a different professional actress using her own natural speaking voice,” OpenAI explained in a blog post.
Conclusion
The AISI’s findings and the Sky voice controversy underscore a crucial point: the development of AI technology must be approached with a keen eye on safety and ethics. The potential risks of AI, from social harms to extreme scenarios, demand rigorous testing and transparency.
What is jailbreaking in the context of AI? Jailbreaking refers to manipulating AI models to bypass their built-in safety protocols, allowing them to generate harmful or inappropriate content.
Why did OpenAI pause the use of Sky’s voice? OpenAI paused Sky’s voice due to concerns that it closely mimics Scarlett Johansson’s voice, though they assert it is not an imitation.
What are the potential risks of AI mentioned by the AISI? The AISI highlights risks such as social harms like bias and misinformation and extreme scenarios where AI could potentially become uncontrollable.