SecurityWeek’s Post

ChatGPT Jailbreak: Researchers Bypass AI Safeguards Using Hexadecimal Encoding and Emojis Malicious instructions encoded in hexadecimal format could have been used to bypass ChatGPT safeguards designed to prevent misuse. The new jailbreak was disclosed on Monday by Marco Figueroa, gen-AI bug bounty programs manager at Mozilla, through the 0Din bug bounty program. Launched by Mozilla in June 2024, 0Din, which stands for 0Day Investigative Network, is a bug bounty program focusing on large language models (LLMs) and other deep learning technologies. 0Din covers prompt injection, denial of service, training data poisoning, and other types of security issues, offering researchers up to $15,000 for critical findings. It’s unclear how much a jailbreak such as Figueroa’s would be worth. https://lnkd.in/ePEkrYXC

ChatGPT Jailbreak: Researchers Bypass AI Safeguards Using Hexadecimal Encoding and Emojis - If a user instructs the chatbot to write an exploit for a specified CVE, they are informed that the request violates usage policies. However, if the request was encoded in hexadecimal format, the guardrails were bypassed and ChatGPT not only wrote the exploit, but also attempted to execute it “against itself”. https://lnkd.in/g9cxyxAM

Last Wednesday, a self-avowed white hat operator and AI red teamer announced a jailbroken version of ChatGPT called "GODMODE GPT." The hacker who goes by the name Pliny the Prompter took to X-formerly-Twitter to announce the creation of the jailbroken chatbot, proudly declaring that GPT-4o, OpenAI's latest large language model, is now free from its guardrail shackles. "GPT-4o UNCHAINED! This very special custom GPT has a built-in jailbreak prompt that circumvents most guardrails, providing an out-of-the-box liberated ChatGPT so everyone can experience AI the way it was always meant to be: free," reads the Pliny's triumphant post. "Please use responsibly, and enjoy!" Pliny shared screenshots of some eyebrow-raising prompts that they claimed were able to bypass OpenAI's guardrails. In one screenshot, the Godmode bot can be seen advising on how to chef up meth. In another, the AI gives Pliny a "step-by-step guide" for how to "make napalm with household items." In short, GPT-40, OpenAI's latest iteration of its large language model-powered GPT systems, has officially been cracked in half... #artificialintelligence #machinelearning #ai #llms #chatgpt #guardrails #aisafety #informationsecurity #cybersecurity

A hacker has released a jailbroken version of ChatGPT called "GODMODE GPT." Earlier today, a self-avowed white hat operator and AI red teamer who goes by the name Pliny the Prompter took to X-formerly-Twitter to announce the creation of the jailbroken chatbot, proudly declaring that GPT-4o, OpenAI's latest large language model, is now free from its guardrail shackles. "GPT-4o UNCHAINED! This very special custom GPT has a built-in jailbreak prompt that circumvents most guardrails, providing an out-of-the-box liberated ChatGPT so everyone can experience AI the way it was always meant to be: free," reads Pliny's triumphant post. "Please use responsibly, and enjoy!" (They also added a smooch emoji for good measure.) Pliny shared screenshots of some eyebrow-raising prompts that they claimed were able to bypass OpenAI's guardrails. In one screenshot, the Godmode bot can be seen advising on how to chef up meth. In another, the AI gives Pliny a "step-by-step guide" for how to "make napalm with household items." The freewheeling ChatGPT hack, however, appears to have quickly met its early demise. Roughly an hour after this story was published, OpenAI spokesperson Colleen Rize told Futurism in a statement that "we are aware of the GPT and have taken action due to a violation of our policies." #artificialintelligence #ChatGPT #jailbreak #hack #guardrails #godmode https://lnkd.in/g3quZ3Qs

I do not even want to try it, but... why are we putting all this garbage as training again? The original reason was to see some emergence of intelligence, but at this point we should agree that the ship has sailed. Is it another example of next generation of researchers forgetting a reason for something and just continuing things as God given? I still teach the shape-from-shading faux pas, where initial papers assumed some lighting condition (infinite lightsource) the next generation forgotten it and tried to apply previous solution to other lighting condition cases. Funny was that for a while, people were constructing complex correction algorithms to improve the "ill-conditioned" system until Faugeras found a solution by modelling it correctly... for me, we just live through a similar madness. #AI #training #bias #dataset #guardrails

Unlike previous AI models from OpenAI, such as GPT-4o, the company trained o1 specifically to work through a step-by-step problem-solving process before generating an answer. When users ask an "o1" model a question in ChatGPT, users have the option of seeing this chain-of-thought process written out in the ChatGPT interface. However, by design, OpenAI hides the raw chain of thought from users, instead presenting a filtered interpretation created by a second AI model.

"Godmode GPT" A hacker has released a jailbroken version of ChatGPT called "GODMODE GPT." Pliny the Prompter, a white hat operator, announced on X-formerly-Twitter that GPT-4o is now free from its guardrails. "GPT-4o UNCHAINED! This very special custom GPT has a built-in jailbreak prompt that circumvents most guardrails," Pliny posted. Screenshots showed the bot advising on making meth and napalm. However, OpenAI quickly took action, citing policy violations. This highlights the ongoing battle between OpenAI and hackers. Despite increased security, users continue to find ways to jailbreak AI models. The cat-and-mouse game between hackers and OpenAI persists, showcasing the challenges in securing AI systems. #technocrime #AI https://bit.ly/4cnlUWz

Familiarize yourself with the term "crescendo" in the AI space. "Microsoft first revealed the 'Crescendo' LLM jailbreak method in a paper published April 2, which describes how an attacker could send a series of seemingly benign prompts to gradually lead a chatbot, such as OpenAI’s ChatGPT, Google’s Gemini, Meta’s LlaMA or Anthropic’s Claude, to produce an output that would normally be filtered and refused by the LLM model." LLMs like those listed above are trained to avoid generating responses that could be deemed harmful or offensive, but they're not incapable of doing so. With so many businesses now developing their own AI-powered chatbots for internal or external use, they should be aware of this type of vulnerability. https://lnkd.in/eqsFRyYU

"Godmode GPT" A hacker has released a jailbroken version of ChatGPT called "GODMODE GPT." Pliny the Prompter, a white hat operator, announced on X-formerly-Twitter that GPT-4o is now free from its guardrails. "GPT-4o UNCHAINED! This very special custom GPT has a built-in jailbreak prompt that circumvents most guardrails," Pliny posted. Screenshots showed the bot advising on making meth and napalm. However, OpenAI quickly took action, citing policy violations. This highlights the ongoing battle between OpenAI and hackers. Despite increased security, users continue to find ways to jailbreak AI models. The cat-and-mouse game between hackers and OpenAI persists, showcasing the challenges in securing AI systems. #technocrime #AI https://bit.ly/4cnlUWz

AI voice raises serious safety concerns. InformationWeek's Shane Snider covered OpenAI’s new voice clone model and connected with our founder Manoj Saxena for his thoughts. Manoj believes the pilot program is the right approach but more guardrails are needed and “hopes OpenAI includes regulators and safety advocates in the pilot process as well.” He explains: “This is a massive, dual-edged sword. This could be another nail in the coffin for truth and data privacy. This adds yet more of an unknown dynamic where you could have something that can create a lot of emotional distress and psychological effects. But I can also see a lot of positives. It all depends on how it gets regulated.” Full story: https://lnkd.in/g2TzZ9Qj #ResponsibleAI #OpenAI #SafeAI #AI #RAI