Security researchers tricked LLMs into giving them cocaine recipes by abusing role models for prompt injection

The authors developed an attack called CoT (Chain of Thought) Forgery that involves using an LLM to spoof the terse style of ...