AI ‘Hallucinations’ Can Become an Enterprise Security Nightmare

In a blog post on the Vulcan Cyber website, researchers Bar Lanyado, Ortel Keizman, and Yair Divinsky illustrated how one could exploit false information generated by ChatGPT about open-source software packages to deliver malicious code into a development environment. 

They explained that they’ve seen ChatGPT generate URLs, references, and even code libraries and functions that do not actually exist.

If ChatGPT is fabricating code libraries or packages, attackers could use these hallucinations to spread malicious packages without using suspicious and already detectable techniques like typosquatting or masquerading, they noted.