Analysis & Opinions - Harvard Business Review

AI Will Increase the Quantity — and Quality — of Phishing Scams

| May 30, 2024

Anyone who has worked at a major organization has likely had to do training on how to spot a phishing attack — the deceptive messages that pretend to be from legitimate sources and aim to trick users into giving away personal information or clicking on harmful links. Phishing emails often exploit sensitive timings and play on a sense of urgency, such as urging the user to update a password. But unfortunately for both companies and employees, gen AI tools are rapidly making these emails more advanced, harder to spot, and significantly more dangerous.

Research we published earlier this year showed that 60% of participants fell victim to artificial intelligence (AI)-automated phishing, which is comparable to the success rates of non-AI-phishing messages created by human experts. Perhaps even more worryingly, our new research demonstrates that the entire phishing process can be automated using LLMs, which reduces the costs of phishing attacks by more than 95% while achieving equal or greater success rates. Phishing has five distinct phases: collecting targets, collecting information about the targets, creating emails, sending emails, and finally validating and improving the emails. With the ability to generate human-like text and converse coherently, large language models (LLMs), such as ChatGPT and Claude, can be used to automate each phase.

Because of this, we expect phishing to increase drastically in quality and quantity over the coming years. The threat level varies across industries, organizations, and teams. Therefore, it is critical to correctly classify the appropriate risk level to determine what level of phishing protection is required and how much, if anything, you should pay for it.

For more information on this publication: Belfer Communications Office
For Academic Citation: Heiding, Fredrik, Bruce Schneier and Arun Vishwanath.“AI Will Increase the Quantity — and Quality — of Phishing Scams.” Harvard Business Review, May 30, 2024.

The Authors