Phishing campaigns and phishing simulators have become a part of security awareness training. Almost every security awareness vendor offers them as a feature, sometimes bundled and sometimes sold as an add on.

At Haekka, we agree that phishing simulations are a valuable feature of a security awareness program; but, we think they are one feature in a suite of features that are required to mitigate the human risk of employee actions and social engineering. Phishing simulations and security awareness training, as it is offered today by the majority of security awareness vendors, does not work to reduce human risk.

Phishing campaigns are not just about training

According to the UK National Cyber Security Centre, a division of the UK government, phishing campaigns are about more than just training:

Lets be honest with each other. Phishing simulations aren't just about training. They are also popular because they produce a metric (e.g. 'Last week 60% of people fell for our phish, this week only 35% fell for it').

Where is the ROI of phishing simulations?

If phishing campaigns are about more than training, how do you find and quantify the ROI of phishing simulations?

We hear often that people run phishing campaigns for 3 reasons:

1. Reduce risk of social engineering attacks.
2. Metrics about risk.
3. Reduce cyber liability insurance.

While all of the above outcomes of phishing simulations have some ROI, the highest value ROI is from the metrics you get from running phishing campaigns. Phishing campaigns, if done regularly and consistently, can help benchmark the effectiveness of your overall security awareness program.

How frequently should phishing simulations be run?

If the primary ROI of phishing campaigns is to provide a metric that serves as a proxy of the performance of your security awareness program and not as a tool to reduce the risk of social engineering attacks or human risk, then it changes the framing of the question of how frequently to run phishing campaigns.

If the goal of phishing campaigns was retention of material or effectiveness of training, there is good data that shows training should be done as frequently as possible, or at least weekly.

When it comes to metrics from phishing campaigns, monthly is the most frequent that we recommend. Even monthly phishing campaigns will miss the impact of new or improved security awareness training methods. Every other month or every quarter is the optimal frequency for phishing campaigns as this gives enough time to test different training methods and interventions. 

What to do in between phishing campaigns

If you are using the metrics from phishing campaigns to measure how well your security awareness training program is working, then you should be making adjustments to your security awareness program based on the results of each phishing campaign. In between each phishing campaign, you should be testing new methods of training such as drip training, triggered training, and user-centric engagement.

When phishing metrics show something is working (less people are falling for phishing campaigns), double down on those methods. If phishing metrics show things are not improving (getting worse or just stagnant), try something new.

Using phishing campaigns to continuously iterate and improve your security awareness program will reduce your human risk in a compounding trajectory.