Attack-based phish testing creates fear, anxiety, and doubt in end users. It does not create awareness, harms IT, and misinforms management.
A recent large-scale IEEE study, conducted from Oct. 2019 through Sept. 2020 of 14773 participants found that attack-based phish testing “can have unexpected side effects that can make employees even more susceptible to phishing“.
Attack-based phish testing fails End Users, fails IT Departments, and fails Company Leadership seeking cybersecurity resilience. This article will summarize how and why these failures occur. It concludes with an alternative phishing test with positive outcomes for all three constituents mentioned above. Traditional phish testing does not improve your cybersecurity resilience or protection. In fact, it’s likely to cause harm in many cases.
Large-Scale IEEE Study shows Phish Testing is Counterproductive
A research paper titled “Phishing in Organizations: Findings from a Large-Scale and Long-Term Study” was published in the IEEE Security & Privacy journal in 2020. The paper presents the findings of a long-term study of phishing attacks in organizations.
This research paper reports attack-based phishing tests were not an accurate indicator of an organization’s vulnerability to phishing attacks. In fact, it reports these tests could increase employee susceptibility to these attacks.
To combat these testing failures, the study suggests organizations should focus on developing a more comprehensive approach to phishing defense. Defenses should include a combination of technical controls, employee training exercises, and enhancing company culture. Also be sure to factor in evolving tactics used by attackers and the human factors that contribute to successful phishing attacks.
Overall, the study highlights the importance of taking a holistic approach to phishing defenses in organizations. Do not relying on traditional attack-based phish testing.
Let’s turn our focus now on three specific failure points in attack-based phishing today: Employee Failures, IT Failures, and Reporting/Metrics Failures.
Fundamental Flaws in Attack-Based Phish Testing
Employee failures
Employees experience phishing attacks as harmful. They rarely create “awareness” on how to spot and avoid phishing attacks, and lead to punishing outcomes for failure. At least one study found that employees repeatedly phish tested with attack emails perform more poorly when facing real-world attacks! Why is this?
Employees are surprised with deceptive emails crafted by IT insiders to play on the fears, insecurities, and emotional state of employees. This is meant to mirror what hackers “might do” while attacking employees. However, in case after case, overzealous IT administrators have too often crossed the line and offended employees with their deception-based attack emails. For example:
Chicago Tribune: Phishing Test goes Terribly Wrong
$650 Bonus for GoDaddy Christmas Gift a Phishing Test
Not Again! Another Phishing Test goes Awry at West Midlands Railway
These are examples of attack-based phishing tests going horribly wrong. Employees logged complaints with HR stating that instead of protecting them from harm, IT was causing harm. Did these campaigns improve the preparedness of employees? Did they enhance the cybersecurity resilience of the companies that practiced the measures? It is highly doubtful. There was very little education happening in these activities. Instead, these measures lead to more fear, anxiety, and insecurity in the employees tested.
IT Failures
IT Teams are overworked, understaffed, and pressured to perform ever more devious and diabolical attack-based phishing to “show results”. These attack-emails turn IT teams from protectors, to causes of harm. This in turn leads to a loss of trust, compassion, and effectiveness in an organizations cybersecurity program and less resilience to attack. Why is this?
IT teams are asked by leadership to doing everything possible to protect our company and its employees from breach and compromise. Phish testing is one protective measure conducted to “identify employees who are prone to clicking“. When results don’t show any failures, management asks IT to perform more “difficult” tests. “Your tests have been too easy!” “Make this a hard phishing test to spot, just like hackers might do to breach our company and its networks!” “I want to see valid results!“
Given these imperatives, it’s not hard to see why IT can take things so far. In fact, all too often they make the news cycle as shown above. The unfortunate reality here is that employees are not learning anything from these tests, and the consequences for failure rarely address the root cause of failure; a lack of knowledge on how to identify any phishing attack and delete it.
So IT Teams fail to implement meaningful training measures to educates staff outside of “yet another phish training video“. Employees rarely watch such videos. Nor do they learn from them. The attack-based phish tests is broken and its many failures are reflected in the news headlines detailing companies breached and data stolen and lost.
Reporting Failures
Testing employees with phishing simulations is meant to achieve two purposes. Report on an organizations susceptibility to future attacks, and to remind employees to stay vigilant in reviewing their email. Neither of these metrics was shown in the IEEE study to be tied to effective attack-based phish testing. Essentially phish testing failure rates underreport employee likelihood of clicking on future attacks. Secondly, the study showed a negative impact from such tests on future user behaviors.
Poor Metrics that are flawed
Attack-based phish testing yields 5 to 10% failure rates in many organizations. When Security professionals are asked “What happened to the other 90 to 95% of users?” they are met with blank stares. The reality is that modern attack-based phish testing does not report on false negative rates of employees who did not see, read, or process the phishing test itself. Overtaxed inboxes of management are selecting all messages and deleting them after scanning subject lines to cope with the influx of email. “If its important, they will email me again” has been often quoted. Consequently, these employees are not being tested and this is impossible to report on with a metric.
An Educational, Positive Alternative Phishing Test
CyberHoot has developed a Phishing Test “By Assignment” that eliminates each of these points of failure found in traditional attack-based phish testing. Our assignment-based phishing test walks users through a phishing puzzle with 6 or 7 pieces. The outcome of our assignment-based phishing test are users who have completed their phishing puzzles. These users are armed with a complete picture of phishing attacks. They see all seven different phishing puzzle pieces together in a single clear image. The now know how to safely, efficiently, and confidently identify and delete phishing attacks.
Managed Service Providers (“MSP”) report a dramatic drop in emails to the help desk asking for help determining whether an email is an attack or not. End users report confidence, efficiency, and security in their decision making while reviewing email. These exercises produce the “Awareness” companies seek in their employees. No solution is full-proof, as nothing is perfect. As we describe our novel phish testing exercise and what outcomes are achieved you will see that the results can be quite positive.
A Positive and educational End User Experience
End users pass through a series of questions outlining seven components of an email and must choose if the example given is safe or malicious. There is a “Help Me” button that provides the end user critical information on what they should be looking for to identify an attack. These parameters quiz end users on: (1) Sender, (2) Subject, (3) Greeting, (4) Spelling, Punctuation, and Grammar, (5) Urgency and/or Emotionality, (6) External Links, and (7) Attachments (if present). In walking through this exercise, end users must pass the quiz or retake the test. The outcome of this phishing exercise is a dramatic improvement in employee knowledge, efficiency, and confidence.
An easier and beneficial IT and/or MSP Experience
IT and MSPs no longer need to configure Allow Lists, X-Headers or run Power Shell scripts on inboxes as is required for Attack-based phish testing. Instead, these assignments are sent directly to inboxes without any IT intervention required. The time consuming activities of writing allow lists are gone. The negative blame employees might lay on IT administrators (or MSPs) in traditional Attack-based phishing tests is also gone. Instead, employees are enlightened, educated, and benefit from a positive experience they now attribute to their IT team. This benefits the relationship between IT/MSP team members and company employees. There are fewer cybersecurity incidents, emails to the helpdesk, and a more valuable relationship between IT/MSP and the company.
A more positive and predictive company Leadership Experience
In the past, leadership received a 5-10% phishing failure rate and a 90 to 95% unknown false negative rate for everyone else. In the novel approach designed by CyberHoot, leadership receives a metric approaching 100% compliance. This metric represents every employee taking and passing the assignment-based phishing test. This provides greater predictive accuracy for cybersecurity resilience than is possible with attack-based phish testing (proven to under report exposure levels). CyberHoot’s new method of phish testing provides peace of mind that every last employee has taken the open book test, studied the puzzle pieces enough to pass the test. Employees now operate with a clear seven piece picture of phish attacks with which to make future decisions.
Phish Testing Conclusions
The cybersecurity industry needs to mature. This 30-40 year old emerging field of study has a long way to go. CyberHoot’s innovation can help bring much needed awareness to end users’ cyber-resilience in processing email. Instead of breeding anxiety, uncertainty, and doubt, these assignment-based phish testing exercises yield confidence, efficiency, and security within your employees.
Isn’t it time for you to have a look yourself? Schedule your demo with CyberHoot here or sign up below for a free 30-day trial with no commitment of any kind. CyberHoot guarantees your satisfaction or you may leave at any time.