#### Abstract

The human is considered as the important link in the phishing attack, and the e-mail security provider encourages users to report suspicious e-mails. However, evidence suggests that reporting is scarce. Therefore, we study how to motivate users to report phishing e-mails in this paper. To solve the problem, a tripartite evolutionary game model among e-mail security providers, e-mail users, and attackers is constructed. We obtain the desired evolutionary stable strategy through solving the replicator dynamics equations. Moreover, the evolution process to the desired evolutionary stable strategy is derived, which can guide the e-mail security provider to make a reasonable incentive mechanism. Lastly, we experiment with a large real-world e-mail network. The experiment results show that our model is effective and practical.

#### 1. Introduction

While e-mail is widely used as an efficient communication tool on the Internet, users are plagued by spam and phishing e-mails. According to the report by Kaspersky Lab, the share of such unwanted e-mails in e-mail traffic amounted to 50.37% in 2020 [1]. The spam wastes user’s time and misuses valuable network resources. Even worse, phishing e-mails can steal personal confidential information and compromise government systems and companies spanning every economic sector [2, 3]. FBI estimated that phishing e-mails caused over $1.8 billion financial loss in 2020 [4]. Therefore, it is important to protect e-mail users from phishing e-mails.

Since phishing e-mails aim at exploiting human weakness, effective mitigation would require addressing issues at the technical and human layers [5, 6]. In the technical layer, machine learning is the main approach to detecting phishing e-mails [2, 7, 8]. However, attackers may bypass detection techniques. Besides, users are the most vulnerable link in the phishing ecosystem [9]. Therefore, users play an important role in preventing phishing attacks. Today, the most widely implemented user-focused intervention is to train individuals to increase their security awareness [10–13]. Major training methods emphasize reporting; they encourage users to report suspicious e-mails to e-mail security providers [14]. This is because that reporting makes early detection possible and allows e-mail security providers to inform other potential victims before the attack spreads. To make reporting easy, e-mail security providers have provided a convenient mechanism (e.g., Cofense Reporter [15]). Though there are many benefits of reporting phishing e-mails, most users do not choose to report phishing e-mails [16–18].

Prior research explained why users do not report phishing e-mails based on Social Cognitive Theory [14]. However, it does not provide an effective incentive mechanism. In this paper, we study how to encourage users to report phishing e-mails? An e-mail security provider (ESP), such as Gmail, is responsible to protect their paying users from phishing e-mails. In other words, ESP should formulate a reasonable incentive scheme to encourage users. We adopt the game theory to analyze the important factors affecting user behavior from the perspective of players’ expected benefits. Meanwhile, each player cannot obtain all the information to make the optimal decision in the actual situation. In other words, they are all bounded rationality [19, 20]. Therefore, we adopt the evolutionary game theory that satisfies this premise. The main contributions of this paper are summarized as follows:(i)We construct a tripartite evolutionary game model among e-mail security providers, e-mail users, and attackers. Then, we obtain an expected evolutionary stable strategy, which is all users choose to report suspicious e-mails, all attackers do not send phishing e-mails, and all e-mail security providers choose to check reported e-mails. Moreover, we derive an evolution process that can guide the e-mail security provider to reach the desired stable state.(ii)To validate the model, we experiment with a large real-world e-mail network. The experiment results show that our model is effective in the real-world e-mail network.(iii)We explore the influence of the attack punishment on the evolution path by numerical simulation. The simulation results show that increasing attack punishment has a great influence on the rate at which attackers evolve toward nonattacking.

The remainder of this paper is organized as follows. In Section 2, an evolutionary game model including e-mail security providers, e-mail users, and phishing attackers is proposed. Section 3 constructs the replicator dynamics equations and obtains the evolutionary stability strategy and its evolution process. In Section 4, we explore the influence of the network topology, the attack punishment, and the user payoff on the evolution path. Section 5 summarizes the whole paper.

#### 2. Evolutionary Game Model

In this section, we define the strategies and payoffs of the game players.

##### 2.1. Problem Statement

The problem consists of three game groups: e-mail security providers, e-mail users, and phishing attackers. The e-mail security provider aims to protect paying users from phishing e-mails with the least cost and increase the number of paying users. The e-mail user aims to obtain a secure e-mail experience with minimal cost. The goal of an attacker is to send phishing e-mails to steal personal information with the minimum risk of being detected by the defender. We assume that the game groups have bounded rationality.

##### 2.2. Analysis of Strategies

In the tripartite evolutionary game model, the e-mail security provider has two alternative strategies: supervision and nonsupervision. On the one hand, ESP can choose supervision, which needs to deploy additional security personnel and equipment to check and store the reported e-mails. While supervision incurs additional cost, it will improve the ESP’s reputation and thus increase the number of paying users. On the other hand, ESP can choose to ignore reported e-mails to reduce the corresponding management cost. However, in the long run, some negative reputation effects will be generated and the number of paying users may decrease.

Attackers have two alternative strategies: attacking and nonattacking. When an attacker sends out phishing e-mails, he may successfully trick users or not. Whether an attack succeeds or not depends on the user behavior. We assume the probability of successfully attacking is one, which equivalently reduces the attack cost. Because this paper mainly studies the strategy of the e-mail user, and the attacker can be simplified. After the attacker sends phishing e-mails, he may be reported by users or not. If he is reported, he will be punished by the ESP selecting supervision.

E-mail users have two alternative strategies: reporting and nonreporting. After receiving an e-mail, the user will check whether the e-mail is a phishing e-mail. When the e-mail is suspected to be a phishing e-mail, the user may report it or not. If the user chooses to report a suspicious e-mail, the reported e-mail may be a phishing e-mail or not. In other words, the user may correctly report a phishing e-mail or not. The user will be rewarded by the ESP choosing supervision if he correctly reports a phishing e-mail. If the user falsely reports an e-mail, he will not be held accountable because of the service principle. Moreover, he will receive feedback from the ESP choosing supervision and thus avoid missing important e-mails. However, no matter which choice e-mail users make, a phishing e-mail will cause losses to e-mail users, including the losses of money and time.

##### 2.3. Payoff Matrix and Expected Benefit

The tripartite payoff matrix is shown in Table 1. , , and , respectively, represent the payoff of users, attackers, and ESPs in the corresponding strategy. Suppose that the proportion of users selecting reporting is ; then, the proportion selecting nonreporting is . Suppose also that the proportion of attackers selecting attacking is , and the proportion selecting nonattacking is . The proportion of ESPs selecting supervision is , and the proportion selecting nonsupervision is . Obviously, , , and .

Because the payoff of each game group will be affected by the strategies of the other two game groups, there are eight combinations of strategies for users, attackers, and ESPs: (reporting, attacking, and supervision), (reporting, nonattacking, and supervision), (nonreporting, attacking, and supervision), (nonreporting, nonattacking, and supervision), (reporting, attacking, and nonsupervision), (reporting, nonattacking, and nonsupervision), (nonreporting, attacking, and nonsupervision), and (nonreporting, nonattacking, and nonsupervision). The payoff of each combination is shown in equations (1)–(8). The parameters and their meanings are shown in Table 2, and all parameter values are not less than zero:

As shown in equations (9)–(11), represents the expected benefit of users adopting reporting, represents the expected benefit of users adopting nonreporting, and represents the expected benefit of users:

As shown in equations (12)–(14), represents the expected benefit of attackers adopting attacking, represents the expected benefit of attackers adopting nonattacking, and indicates the expected benefit of attackers:

Similarly, represents the expected benefit of ESPs employing supervision, represents the expected benefit of ESPs employing nonsupervision, and indicates the expected benefit of ESPs, as shown in the following equation:

#### 3. Equilibrium Analysis of the Evolutionary Game Model

In this part, we construct replicator dynamics equations. By analyzing the Jacobian matrix, we obtain evolutionary stable strategies and their evolution process.

##### 3.1. Replicator Dynamics

The replicator dynamics equation of users is shown in the following equation:

The replicator dynamics equation of attackers is shown in the following equation:

The replicator dynamics equation of ESPs is shown in the following equation:

##### 3.2. Equilibrium Solutions and Stability Analysis

To get the equilibrium solution of the above model, a replicator dynamics equation set is required, as shown in the following equation:

In (19), there are eight pure-strategy equilibrium points *E*_{1} (0, 0, 0), *E*_{2} (0, 0, 1), *E*_{3} (0, 1, 0), *E*_{4} (1, 0, 0), *E*_{5} (1, 1, 0), *E*_{6} (1, 0, 1), *E*_{7} (0, 1, 1), and *E*_{8} (1, 1, 1). In general, there exists a mix-strategy equilibrium point *E*_{9} (). According to Selten [21] and Ritzberger and Wainwright [22], if and only if a strategy combination is a pure-strategy Nash equilibrium, it will be asymptotically stable in the dynamic replication system of the tripartite evolutionary game. Moreover, the asymptotically stable equilibrium point must be the evolutionary stable strategy (ESS). Thus, the ESS must be a pure-strategy Nash equilibrium [23]. Hence, *E*_{9} is not an evolutionary stable strategy because it is a mix strategy. In the following part, we analyze the asymptotic stability of the other eight equilibrium points. The Jacobian matrix of the tripartite evolutionary game is as follows:

According to Lyapunov [24], a point is an evolutionary stable strategy if and only if all eigenvalues of the Jacobian matrix corresponding to the point are negative. The eigenvalues can be obtained by solving the Jacobian matrix, and then, the asymptotic stability of each point is analyzed. As shown in Table 3, *E*_{3} (0, 1, 0), *E*_{6} (1, 0, 1), *E*_{7} (0, 1, 1), and *E*_{8} (1, 1, 1) have the possibility to be the ESS, and the remaining four equilibrium points are saddle points.

The strategies represented by *E*_{3} (0, 1, 0) are that users do not report suspicious e-mails, attackers send phishing e-mails, and ESPs do not supervise reported e-mails. The prerequisite for *E*_{3} (0, 1, 0) to be the ESS is , that is, the reputation positive effects generated by supervision are less than the supervision cost.

The strategies represented by *E*_{6} (1, 0, 1) are that users report suspicious e-mails, attackers do not send phishing e-mails, and ESPs supervise reported e-mails. The prerequisite for *E*_{6} (1, 0, 1) to be the ESS is , that is, the reporting cost is less than the user’s payoff for falsely reporting an e-mail, the attack gain is less than the attack loss, and the reputation positive effects generated by supervision are greater than the supervision costs. This prerequisite can prompt the ESP to implement supervision and eventually enable attackers and users to evolve into nonattacking and reporting. The equilibrium point is the final stable state expected by this paper.

The strategies represented by *E*_{7} (0, 1, 1) are that users do not report suspicious e-mails, attackers send phishing e-mails, and ESPs supervise reported e-mails. The prerequisite for *E*_{7} (0, 1, 1) to be the ESS is and , that is, the reporting reward from ESP is less than the report cost, and the reputation positive effects generated by supervision are greater than the supervision cost.

The strategies represented by *E*_{8} (1, 1, 1) are that users report suspicious e-mails, attackers send phishing e-mails, and ESPs supervise reported e-mails. The prerequisite for *E*_{8} (1, 1, 1) to be the ESS is , that is, the reporting reward from ESP is greater than the report cost, the attack gain is greater than attack loss, and the reputation positive effects generated by supervision are greater than a third of the supervision costs plus reward cost.

As shown in Figure 1, the evolution process is *E*_{3} (0, 1, 0)⟶*E*_{7}(0, 1, 1)⟶*E*_{8} (1, 1, 1)⟶*E*_{6} (1, 0, 1).

*E*_{3} (0, 1, 0) is the initial state, namely, nonreporting, attacking, and nonsupervision. All ESPs choose supervision if . Therefore, the state *E*_{3} (0, 1, 0) will be converted to *E*_{7} (0, 1, 1) if the positive reputation effect is more than the supervision cost. The state *E*_{7} (0, 1, 1) will be converted to *E*_{8} (1, 1, 1) if , that is, the reporting reward is more than the reporting cost. Therefore, in this stage, the ESP should increase the reporting reward or decrease the reporting cost . Lastly, the state *E*_{8} (1, 1, 1) will be converted to *E*_{6} (1, 0, 1) if , that is, the attack loss is more than the attack gain. Therefore, in this stage, the ESP should increase attack punishment strength.

#### 4. Numerical Simulation

In this part, we first verify that our model is effective in the real-world e-mail network. Then, we explore the influence of two important parameters on the evolution path.

##### 4.1. Numerical Simulation of Network Topology

The replicator dynamic equation assumes that e-mail users are evenly mixed, that is, the user can learn from each other. However, in practice, the e-mail user learns from his neighbourhood. Therefore, we experiment with a large real-world e-mail network. The e-mail network dataset was collected in North University of China from September 2016 to March 2018 and includes 452 e-mail users and more than 10000 edges. The initial parameters are set as follows: , , , , , , , , and . Besides, to objectively evaluate the evolution path of players, we start from a neutral point. In other words, we set the initial proportions of users, attackers, and ESPs as 0.5. We simulated with the above parameters for 1000 times. Figure 2 shows the evolution paths of users, attackers, and ESPs in an evenly mixed e-mail network. Figure 3 shows the evolution paths of users, attackers, and ESPs in the real-world e-mail network. The results show that the network structure of e-mail users has little influence on the evolution path. Thus, our model is practical and effective.

##### 4.2. Numerical Simulation of Variable Parameters

Among all the parameters, there are two important parameters in the model: the user payoff for falsely reporting an e-mail and the attacker loss when the phishing e-mail is reported to ESP adopting supervision . In this part, the influences of the two parameters on the evolution paths of participants will be studied. In practice, the user group is neutral, the attacker group prefers to send phishing e-mails to get gain, and the ESP prefers to nonsupervision to reduce the corresponding management cost. Thus, in the following simulations, we set the initial proportion of users selecting reporting, attackers sending phishing e-mails, ESP selecting supervision as 0.5, 0.9, and 0.1, respectively.

In this paragraph, we study the influence of on the evolution path. We set , , , , , , , and , keeping other parameters and the initial proportions of players unchanged. As shown in Figure 4, the convergence rate of e-mail users and attackers is accelerated as increases. is the user payoff for falsely reporting an e-mail to the ESP selecting supervision. On the one hand, the reported e-mail is more important; will increase. On the other hand, the feedback from ESP is sooner, and is larger. Therefore, increasing the feedback speed can accelerate the convergence speed of users and attackers. Moreover, a suspicious e-mail is more important; the user is more willing to report the e-mail.

In this paragraph, we study the influence of on the evolution path. We set , , , , , , , and , keeping other parameters and the initial proportions of players unchanged. As shown in Figure 5, the convergence rate of attackers is accelerated as increases. Therefore, the ESP should increase attack punishment strength.

#### 5. Conclusions

The main goal of this paper is to mitigate phishing e-mails from the human layer. As the human is considered as the important link in the phishing attack, the e-mail security provider can reduce the phishing attack through cooperating with e-mail users. Therefore, we construct a tripartite evolutionary game model, which considers the payoffs of e-mail security providers, e-mail users, and attackers. Through analyzing the eigenvalues of the Jacobian matrix corresponding to each equilibrium point, we obtain four possible evolutionary stable strategies. Moreover, we obtain the evolution process of the four evolutionary stable strategies, which can guide the e-mail security provider to make a reasonable mechanism to reach the desired state. Finally, we verify that our model is effective in a real-world e-mail network.

#### Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

#### Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

#### Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant no. 61772478).