Wireless Communications and Mobile Computing

Volume 2018, Article ID 4965343, 14 pages

https://doi.org/10.1155/2018/4965343

## Packet Forwarding Strategies in Multiagent Systems: An Evolutionary Game Approach

^{1}School of Information Management, Shanghai Lixin University of Accounting and Finance, Shanghai 201620, China^{2}School of Information Science and Technology, Donghua University, Shanghai 201620, China

Correspondence should be addressed to Yuanjie Li; moc.qq@1110434

Received 19 March 2018; Accepted 24 June 2018; Published 8 July 2018

Academic Editor: Maode Ma

Copyright © 2018 Yuanjie Li and Xiaojun Wu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

In multiagent systems (MASs), agents need to forward packets to each other to accomplish a target task. In this paper, we study packet forwarding among agents using evolutionary game theory under the mechanisms of Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA). Packet forwarding among agents plays a key role to stabilize the whole MAS. We study the transfer probability of packet forwarding of agents at the idle state or the busy state and computer the probability of the packet forwarding for a MAS. When agents make their decisions to select* Forward* or* No-Forward* strategy, a packet forwarding evolutionary game model is built to reflect the utilities of different packet forwarding strategies. Two incentive mechanisms are introduced into the game model. One is to motivate agents to strengthen cooperation; the other is to encourage agents to select the* No-Forward* strategy to save energy while they are in the busy state. The parameter value that encourages an agent to select the No-Forward strategy is inversely proportional to the average probability of the packet forwarding. The replicator dynamics of agent packet forwarding strategy evolution are given. We propose and prove the theorems indicating that evolutionarily stable strategies (ESSs) can be attained. The results of simulation experiments verify the correctness of the proposed theorems and the effects of the two incentive mechanisms and the probability of packet forwarding, which assures the robustness of evolutionary stable points among agents in MASs.

#### 1. Introduction

Multiagent systems (MASs) are computerized systems composed of multiple interacting intelligent agents that have limited energy and self-organization ability. To achieve a common goal, MASs require them to self-organize into a network and collaborate with one another [1]. MASs are used most often in the engineering and technology fields. Examples of applications are disaster response, mobile robots, spacecraft, and sensor network [2].

The agents need to interact with each other as much as possible to accomplish a target task. Cooperation is a fundamental problem in the distributed control community; the agents must effectively cooperate with each other for mutual benefit [3, 4]. The cooperative results lie in the actions taken by the interacting agents. In the process of cooperation, agents transmit messages to each other through packet forwarding. We assume that all agents in MASs are rational and aim at maximizing their own profits. When packet forwarding will incur certain costs, each agent makes a decision whether to forward a packet or not according to its own benefit. Hence, the pack forwarding can be described as a game, where players are rational agents with selfish behaviors and the strategy of a player is to* Forward* or* No-Forward* a packet. Agents can make a decision to forward packets or not according to their benefits.

As a mathematical tool, game theory mainly focuses on the competitive and cooperative relationship of the participants, and it has been widely applied in the field of cooperation. Typical examples of evolutionary games include repeated prisoners’ dilemma and snowdrift games [5–7]. In [8], Shen et al. compare different methods of games and regard the evolutionary game as a good method to solve the problem of node cooperation. In the evolutionary game process, the individuals maximize their utilities by selecting a higher gain strategy, the ratio of individuals who select the corresponding strategies tends to be stable, and the population ultimately will attain a dynamic equilibrium state. Thus, the study of this cooperative packet forwarding model and evolutionary game theory in MASs is appropriate.

In the paper, using a Markov chain model, we calculate the probability of packet forwarding while agents are in idle or busy states and analyze the packet forwarding behaviors of the agents. We then build a packet forwarding strategies game model, which defines the payoffs of the different strategies. Moreover, we introduce two incentive mechanisms into the game, one to increase agent cooperation by selecting* Forward* strategy and the other to increase agent noncooperation by selecting* No-Forward* strategy while the agent is in the busy state. Furthermore, using replicator dynamics, we obtain and prove the evolutionary stability theorems and inferences, which establish the conditions to attain steady states for the packet forwarding strategies game. The main contributions of this paper are as follows:

(1) The Markov chain model is used to calculate the probability of packet forwarding for an agent at the idle or busy state. We establish a packet forwarding strategies game model with two incentive mechanisms. One incentive mechanism is to promote cooperation among agents by forwarding packets and the other is to promote noncooperation among agents by selecting the* No-Forward* strategy to save energy while the agent is in the busy state.

(2) Replicator dynamics is used to prove the evolutionary stability theorems and inferences, which provide the conditions to attain stable states for the packet forwarding strategies game model in MASs.

(3) The experiments verify the correctness of the provided theorems and inferences. Also, the experiments show the effects of two incentive mechanisms and the probability of packet forwarding of agents on the rate of convergence to stable states.

The rest of the paper is organized as follows. We first discuss the related research in Section 2 and set up a packet forwarding Markov model in Section 3. We propose an evolutionary game model of packet forwarding strategies and present the theorems including the conditions of ESSs in Sections 4 and 5. The simulation results are shown in Section 6 and, finally, Section 7 is the conclusion.

#### 2. Related Work

In recent years, to provide an incentive for the agents/nodes to cooperate, two mechanisms have been devised: incentive mechanisms and punishment mechanisms, which incentivize agents/nodes to forward the others’ packets and punish misbehaving nodes. Most research involves strengthening packet forwarding among nodes/agents by setting up incentive-based systems [9, 10] or reputation-based systems [11, 12]. Most incentive-based mechanisms give a reward to nodes/agents for participating in packet forwarding [9]. In a reputation-based system, a node’s behavior is monitored and evaluated by its neighbors and each node is required to keep track of its neighbors’ reputation values, which are updated based on weighted calculations of the value of the node’s own observations and the value of the other nodes’ recommendations [12].

Also packet forwarding strategies and cooperative strategies among nodes have been studied by different types of game theory in wireless networks, such as repeated games [13, 14], no-cooperation games [15–17], bargaining games [18], and evolutionary games [19–25].

Zhu et al. [13] propose an adaptive repeated game scheme to ensure the cooperation among nodes in wireless networks and implement a self-learning algorithm to improve cooperation probabilities. The simulation results show that the selfish nodes are indeed more likely to cooperate with each other by the schemes above, but they may be unsuitable for MASs. To stimulate selfish nodes to cooperate, Xu et al. [17] propose a Win-Stay, Lose-Likely Shift (WSLLS) approach in a Prisoner’s Dilemma (PD) game and a utility-based function is applied to evaluate a player’s (i.e., node) performance. Experimental results show that the approach is effective in stimulating cooperation in different settings. Akkarajitsakul et al. [18] propose an approach to cooperatively forward packets based on coalition formation among mobile nodes. They use a bargaining game to find the most suitable probabilities in which each node would help other mobile nodes. They present a distributed algorithm to obtain the stable coalitions and they use a Markov-chain-based analysis to evaluate the stable coalitional structures that are obtained from the distributed algorithm. Performance evaluation results show that the mobile nodes with coalition formation have higher payoffs than mobile nodes acting alone.

Among these methods of evolutionary game theory, some stimulation mechanisms are proposed to incentivize the nodes’ cooperation. Shen et al. [19] propose stimulating mechanisms to promote trust cooperation among nodes and analyze the dynamic evolutionary process of nodes selecting the trust strategy and derive several conditions with which the networks can attain the stable states. Li et al. [20], based on [19], consider a factor of packet loss in the data retransmission process and introduce a strategy adjustment mechanism into the evolutionary game process. This strategy adjustment mechanism compensates for the fact that the replicator dynamic model cannot reflect the requirements of individual strategy adjustments. The experiments show that the rate of convergence to reach the stable state of the strategy adjustment mechanism is faster than that of the normal replicator dynamic evolutionary method. Based on evolutionary game theory, Chen et al. [22] adopt a suitable dynamic incentive mechanism for WSNs, which emphasizes the nodes’ adjust strategies forwardly and passively. This mechanism enables the selfish nodes to cooperate with each other and keep the network normal.

Among these methods of evolutionary game theory, some researchers study packet forwarding among nodes in different scenarios. Li et al. [23] study the adaptive packet forwarding of potential selfish nodes in mobile social networks (MSNs) and propose an incentive compatible multiple-copy packet forwarding (ICMPF) protocol to reduce the delivery overhead and to perpetuate a successful packet delivery. They design an evolutionary game model to guide the forwarding behavior of interaction nodes. Finally, they prove that the strategy dynamics eventually achieves the ESS and develop another strategy dynamics method which achieves the ESS. This method includes the nodes’ distributed learning algorithm. Considering the noisy observation of transmissions and the loss of packets, Tang et al. [24] study packet forwarding among cooperative nodes in a one-hop unreliable channel. Based on evolutionary game theory, they propose an indirect reciprocity framework and enforce packet forwarding strategies in mobile ad hoc networks (MANETs). They also analyze the evolutionary dynamics of cooperative strategies game model and calculate the threshold of cost-to-benefit ratio to ensure cooperation among nodes. Lastly, the proposed cooperative solution is verified by the numerical simulations. To enforce cooperation in the case of channel noise, Wang et al. [25] focus on one-hop information exchange and design a packet forwarding game model with imperfect private monitoring and propose a state machine-based strategy to reach Nash equilibrium which proved to be a sequential state with carefully designed system parameters.

Compared to [19–25], our work is distinctive. Our packet forwarding strategy is derived partly from trust using game theory [19, 20]. Considering the packet forwarding randomness of an agent’s communication state, derived from the Markov model [26], we calculate the probability of packet forwarding of agents at the idle or busy state using properties of a Markov chain, build a packet forwarding game model with the probability of packet forwarding, and introduce two incentive mechanisms in the game model. One incentive mechanism is to promote agents to strengthen cooperation and the other is to encourage agents to select* No-Forward* strategy to save energy while they are in the busy state. We set the parameter value inversely proportional to the probability of packet forwarding. In contrast, there is only one simple incentive cooperation mechanism in [19, 20]. Furthermore, we consider the agent’s payoff impacted by the probability of packet forwarding, which is related to whether the agent’s communication state is idle or busy, while Wang et al. [25] consider only the impact of probability of packet loss on node payoff. Li et al. [23] and Tang et al. [24] calculate the payoffs of forwarding packets between nodes without taking into account the interaction cooperation of nodes, which is different from ours. Thus, our packet forwarding strategies game is different from the related literature described above.

#### 3. Packet Forwarding Markov Model

##### 3.1. Packet Forwarding Markov Chain

Packet forwarding among agents in MASs is not a deterministic process. The present value of the process is independent of all past values. The process is a no-after-effects process and hence has Markov properties.

Time series of packet forwarding can be expressed by a Markov chain. If , then

In this section, we build a Markov model of packet forwarding among agents in MASs. Let us assume the following:

(1) If an agent has a new packet to forward, it forwards it at the beginning of the next time-slot.

(2) If an agent successfully forwards its packet, it can forward a new packet in the next time-slot.

(3) If an agent detects a collision, it forwards its packet in each subsequent time-slot until the packet is successfully forwarded.

With the assumptions, each agent is in either an idle state or a busy (backlogged) state. An agent is in the busy state if the number of packets that need to be transmitted exceeds its transmitting capacity, which leads to buffer overflow and loss of data packets. Furthermore, because agents have limited computing and communication capability with a multihop and many-to-one communication method, when the wireless communication channels present noise, or the network topology changes, or the packets suddenly increased because of an emergency, the agents may easily be in a busy state that will cause network delay and loss of data packets. The decision to forward the packet or not depends only on whether the agent is busy or not. Therefore, the packet forwarding decision in one time-slot for each agent is Markovian. Since agents always have packets to forward, the notations are shown in Table 1.