Abstract
Federated learning (FL) is an emerging collaborative machine learning method. In FL processing, the data quality shared by users directly affects the accuracy of the federated learning model, and how to encourage more data owners to share data is crucial. In other words, how to design a good incentive mechanism is the key problem in FL. In this paper, we propose an incentive mechanism based on the enhanced Shapley value method for FL. In the proposed mechanism, the enhanced Shapley value method is proposed to measure income distribution, which takes multiple influence factors as weights. The analytic hierarchy process (AHP) is used to find the corresponding weight value of the influence factors. Finally, the numerical experiments are carried to verify the performance of the proposed incentive mechanism. The results show that compared with the Shapley value method considering the single factor, the income distribution of all participants can better reflect multiple factor contribution when using the enhanced Shapley value method.
1. Introduction
With the rapid development of the artificial intelligence technology, it is quite common for cross-organizational and cross-institutional data use. How to ensure data privacy and security while realizing data sharing [1, 2] have attracted extensive attention, which is a fundamental challenge in the era of artificial intelligence [3–6]. Federated learning (FL) is a new distributed machine learning method to address the data island problem, and it was first proposed by Google in 2016 [7–9], which can be applied to the Android phone user’s local model updating and training between multiple participants or multiple computing nodes. Nowadays, it has been used in all walks of life widely.
In the process of FL, the incentive mechanism is an important research topic [10–12]. To make all data owners contribute their data for model training actively, it is necessary to establish a reasonable incentive mechanism to encourage data owners to share valuable data. A valuable incentive mechanism can enable all participants to continuously and carry out efficient model training and improve the final trained federated model more accuracy. When there is no a meaningful incentive mechanism, all participants do not update data in real time in the process of data federation and do not contribute to FL actively, which will not obtain the accurate training model, and all participants dare not use the trained model. It will also cause a waste of time and economic investment of all participants.
Recently, there are a series of research results about the incentive mechanism of FL. In [11], the authors proposed an incentive scheme for a local, client machine learning model for FL, which solves the problem that participants may opt out of the federated system and provides useless information to the federated system without satisfactory incentives. To facilitate collaborative machine learning among multiple model owners, the work in [12] proposed a hierarchical incentive mechanism framework, which solves the problem of hierarchical structure within federated systems. Zhan et al. [13] designed a deep reinforcement learning-based incentive mechanism, which solves unique the nonsharing of data and difficult contribution evaluation. Yu et al. [14] proposed a divide budget incentive method, which solves the restriction of temporary mismatch between contribution and reward in FL. From these researches, we note that participant’s submodel influences the quality of trained model generated from FL. Generally, the participants have an enthusiasm to cooperate with each other for improving the quality of model. In [15, 16], all participants in FL are considered as having equivalent contributions. However, the contributions of participants have distinct in some cases. For example, the FL model is applied in WeBank, which has cooperated with such financial institutions, insurance companies, and enterprises.
In these existing works, they do not take multiple distribution factors such as cost input, risk factor, and data quality that affect the distribution of benefits. To obtain income distribution fair and just, we proposed an incentive mechanism based on the enhanced Shapley value method for FL. In the proposed method, all federated participants get different federated incomes according the contribution in model training, and there is no free-riding among all participants.
The major contributions of this paper are summarized as the threefold: (1)We construct the federated incentive model for multiple participants, introduce the cooperative game to the incentive model in FL, and consider multiple factors that affect the income distribution of participants in the FL system(2)To allocate the income for the federated participants more fair, we propose the analytic hierarchy process (AHP) to construct the expert judgment matrix according the multiple contribution factors, which achieves the corresponding weights of all the influencing factors(3)The effectiveness of this incentive mechanism is verified through numerical experiments. Experimental results show that the income distribution of all participants can better reflect multiple factor contribution
The rest of this paper is organized as follows: The related work and our main contribution of this paper are introduced in Section 2. The definition of FL, cooperative games, Shapley value, and the AHP are introduced in Section 3. The fair federated model and incentive enhanced Shapley value method are established in Section 4. The rationality of the method is verified by numerical experiments and numerical comparison in Section 5. The conclusion and future work are drawn in Section 6.
2. Related Work
The model training accuracy of FL is affected by incentive mechanism, which has been widely concerned at present [15–17]. In [18, 19], the authors proposed an incentive mechanism which combined the contract and reputation can improve the efficiency of federated model training. In [20], an optimization method in the FL wireless networks was formulated, which captures two trade-offs and obtains the global optimal solution by characterizing the closed-form solutions to all subproblems. In [21, 22], within the Stackelberg framework, a motivation-based interaction model was established between global servers and participating devices to encourage participants taking part in cooperation. Based on fair allocation for wireless network resources, Li et al. [23] proposed a new objective of resource optimization, which promote a fair society from the perspective of fairness, flexibility, and efficiency. For the linear regression model, the work [24] designed an incentive payment structure to encourage agents provide high-quality data, which can describe the impact of data points on the loss function of the model. The game framework was proposed in [25, 26] to solve the efficiency problem of multiparty model.
In the incentive mechanism, the return fairness is a worthy studying problem. The work of [27] studied the dynamics of coalition formation under the bounded rationality condition, which considers the situation that the profit of each team is given by the submodular function, and proposed a profit distribution scheme based on the concept of marginal utility. Xiong et al. [28] proposed an incentive mechanism for the multiattribute user selection, which effectively improves sensing user quality for mobile crowdsensing. The Shapley value method is used to distribute fair the benefits generated in the process of the coalition [29], which is the most widely used method to evaluate fairness [30, 31]. In [32], the authors combined the quality estimation with a monetary incentive and used the Shapley value method to determine the cost of each household, which was based on the effect of goodness in saving and remaining points. In [33], a repertoire of efficient algorithms was proposed to meet the time-consuming challenge in the Shapley value requiring process. Liu et al. [34] proposed a blockchain-based FedCoin peer-to-peer payment incentive mechanism for FL and used the Shapley value method to distribute federated revenue.
In the existing works, they do not take multiple related factors such as cost input, risk factor, and data quality that affect the distribution of benefits. In order to fully consider the impact of multiple factors on the fairness of distribution, an incentive mechanism for the FL system is proposed to achieve fairness and justice, which uses the enhanced Shapley value method. The proposed incentive mechanism introduces the cooperative game to the incentive model and considers multiple factors that affect the income distribution of participants. The analytic hierarchy process (AHP) is used to construct the expert judgment matrix, which achieves the corresponding weights of all the influencing factors.
3. Preliminaries
3.1. Definition of Federated Learning
Let be defined as data owners, and they all hope to merge their respective data to train a machine learning model. A model is trained by a traditional method, which puts all data together. A model can be cooperatively trained by the data owners of claimed FL system in learning process, which the data never leaked to other participant via any data owner in this process. Moreover, the precision of , defined as , should be pretty close to the property of and . Generally, if there is a satisfies the condition
we say that the FL algorithm has -accuracy loss [4].
3.2. Cooperative Games
The is defined cooperative game and satisfies two conditions as follows [29, 35, 36]: where is a finite set of participants, , is characteristic function, is the set of all the subsets of . Let is participants’ income function, indicates the coalition income and (defined in formula (6)) is the income of participant in , which satisfy two constraints:
Formula (4) is named as collective rationality, which indicates that the sum of all participants’ incomes from the maximum income in should be equivalent to , and formula (5) is claimed coalition rationality, which shows that the sum of profit allocated to all probable coalitions must be greater than or equivalent to . When , the formula (5) is individual rationality.
3.3. Shapley Value
Shapley value was presented by Shapley [29, 37] in the cooperative game theory, which can effectively solve the problem of cooperative income distribution and define as the following equation: where , , is the quantity of participants in subset , is weight coefficient, is the income of subset and satisfies Equations (2) and (3), the term evaluates the marginal contribution of to the coalition , and shows the benefit of other players in subset other than .
4. Federated Learning Incentive Mechanism
Before establishing the FL incentive model, we make the following assumptions: (1)All participants have the ability to pay for FL, and in the process of income distribution, they adopt the best income distribution scheme.(2)All participants are willing to participate in the coalition and do not withdraw from each sub-coalition, and all parties are very satisfied with the final income distribution scheme.(3)All participants are completely trustworthy, and there is no cheating.(4)FL should adopt multiparty agreement to recognize the income distribution scheme to ensure the smooth implementation of the strategy.
4.1. FL Incentive Model
The proposed FL incentive model is as shown in Figure 1, and in this model, we assume that there are factors and participants and establish their federated models. We use different colors to represent the process of different factors for different participants, the direction of model updating, and income distribution in the whole FL process.
Step 1. Under the influence of factor 1, factor 2,…, and factor on the federated income, we distribute model 1, model 2,…, and model to participant 1, participant 2,…, and participant and then train the submodels respective of data participants. The data of the participants do not leave the local, to protect the privacy of the data in their respective databases.
Step 2. Every time the model runs, the parameters of the submodels are updated, until the end of the model run, and the models are aggregating to get the federated aggregation model we need.
Step 3. Every time the model runs, using formula (19), each participant will receive the income from the federated expected to cost in round .
Step 4. When the model is running finished, the parameters of all submodels are aggregating to get the final aggregated federated model. Finally, the federated income is allocated according to the enhanced Shapley value method, and each participant will achieve income from formula (20).
The income distribution of each participant is affected by factors, which are allocated to rely on the contribution of each participant to the whole federation. This design makes participants get the distributed federated benefits more fairly and get an accurate federated model. Because the influencing factors impact income of participants, the enthusiasm of participants will be affected and, finally, impact the accuracy of the federated model. If each participant actively cooperates in the federated process and tries to take part in the FL model to update data in the local database in real time, the parameters of the model training will also be updated, and the final federated aggregation model will also be updated. This cycle will continue until the end of the whole model training. Finally, the total federated income generated is distributed according to the enhanced Shapley value method.
4.2. Contribution Assessment
To encourage the participants to actively participate in the FL, we set each participant to be treated fairly based on their contribution and meet three fair conditions at the same time: (1)Contribution Fairness. Each participant’s income is related to its contribution to the federation.(2)Expectation Fairness of Risk-Taking. The expected risk and time risk between each participant are minimized.(3)Expectation Fairness. The expected and time risks taken from each participant vary as little as possible.
We denote the cost of participant and the contribution, and includes risk-taking, data quality, effort degree, and cost input. And we shall use the procurement auction method [41] to estimate . Further, we define the expected risk and time risk by the following dynamic system: where represents an indicator function is
Here, when , the time series will increase, and shows that participant is the average cost of contribution for the federation. So we have a federally maximized . satisfying fair condition (1), where is the total quantity of runs of the model and represents the contribution made by participant to the federation. We should treat all participants fairly to minimize their expected loss and distribution. Therefore, we introduce the Lyapunov function formulated as follows [42]:
For the convenience of calculation, the root operator is omitted calculation, which will not change the good properties of formula (11). With the change of time, the expected risk deviation of the participants is as follows:
Since is a control variable of the income function of the participant , we extract the formula containing it from formula (12) and get the following result:
The expected loss is the distribution function of and . If is minimized and fair conditions (2) and (3) are satisfied, the expected loss will gradually decrease with time. Let is the target number, which indicates the inequality of expected loss and waiting time between participants, and it should be minimized, where is a federated regularization term to constraint the equilibrium between the two targets. Therefore, the federation is aimed at the following function: and the constraint condition is where means the cost of participants in the current payment budget, which is determined by their expected loss and the time they spend waiting for the reward to be fully paid, and the represents the income that participant received from the federation in round . Let us take the derivative of formula (15) with respect to , and set the first derivative equal to to obtain :
Moreover, by using the second derivatives of (15) with respect to , we obtain the following result:
When the federated model runs in the round , the contribution of the participants to the federation is , and the total compensation of the participants is . If budget is not enough to fully pay all participants’ compensation in round , the federated organization will pay them in installments within a certain period. The federated organization will pay in installments in round according to the following equation:
For the income distribution scheme in Figure 1, when and are equal to , this is a very important condition, and participant does not invest extra cost; then, . After that, participant will use the enhanced Shapley value method to evaluate its contribution based on the federation and allocate the future income. This income allocation method will first consider the participants whose expected risk is not equal to and also consider the contributions of other participants to the federation. According to Figure 1 and the above theoretical analysis, we present the pseudo-code of FL incentive in Algorithm 1.
| ||||||||||||||||||||||||||||||||
4.3. The Enhanced Shapley Value Method
To ensure that all participants are satisfied with the federated income distribution scheme and make the distribution mechanism play an incentive role, all participants can actively contribute to the federated model. The Shapley value method considers that the contribution degree of each participant as ignores the influence of other factors. Therefore, it is necessary to consider the effects of multiple factors on federal income distribution, by taking multiple influence factors as weights, and the enhanced Shapley value method is proposed to measure income distribution [28, 40]. where shows the anticipated income for federated model after improvement and under a federated game condition in the round, is the revenue of the grand coalition, is the income distribution of participant using the Shapley value method in the round of FL. In particular, when , is calculated using formula 6, and indicates the difference between the comprehensive evaluation factor and the average factor introduced by participant , and it refers to the comprehensive factor as an incentive factor in the federated distribution of incomes.
is the sum of the product of the measured values , ,…, of the factors and the corresponding factor weight , ,…,, and . Thus, we have
We solve these weights of influence factors , ,…, with the AHP. It is easy to verify that formula (20) satisfies formulas (4) and (5).
Proof. According to formulas (4) and (23), formula (20) can be proofed as follows: Hence, formula (20) satisfies formula (4). In addition, according to formulas (5) and (23), formula (20) is proofed as the following: Hence, formula (20) satisfies formula (5).
To solve the Shapley value correction factor of the federated income distribution, the AHP is used to assign the importance degree according to the scale of 1-9. The explanation of scale 1-9 is presented in Table 1 [39].
In Table 1, index indicates the intensity of importance on an absolute scale. The judgment matrix in this paper is constructed by the joint score of the expert group, and they score according to the rules in Table 1. These experts have a strong background in economics, computer, and mathematics. Hence, according to the joint score of the expert group, the judgment matrix is obtained.
and satisfied , , and . The matrix is given by normalizing, and according to each column of the matrix , , , ,, , comes from each - row element of matrix and represents the measurement value of factors affecting federated revenue; represents the -th factor affecting the individual’s federated income, and .
Sum matrix is according to row vector, , and achieves the vector . and then, normalizing the vector , . Consequently, the weight value of judgment matrix is obtained.
Since the judgment matrix is established based on the experience and estimation of experts, some differences will appear between the judgment matrix and the real situation, and it needs to test the consistency of the matrix . If the matrix is consistent, its eigenvector can truly reflect the reasons that affect federated income. In this case, we can use the random consistency index (RI) [38, 39] in Table 2. where is the largest eigenvalue of the matrix , the CI represents public consistency index, RI is the average random consistency index, and CR is the consistency ratio. As we refer to Table 2, matrix needs to be satisfied with the consistency.
5. Numerical Results
We assume that there are three banks: bank 1, bank 2, and bank 3, which are taking part in the FL model. The incomes of three banks not participating in FL are , , and . If bank 1 and bank 2 coalition can make income , bank 1 and bank 3 coalition can make income , bank 2 and bank 3 coalition can make income , and banks 1, 2, and 3 coalition can make income . After the three banks unite together, the income distribution of each bank is solved.
5.1. Illustrating Result of the Shapley Value Method
In this subsection, the Shapley value method does not consider any factors to the federated income but distribute the income according to the average distribution method. Therefore, we can calculate the income distribution value of three banks after FL according to the relevant knowledge of cooperative game theory and Equations (6) and (7). The details are shown in Table 3.
Therefore, add the last row of Table 3, and we achieve the federated income value allocated by bank 1 which is the following: ; similarly, the federated income value allocated by bank 2 is as follows: , and the federated income value allocated by bank 3 is as follows: .
5.2. Illustrating Result of the Enhanced Shapley Value Method
In the FL incentive model, the income distribution of the federated bank depends not only on the marginal contribution but also on other factors that affect the total income. In this example, we mainly consider the risk factor, data quality, effort degree, and cost input. For banks with higher risk, higher input data quality, higher level of cooperation, and perhaps higher-cost investment, the weight of benefit distribution should be appropriately increased. According to the enhanced Shapley value model and the following AHP flow chart in Figure 2, experts give scores to the influence factors according to Table 1, and we achieve the judgment matrix . Here, matrix is the weight matrix of reference layer to target layer.
Through the weight matrix , the weight value of reference layer to target layer can be calculated. Similarly, we can obtain the weight matrix of scheme layer to reference layer and the corresponding weight. These weight values of influence factors and the corresponding , and CR values are solved in the following sheet.
Suppose three banks construct judgment matrix according to their own actual situation, and applying to formulas (20)–(31), we achieve the following conclusions: (1)Expert judgment matrix and weight (2)Expert judgment matrix and weight (3)Expert judgment matrix and weight (4)Expert judgment matrix and weight (5)Expert judgment matrix and weight
Consequently, we let and then, using formula (30), we can calculate and According to formula (22), means that the second bank’s enthusiasm in the FL process is significantly lower than the average level of the federated banks and should be punished. Therefore, using formulas (20) and (21) can calculate the income distribution of the three banks in FL as follows:
Through the numerical calculation of the enhanced Shapley method, we achieve the expected result which is the following:
After verification, it is not difficult to find that in the case of the income distribution scheme of the enhanced Shapley value method, the income of the three banks is consistent with the theory.
5.3. Numerical Comparison
In this subsection, we compare the results of income distribution using the Shapley value method with the results of income distribution using the enhanced Shapley value method [40] in Table 4 and simulate their values in Figure 3.
Table 4 indicates that the Shapley value method does not consider any factors and the corresponding weights in the income distribution; however, in the enhanced Shapley value method, we consider the factors that affect income distribution and the corresponding weights. It is not difficult to find out bank 1 has the largest weight, indicating that bank 1 is the most positive in the whole FL process, followed by the third bank, and the second bank is the worst, which should be punished.
From Figure 3, we can clearly see that income allocated by the Shapley value method shows that the first bank and the second bank have the same income. However, relying on the enhanced Shapley value method, it is easy to find that the second bank is the least profitable.
6. Conclusion and Future Work
Federated learning (FL) is a distributed machine learning framework, and how to design an incentive mechanism for federated participants to sustain participate in model training is very important to train an accurate federated model. In this paper, we propose an incentive mechanism using the enhanced Shapley value method, and this incentive mechanism considers four factors that affect the federated income distribution, which can better reflect the fairness of income distribution. Further, the analytic hierarchy process is used to calculate the corresponding weight of four factors. Numerical results verify that the income distribution of all participants can better reflect multiple factor contribution when using the incentive mechanism with the enhanced Shapley value method, which gets a better incentive for federated participants.
In the next step, we will apply the proposed incentive mechanism to solve the problem of participants’ income distribution in FL train models. This incentive mechanism can be applied in WeBank, such as financial institutions, insurance companies, and enterprises, for reducing loan risks and improving economic earning significantly in practice.
Data Availability
The data used to support the finding of this study are included in the article.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. U1836205), the National Natural Science Foundation of China (No. 71961003), the Science and Technology Program of Guizhou Province (No. Guizhou-Science-Contract-Platform-Talent [2020]5017), the Natural Science Foundation of Department of Education of Guizhou Province (Nos. Qian-Education-Contact-Telent [2013]09 and Qian-Education-Contact-KY [2021]140), the 13th Five-Year National Cryptography Development Foundation (No. MMJJ20170129), the Research Project of Guizhou University for Talent Introduction (No. [2020]61), the Cultivation Project of Guizhou University (No. [2019]56), and the Open Fund of Key Laboratory of Advanced Manufacturing Technology, Ministry of Education, GZUAMT2021KF[01].