#### Abstract

This paper studies the impacts of antenna selection algorithms in decode-and-forward (DF) cooperative nonorthogonal multiple access (NOMA) networks, where the secure information from the relay can be overheard by an eavesdropper in the networks. In order to ensure the secure transmission, an optimal antenna selection algorithm is proposed to choose one best relay’s antenna to assist the secure transmission. We study the impact of antenna selection on the system secure communication through deriving the analytical expression of the secrecy outage probability along with the asymptotic expression in the high regime of signal-to-noise ratio (SNR) and main-to-eavesdropper ratio (MER). From the analytical and asymptotic expressions, we find that the system secure performance is highly dependent on the system parameters such as the number of antennas at the relay, SNR, and MER. In particular, the secrecy diversity order of the system is equal to the antenna number, when the interference from the second user is limited.

#### 1. Introduction

In recent years, the wireless data rate has been explosively increasing [1–4], and many wireless services have been emerging, which requires the rapid development of wireless transmission [5–8]. As one of the most promising techniques, nonorthogonal multiple access (NOMA) provides superhigh data rate over the orthogonal multiple access, and hence it has been recognized as one candidate for the next-generation wireless communication networks [9, 10].

Moreover, NOMA has been demonstrated to be compatible with other emerging technologies [11–14]. In particular, the impact of imperfect self-interference cancellation for full-duplex cooperative NOMA is investigated in [12], while authors in [13] study the combination of NOMA and relay selection and results show that the proposed strategy can provide maximal diversity gain. Considering multiple sources, relay sharing scheme is proposed in [14] and results show that the proposed scheme outperforms OMA under perfect or imperfect SIC.

Due to the broadcast nature of wireless transmission, it is of vital importance to guarantee the transmission security from the application layer [15–19] to physical layer [20, 21]. The authors in [22] provided the framework of wiretap channels, which gives a general way to analyze and design the secure physical-layer security. In general, the performances of secure communications is affected by many wireless channel parameters, such as the channel correlation [23–25] and fading characteristics [26–28]. Recently, researchers turned to study the physical-layer security for NOMA communication systems. In [29], the authors proposed the transmission antenna selection for NOMA systems with multiple antennas, in order to ensure the secure communication. The authors in [30] optimized the secrecy sum rate of multiple users, in order to optimize the system design. Moreover, the authors in [31] investigated the downlink NOMA communication systems and proposed a secure beamforming algorithm to enhance the physical-layer security.

To further enhance the physical-layer security of NOMA systems, researchers proposed the spatial selection, such as antenna selection or relay selection [32]. The authors in [33] extended this work into amplify-and-forward (AF) and decode-and-forward (DF) modes and devised a novel relay selection scheme which consists of two stages. The analytical results in this paper depicted that the proposed scheme outperformed the conventional relay selection schemes. The authors in [34] studied the NOMA system over Nakagami-m fading channels and verified that relay selection could enhance the wireless security for the fixed-gain relaying mode. In [13], the authors applied the relay selection technique into NOMA systems with multiple relays and demonstrated that the proposed scheme could achieve the maximum secrecy diversity gain. Moreover, the authors in [14] studied the NOMA systems with multiple sources, and devised a relay sharing scheme which was demonstrated to outperform the conventional OMA with either perfect or imperfect SIC. In [35], the authors studied the NOMA communication systems with AF relay, and derived the analytical outage expression for the cooperative NOMA networks.

However, as to the best of our knowledge, there are no literatures on the effects of the antenna selection algorithms on the secrecy outage probability for cooperative NOMA. Motivated by that, based on the optimal antenna selection algorithm, this paper studies the secrecy outage performance for decode-and-forward (DF) cooperative NOMA networks. (DF relay can decode the wireless signal before forwarding to the destination note, which can improve the link quality. While amplify-and-forward (AF) relay only amplifies its received signal and the link quality can not be improved. Considering the advantages of DF relay scheme compared with AF scheme, only DF cooperative network is studied in this paper.) By using NOMA protocol and with the help of the relay station, the source station transfers messages to two destinations simultaneously. The best antenna of relay is selected to assist the secure transmission from the source to the destinations. The impact of the system parameters is analyzed through deriving the exact analytical expressions on the secrecy outage performance as well as the asymptotic results with large SNR and main-to-eavesdropper ratio (MER). We find that the system secure performance is highly dependent on the system parameters such as the number of antennas at the relay, SNR, and MER. Specifically, the secrecy diversity order of the system is equal to the antenna number, when the interference from the second user is limited. Furthermore, simulation results are provided to validate the theoretical analysis.

The main contributions of this paper are summarized as follows:(1)An optimal antenna selection for cooperative NOMA networks is proposed and the closed-form exact expressions and the asymptotic expressions on the secrecy outage performance are derived.(2)According to the asymptotic analysis, the impacts of the system parameters on the secrecy outage probability are revealed. We find that, in the case of limited interference, the secrecy diversity order of the system is equal to the antenna number

*Notations.* We use the notations and to represent the logarithms with the base of 2 and natural, respectively. Moreover, we use notations and to represent the probability density function and cumulative distribution function of the random variable , respectively. Notation returns the expectation result of a random event . In addition, indicates that the random variable follows a circularly symmetric complex-valued Gaussian distribution of the mean and variance . We use the notation - to denote the link from to .

#### 2. System Model and Selection Algorithm

The system model of two-slot decode-and-forward (DF) cooperative NOMA networks is given in Figure 1. The considered system consists of one source station, a DF relay with antennas, two destinations, and one passive eavesdropper, denoted as , , , and , respectively. The relay station is equipped with antennas, and we use to denote the -th antenna of . It is assumed that perfect channel status information (CSI) of , , and can be obtained by the relay station through dedicated feedback channel [36–39]. Since the eavesdropper does not transmit any signal all the time [40], the relay cannot obtain the CSI of the eavesdropping links. Due to the long distance or severe shadow fading [41–43], there exists no direct link between the source and the destinations. Then with the assistant of the relay station, the source station can transfer messages to two destinations by using the NOMA protocol. Meanwhile, the wireless signal from the relay station may be intercepted by the eavesdropper. Both and are assumed to be equipped with a single antenna and operate in the half-duplex time-division mode. To reduce the information leakage to the eavesdropper, the* best* antenna in is selected to decode the signal and forward to the destinations. As mentioned before, the destinations share the wireless channel through NOMA protocol. In addition, all of the wireless channels are assumed to be quasi-static Rayleigh block fading and statistically independent of each other.

In the first slot, a mixture signal is transmitted from the source station to the selected relay’s antenna. Thus, the received signal at -th antenna of can be expressed aswhere , are the messages transmitted from to and , respectively. is the additive white Gaussian noise (AWGN) received at . We use to denote the total transmission power of the source station and use to represent the instantaneous channel fading coefficient between and . The power allocation factors for and are denoted by , respectively. According to the NOMA protocol [13], low rate and urgent traffic are carried by the first data stream, while the second data stream is used for opportunistic communications with long time tasks. To satisfy the total power requirement, and meet the following constraint: and .

In this paper, we assume that perfect successive interference cancellation (SIC) receiver [44, 45] is used at . Thus, is treated as interference before is decoded. Therefore, the SINRs for and at can be obtained asandrespectively, where is used to denote the channel fading gain between and .

Consequently, the achievable rate for and at can be expressed as follows:andwhere .

Both of the data streams, i.e., and , are decoded by the selected antenna in the second slot and forwarded to the destinations. Meanwhile, the wireless signal from the relay may be overheard by the passive eavesdropper. Given that is selected and the received signal at and can be given as where , are the instantaneous channel fading coefficient of and links, respectively, is the transmission power of , , and denote the AWGN received at and , respectively.

Moreover, it is assumed that SIC receiver is used at both and . That is, is treated as noise when they are trying to decode . Thus, the achievable rates of at and are given as (4)where and denote the channel fading gain of links and -, respectively, and .

Conditioned on that is decoded successfully, then the achievable rates of at and can be expressed asandThus, the secrecy capacity of the relaying system can be written as [46]where .

Based on the theoretical analysis on the secrecy capacity, in order to minimize the secrecy outage probability of cooperative NOMA system, an optimal antenna selection algorithm is proposed in the following section.

The correctly decoding antenna subset is defined in the first step of the proposed algorithm as follows:withwhere and denote the minimum rate requirements for and , respectively.

It is easy to prove that both and are monotone increasing functions with respect to . In the second step of the selection algorithm, since the relay has no CSI of the eavesdropper, the optimal antenna selection algorithm can be performed as follows:

#### 3. Performance Analysis

In this section, the theoretical analysis on the secrecy outage probability for the cooperative NOMA networks will be derived, as well as the asymptotic results. From the analysis, a deep insight on the effects of the system parameters, such as the antenna number and the power allocation factors, on the outage performance is present.

##### 3.1. Exact Expression on Secrecy Outage Probability

In this section, the outage probability of the first step in (13) is analyzed. Given that relay antenna is randomly selected, is defined as the probability that belongs to the decoding antenna subset . By substituting (4) and (5) into (13), we have with

Note that when , no antenna can decode correctly, and the outage occurs definitely. Thus, we only need to consider the case that .

We use to denote the size of subset . Because of the statistical independence between links, the probability that equals can be written as

In particular, the outage probability that the size of equals zero can be given as

We turn to consider the performance of the second step in (15), given that is selected. The scenarios of successful secrecy transmission can be divided into two cases as follows. The first case is that both and can decode successfully, and the secrecy capacity is better than the minimum requirement simultaneously. The probability of the first case can be given as

By applying necessary mathematical derivation and substituting (8), (9), (10), and (11) into (20), we obtainwhere and .

The second case is that cannot decode correctly, but can. In this case, the secrecy capacity equals the achievable rate of at . The probability of the second case can be written asSimilarly, substituting (8), (9), and (10) into (22), we obtain

Using the full probability formula, the outage probability of this step can be calculated aswhere is the size of the decoding antenna subset.

Thus, the secrecy outage probability of the cooperative NOMA system for is

According to the order statistics [47], the CDF of can be expressed asDue to the independence of channel fading gains , the cumulative distribution function can be rewritten as

By using the binomial theorem on (27), we obtain

Using the analysis result in (28), the probability density function of can be expressed as

Since is independent of , the PDF of remains the same with that of ; i.e.,

By applying (29) and (30) on (21) and (23), we havewithand

Using (31) and (33), in (24) and the secrecy outage probability in (25) for the optimal antenna selection can be expressed as in (34). The details about the numerical computation and associated analysis can be found in the literature, such as the works [48–52] or [53–56].

##### 3.2. Asymptotic Outage Probability Analysis

We present the asymptotic analysis in this section to reveal a deep insight on the effects of the system parameters on the outage probability. The asymptotic analysis focuses on the system behavior when the noise can be neglected. In this case, the main contribution or the bottleneck of the system performance can be revealed. Based on the asymptotic analysis, we can propose the optimization scheme on the system parameters. According to the asymptotic results, we can show behavior of the outage performance when both the transmission and the main-to-eavesdropper ratio (MER) are large enough.

Firstly, we give the definition of MER, which denotes the ratio of average fading power of to that of ; i.e.,

Lemma 1. *Given the definition of function , we have the following equation:*

*Proof. *See Appendix A.

Lemma 2. *Given the definition of function , if , we have the following approximation result:*

*Proof. *See Appendix B.

Consider the asymptotic expression of in (31). By using the approximation that , we have Then, by using Lemma 2, we obtain

Similarly, consider the asymptotic expression of in (33), and using the approximation yields

When , the asymptotic result of (18) can be given as

Specifically, when , we have

By substituting (39), (40), and (41) into (34), we can get the asymptotic expression on the outage probability

Particularly, when the system is interference limited, i.e., , we get

From the asymptotic expression in (43) and (44), we can have the following remarks. (a) The asymptotic expression of the secrecy outage probability is jointly determined by the SNR, MER, and the antenna number. (b) For the interference limited system, the secrecy outage probability is only determined by the MER, and the diversity order of the outage probability equals the number of the antennas.

#### 4. Simulation Results

In this section, we provide numerical results to verify the theoretical analysis, which shows the impacts of the system parameters, such as MER (), the number of antennas (), the power allocation factor (), and the rate requirement () on the secrecy outage probability as well as the asymptotic results. The Rayleigh fading channels are used in our simulation cases for all links in the NOMA relaying networks. Without loss of generality, in all simulation cases, we set that the transmission power of the relay is the same as that of the source station.

As a function of SNR (), the effects of on the outage probability is shown in Figure 2 with dB. We set dB, bps/Hz, bps/Hz, and , and the antenna number changes from 1 to 3. From this figure, we can see that large can introduce extra freedom the wireless fading channels, which can improve the outage probability with low SNR region. Also, when SNR grows to be large enough, an error floor occurs for the outage probability, which is determined by the MER and the antenna number. The reason is that when SNR approaches infinity, the bottleneck of the system performance is the interference introduced by NOMA protocol.

Figure 3 shows the effects of on the outage probability as a function of SNR. The value of changes from 1, 5 to 10. From this figure, we find that large means that the average fading power of the first slot is large, which can obviously enlarge the size of the decoding antenna subset . Thus, the secrecy outage probability can be reduced.

Figure 4 shows the effects of power allocation factor on the outage probability as a function of SNR. The value of changes as 0.8, 0.9, and 0.95. Under this configuration, we can see that small can get better system performance. The reason is that, in this case, the bottleneck is the capacity of data stream . Small means more power can be allocated for , which can improve the secrecy outage probability.

The effects of on the outage probability are depicted in Figure 5 with bps/Hz as a function of SNR. The value of changes from 1.0 and 1.5 to 2.0 bps/Hz. Since small can enlarge the size of , large freedom for the second hop can be obtained to improve the secure performance. Thus, we can see from the figure that small can significantly improve the outage performance.

As a function of MER , the analytical SOP in (34) and (44) for cooperative NOMA system is present in Figure 6. Considering the interference limited scenarios, we set dB, bps/Hz, bps/Hz, and , and the number of antennas increases from 1 to 3. From this figure, it is seen that the analytical SOP curves match the simulation SOP curves well in all regions, while, in the high SNR region, the simulation SOP curves converge to that of the asymptotic line. The accuracy of the theoretical analysis can be verified in all the simulation cases. Furthermore, we find that the diversity order of the outage probability is equal to the number of the antennas.

The effects of on the outage probability with is shown in Figure 7. We can see that, in high SNR region, all lines approach an error floor, and small results in worse outage performance. The reason is that denotes the average fading power of the main-to-eavesdropper ration, and large can lower the outage probability of the system.

Figure 8 shows the effects of on the outage probability with dB as a function of SNR. The other parameters are set as the same as Figure 7. The value of is set as 0.1 and 0.4. We can see from the figure that small can lower the outage probability, which is consistent with intuition.

#### 5. Conclusion

In this paper, we propose an optimal antenna selection algorithm to enhance the secure performance for DF cooperative NOMA networks. The exact analytical expressions on the secrecy outage performance are derived, as well as the asymptotic results in the high regime of signal-to-noise ratio (SNR) and main-to-eavesdropper ratio (MER). From the analytical and asymptotic expressions, we find that the system secure performance is highly dependent on the system parameters such as the number of antennas at the relay, SNR, and MER. Specifically, the secrecy diversity order of the system is equal to the antenna number, when the interference from the second user is limited. In the future works, we will incorporate other wireless transmission techniques such as the works in [57–61], in order to further enhance the system performance and ensure the secure communications.

#### Appendix

#### A. Proof of Lemma 1

Considering the definition of , by using the binomial theorem, we can easily find that . Applying variable substitution on , we haveSimilarly, we can prove .

Generally, consider the definition of , and we haveBy applying binomial theorem on , we have

Since , according to (A.3), we obtain . Furthermore, with mathematical induction, we can prove that

Lemma 1 is proved.

#### B. Proof of Lemma 2

*Proof. *For , using the series expansion equation that , we haveBy using Lemma 1, we haveSince , we can ignore the high order items. Thus, can be rewritten asLemma 2 is proved.

#### Data Availability

The data used to support the findings of this study are included within the article.

#### Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

#### Acknowledgments

This work was supported in part by the Scientific Research Project of Education Department of Guangdong, China, under Grant 2017GKTSCX045, in part by the Science and Technology Program of Guangzhou, China, under Grant 201707010389, 201807010103 and 201804010127, in part by the Scientific Research Project of Guangzhou Municipal University under Grant 1201620439, in part by the Qingshanhu Young Scholar Program in GZPYP under Grant 2016Q001, and in part by Comba Research Funds under Grant H2017007, in part by the Guangdong Natural Science Funds for Distinguished Young Scholar under Grant 2014A030306027, in part by the Innovation Team Project of Guangdong Province University under Grant 2016KCXTD017.