Enhancing Physical Layer Security of Cooperative Nonorthogonal Multiple Access Networks via Artificial Noise

Abstract

The massive connectivity requirement and security issues have become major factors restricting the further development of the Internet of Things. Nonorthogonal multiple access (NOMA) can be combined with physical layer security (PLS) to achieve massive connectivity and secure transmission. This article investigates the PLS performance for a downlink communication system over Nakagami-m fading channels, with full-duplex (FD) cooperative NOMA transmission aided by artificial noise (AN). While direct communication is built between the base station and two NOMA users, the strong user is employed as an FD relay as well as the jammer to enhance the PLS of the legitimate transmission in the presence of a passive eavesdropper (Eve). Closed-form analytical expressions in terms of the outage probability of the legitimate users and the intercept probability of Eve are derived to evaluate the security–reliability trade-off (SRT) of the proposed scheme. Monte Carlo simulations are provided to validate the veracity of the theoretic analyses and illustrate that the proposed scheme is superior in terms of SRT to the benchmark schemes in the low SNR region. Furthermore, the results reveal that the SRT performance of the two NOMA users can be enhanced through a proper AN-bearing ratio and power allocation optimization.

1. Introduction

The Internet of Things (IoT) is widely used in smart cities, transportation, automobiles, manufacturing, agriculture, healthcare, wearable devices, etc. As the use of communication equipment grows exponentially, the huge connection demand poses severe challenges to the limited communication resource utilization of 5G mobile networks [1,2]. Nonorthogonal multiple access (NOMA) has been considered as a potential technology for improving spectral efficiency and meeting capacity gain [3]. The core idea of power domain NOMA is to exploit superposition coding and successive interference cancellation (SIC) to detect the desired signals and maintain mutual interference between users within a controllable range through reasonable control of power [4]. At the same time, cooperative communication is an effective method to expand the transmission coverage of wireless networks and improve spectral efficiency [5,6]. Reference [7] for the first time integrated cooperative communication with NOMA technology, which can further improve system efficiency in terms of capacity and reliability. Since then, many researchers have studied the cooperative NOMA (Co-NOMA) system to improve the spectrum usage efficiency under the current situation of increasingly valuable spectrum resources [8,9,10,11,12,13,14,15,16].

In addition to the transmission with high throughput, different security threats are another important restriction on further IoT applications. Besides the information leakage caused by the open electromagnetic environment, NOMA users can multiplex simultaneously within a single resource block, which also makes wireless communication more vulnerable to security and privacy threats. Compared with the traditional upper-layer security technology, physical layer security (PLS) does not need to consume a lot of communication resources or infrastructure to share encryption technology between legitimate users, which is more suitable against wiretapping attacks in IoT networks [17]. Therefore, various PLS technology adopted by the NOMA system, such as relay-aided diversity and cooperative jamming, has gradually attracted widespread attention from academia and is regarded as a research hotspot to achieve massive connectivity and secure transmission [15,16,17,18,19,20,21,22,23,24,25].

1.1. Related Work

In Co-NOMA systems, either relay nodes deployed in a dedicated manner [7,8,9] or strong user relays [10,11,12,13,14] with good channel conditions can be exploited to convey information-bearing signals. In [7], the amplify-and-forward (AF) relaying ability was exploited instead of utilizing the decode-and-forward (DF) relaying technique. The authors of [8] proposed an adaptive uplink and downlink cooperative jamming scheme in the scenario of untrusted relays for security enhancement. Compared with relay-assisted Co-NOMA, user-assisted Co-NOMA saves the infrastructure cost of dedicated relays and provides better use of spatial diversity, making it an attractive solution for cooperation in multi-user NOMA systems. An on/off mechanism was proposed in [11,12] based on whether the cooperative relay link is necessary. In [11], the on/off relay decision was made in terms of the quality of service (QoS) requirement of the cell-edge user. In [12], the intra-cell user forwards the signals of the cell-edge user after SIC only when the signal-to-interference-and-noise ratio (SINR) is greater than a predetermined threshold value. The authors of [13] proposed a dynamic DF-based Co-NOMA scheme with user pairing strategies for spatially random users, which does not consume additional time slots and avoids self-interference in the full duplex (FD) mode. In [14], space–time block code-aided Co-NOMA was exploited to reduce the number of SIC executed, and the performance was evaluated against practical challenges, such as imperfect timing synchronization and imperfect channel state information.

Due to the openness of the transmission medium in the wireless communication network, it is necessary to investigate the security issues in IoT. Relay selection (RS) can be exploited as a promising scheme to enhance PLS performance in the multi-relay wiretap Co-NOMA systems. In [15], a novel priority-based buffer-assisted RS scheme was adopted to seamlessly combine the orthogonal multiple access transmission and NOMA, which significantly improved the data throughput in regions with both high and low signal-to-noise ratios (SNRs). Three RS schemes—optimal dual RS scheme, two-step single RS scheme, and optimal single RS scheme—were proposed, and the secrecy outage performance was analyzed in [16]. The authors of [18] proposed DF/AF-based optimal RS to improve reliability and security in Co-NOMA networks in the presence of untrusted users. In [19], a cognitive collaboration method based on the combination of legitimate links and eavesdropping links was proposed in Co-NOMA cognitive wireless networks. It was also concluded in [16,19] that increasing the transmit power and the number of relays could further improve the security. A new cooperative relaying scheme based on a partial NOMA system was proposed in [20]. In order to further improve the PLS of the proposed system, four cooperative schemes were designed in the consideration of the power allocations, channel conditions, decoding principles, imperfect SIC and eavesdropper (Eve) locations.

Adopting artificial noise (AN) or jamming signals to degrade Eve’s channel is also a popular PLS technique. For better overall system performance when using AN technology, two RS schemes—optimal relay–receiver pair selection and suboptimal RS—were combined in [21]. In the multi-antenna Co-NOMA-based IoT network [22], AN is generated from both the base station (BS) and the strong IoT node for achieving secure transmission of the system. Compared with half-duplex (HD), FD technology presents an opportunity to enable nodes to complete simultaneous receiving and transmitting on the given frequency band and double or even triple the data rate of the system, which not only improves spectral efficiency but also enhances PLS performance. Reference [23] introduced a novel communication scheme that combined NOMA, beamforming, simultaneous wireless information and power transfer, and FD techniques to achieve a higher sum rate in the downlink commutation system. Additionally, three different cases of self-interference (SI) signal in FD mode were adopted to investigate the impact of SI cancellation on system secure performance. In [24], one source established communication directly with a multi-antenna near-user, while the source used multiple FD relays to communicate with the distant user in the case of passive Eves. To solve this wiretapping problem, a new FD-based two-stage AN scheme was proposed, where AN is not only an interference signal to prevent eavesdropping but also acts as a secret key to enhance the confidentiality of the distant user. The authors of [25] carried out the secrecy outage performance and secrecy diversity order analysis in large-scale NOMA with the aid of FD and AN.

As mentioned above, we find that (1) Co-NOMA technology assisted by FD can improve the spectral efficiency and system throughput; (2) based on the broadcast nature of wireless transmission, the above Co-NOMA system is vulnerable to be wiretapped from different Eve, and there is still a paucity of contributions on enhancing PLS performance in the Co-NOMA network; and (3) all of the above works focused on the system models over the Rayleigh fading channels. Additionally, there is a lack of performance evaluation metrics, i.e., considering outage probability (OP) and intercept probability (IP) in terms of the Co-NOMA system. Furthermore, to the best of our knowledge, there is little research on PLS under wiretap Co-NOMA network over the Nakagami-m fading channel. Motivated by that mentioned above, it is of great importance to investigate the security issues of a Co-NOMA network and design an efficient PLS enhancement scheme.

1.2. Contributions and Article Structure

In this article, we specifically consider the scenario of a downlink Co-NOMA system in the presence of a passive Eve, where a BS communicates directly with two NOMA users. We consider a more broadly representative wireless channel model with the app cation value: the Nakagami-m fading channel, which can better describe channel fading under actual conditions. In such a system, the weak user may not be able to successfully decode the confidential information transmitted from the BS. Different from the HD mode exploited in [19,22], we employ the user relay aided by FD mode to further improve spectral efficiency. Therefore, based on the concept of Co-NOMA, the strong user decodes the information signal of the weak user after performing SIC successfully, and then re-encodes and retransmits the decoded information to the weak user. Moreover, we further achieve secure communication with legitimate users by exploiting the power-splitting technique. At the strong user, some of the power is allocated to the transmission of the information signal, and the rest is allocated to the generation of AN. The authors in [25] assumed that AN is known to all legitimate users and remains unknown to Eve. In our article, AN is only known to the strong user, while unknown to the weak user and Eve, providing a lower bound of performance.

The key contributions of this article are summarized as follows:

• We exploit both FD and AN techniques to improve the PLS performance of the Co-NOMA system, where the BS communicates directly with the strong user, while the weak user receives signals from both the BS and the strong user. In addition, the impact of the residual self-interference of FD on the system is also taken into account.
• The closed-form expressions for the exact OP (reliability) and IP (security) are derived to demonstrate the security–reliability trade-off (SRT) of the proposed AN-aided Co-NOMA scheme. In order to further investigate the superiority of the proposed scheme, the SRT performance of Co-NOMA without AN and non-cooperative NOMA (Nco-NOMA) schemes are also compared.
• We conduct extensive numerical simulations to verify the derivations in different system settings. It shows that Co-NOMA with AN can offer better SRT performance on the weak user over benchmark methods in the low SNR region, while not degrading the performance of the strong user under the condition that the co-channel interference can be sufficiently suppressed. Furthermore, optimal performance can be achieved by adjusting the power-sharing coefficients between the BS and the strong user as well as the ratio between the AN bearing and information bearing at the strong user.

The rest of this article is organized as follows. In Section 2, we present the system model and propose the AN scheme. Section 3 analyzes the SRT performance of the proposed AN-aided Co-NOMA scheme and builds the comparison with two other schemes. Numerical results are articulated in detail to validate the derived theoretical results in Section 4. Finally, Section 5 draws conclusions and suggests future works.

2. System Model

The considered AN-aided FD Co-NOMA system is illustrated in Figure 1, which consists of one BS, two legitimate users and a passive Eve. With the purpose of further improving the SRT performance of the weak user $U_2$, the strong user $U_1$ acts as a user relay, employing the decode-and-forward (DF) strategy. Therefore, two antennas are equipped at $U_1$, where one antenna receives signals from the BS and the other antenna forwards successfully decoded $x_2$ to $U_2$. In addition, since HD technology requires additional time slot forwarding information, $U_1$ works in the FD mode to send $x_2$ and AN to $U_2$ while receiving signals from the BS at the same time. The BS and $U_2$ only need to send and receive messages, respectively, and thus, only a single antenna is equipped and works in HD mode.

At the beginning of each time slot, the BS and $U_1$ are respectively equipped with $P_s$ and $P_1$ power levels, which satisfy $P_s+P_1=P$, and P is the total power of the system. We assume that a complex channel estimation algorithm is used with enough training information to obtain full channel state information (CSI) at the receivers. The estimation errors caused by imperfect CSI [26] can be left for future investigations. Based on the broadcast nature of wireless channels, Eve is able to intercept messages from the BS and the user relay. All channels are assumed to remain constant in one fading block but change independently between different fading blocks. Specifically, the channels coefficients from BS to $U_1$, $U_2$ and Eve, $U_1$ to $U_2$ and Eve are represented by $h_{\theta },\theta \in \left\{1,2,se,r,u_{1}e\right\}$, respectively. It is assumed that all of the above channels undergo independent block Nakagami-m fading. Without loss of generality, the zero mean additive white Gaussian noise (AWGN) at $U_i,i\in \left\{1,2\right\}$ and Eve is denoted by $n_i$ and $n_e$, with variances $N_0$ and $N_e$, respectively.

2.1. Co-NOMA with AN

Referring to [10,27], the total number of time slots is n, and we assume that n is large enough, i.e., $n\to \infty$. A communication slot is defined as a time block during which the transmission of the message from the BS to two NOMA users is completed. We assume that the time slot boundaries are fully synchronized across all users. During the t-th time slot, the BS broadcasts superimpose signal $\sqrt{a_1{P}_s}{x}_1+\sqrt{a_2{P}_s}{x}_2$ to $U_1$ and $U_2$, where $x_i$ is a normalized signal with unit power received by $U_i$. According to the fundamental principle of NOMA, more power is distributed to the user with poor channels to maintain better fairness and QoS. Thus, we define $a_i$ as the power allocation coefficient, which satisfies $a_2>a_1>0$ and $a_1+a_2=1$. With the purpose of further enhancing the SRT performance of $U_2$, $U_1$ forwards the re-encoded weak user signal $x_2$ with power $\xi P_1$, where $\xi \in \left[0,1\right]$ is the information-bearing ratio. At the same time, the remaining power $\left(1-\xi \right)P_1$ is exploited to generate AN $x_j$ to confuse Eve. When $\xi =1$, it turns into the scenario of Co-NOMA without AN, where $U_1$ forwards the signal of $x_j$ with full power $P_1$. Similar to [28], $x_j$ is a normalized signal with unit power generated by a pseudo-random sequences generator. Throughout this article, a pessimistic assumption is made that AN is only known to $U_1$ to provide the lower bounds of performance, which will degrade the channels of the eavesdropping link as well as the relay link. Due to computational limitations and practical non-ideality, $U_1$ is unable to completely eliminate residual loop self-interference (LI) when working in FD mode [27]. Therefore, $U_1$ receives the superimposed mixed signal and LI signal simultaneously. The LI cancellation coefficient is defined as $\varphi$, and $0<\varphi \le 1$ corresponds to the cancellation level for the LI signal. Thus, the received signals at $U_1$ can be expressed as

$$\begin{array}{c}{y}_1\left(t\right)=h_1\left(\sqrt{a_1{P}_s}{x}_1\left(t\right)+\sqrt{a_2{P}_s}{x}_2\left(t\right)\right)+h_{LI}\left(\sqrt{\varphi \xi P_1}{x}_2(t-\tau )+\sqrt{\varphi \left(1-\xi \right)P_1}{x}_j\left(t\right)\right)+n_1\left(t\right),\end{array}$$

(1)

where $\tau$ is the processing delay at $U_1$, which satisfies the relation $t\ge \tau$. Referring [29] chapter 6.2, based on the principle of the downlink NOMA system, the BS sends separate (i.e., independent) information to multiple receivers (the users). Therefore, the noncorrelation between signals $x_i$ and $n_i$ is assumed in the following parts.

Based on the SIC technique [3], $U_1$ first decodes the signal of $x_2$ by treating $x_1$ as interference. After reconstructing and subtracting the extracted signal $x_2$ from the superimposed mixed signals and then decoding its own signal $x_1$, it is assumed that perfect SIC is performed at at $U_1$, and thus the received SINR at $U_1$ to decode $x_1$ and $x_2$ are respectively given by

$${\gamma }_{S,1}^2=\frac{a_2{\rho }_s{\left|h_1\right|}^2}{a_1{\rho }_s{\left|h_1\right|}^2+\varphi {\rho }_1{\left|h_{LI}\right|}^2+1},$$

(2)

$${\gamma }_{S,1}^1=\frac{a_1{\rho }_s{\left|h_1\right|}^2}{\varphi {\rho }_1{\left|h_{LI}\right|}^2+1},$$

(3)

where ${\rho }_s=\frac{P_s}{N_0}$, ${\rho }_1=\frac{P_1}{N_0}$. $U_1$ forwards the signal of $x_2$ to $U_2$ after successfully extracting the signal. Therefore, the received signal from the direct link and the relay link at $U_2$ can be respectively represented by

$$y_{S,2}\left(t\right)=h_2\left(\sqrt{a_1{P}_s}{x}_1\left(t\right)+\sqrt{a_2{P}_s}{x}_2\left(t\right)\right)+n_{s,2}\left(t\right),$$

(4)

$$y_{1,2}\left(t\right)=h_r\left(\sqrt{\xi P_1}{x}_2(t-\tau )+\sqrt{\left(1-\xi \right)P_1}{x}_j\left(t\right)\right)+n_{1,2}\left(t\right).$$

(5)

To simplify the calculation, it is assumed that $n_{s,2}=n_{1,2}=n_2$. $U_2$ is assigned more power, which can decode its own signal by treating the signal of $x_1$ as noise. Due to the existence of residue interference between the direct link and relay link, the received SINR at $U_2$ to decode $x_2$ corresponding to the direct link and the relay link can be respectively written as

$${\gamma }_{S,2}^2=\frac{a_2{\rho }_s{\left|h_2\right|}^2}{a_1{\rho }_s{\left|h_2\right|}^2+\kappa {\rho }_1{\left|h_r\right|}^2+1},$$

(6)

$${\gamma }_{1,2}^2=\frac{\xi {\rho }_1{\left|h_r\right|}^2}{\left(1-\xi \right){\rho }_1{\left|h_r\right|}^2+\kappa {\rho }_s{\left|h_2\right|}^2+1},$$

(7)

where $\kappa$ denotes the impact levels of residue interference.

As described in [30,31], the transmitted signals of the direct link from the BS and the relay link from $U_1$ exist in a temporal separation. Referring to [30,32], we assume that the signals from $U_1$ and the BS can be fully distinguished by $U_2$ to obtain theoretical results for the actual NOMA system. This assumption is consistent with the ideal operation of rake receivers widely used in wireless systems, as well as a theoretical method called linear relay introduced in [33]. Here, we provide the upper bounds of (6) and (7) in the following sections. Then, (6) and (7) can be respectively rewritten as

$${\gamma }_{S,2}^2=\frac{a_2{\rho }_s{\left|h_2\right|}^2}{a_1{\rho }_s{\left|h_2\right|}^2+1},$$

(8)

$${\gamma }_{1,2}^2=\frac{\xi {\rho }_1{\left|h_r\right|}^2}{\left(1-\xi \right){\rho }_1{\left|h_r\right|}^2+1}.$$

(9)

$U_2$ applies the selection combining (SC) technique to achieve enhanced performance while choosing the signal corresponding to the largest SINR as the output signal. Therefore, the output SINR after SC at $U_2$ is given as

$${\gamma }_{U_2}^{SC}=max\left\{min\left({\gamma }_{S,1}^2,{\gamma }_{1,2}^2\right),{\gamma }_{S,2}^2\right\}.$$

(10)

Similar to $U_2$, the received signal for the direct link and the relay link at Eve can be represented by

$$y_{S,E}\left(t\right)=h_{se}\left(\sqrt{a_1{P}_s}{x}_1\left(t\right)+\sqrt{a_2{P}_s}{x}_2\left(t\right)\right)+n_{s,e}\left(t\right),$$

(11)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill y_{1,E}\left(t\right)=h_{u_{1}e}\left(\sqrt{\xi P_1}{x}_2(t-\tau )+\sqrt{\left(1-\xi \right)P_1}{x}_j\left(t\right)\right)+n_{1,e}\left(t\right).\end{array}\end{array}$$

(12)

Similar to [19], we consider the case with a weaker intercepting ability of Eve in the NOMA network. Specifically, Eve cannot unambiguously distinguish between multi-user data streams due to the inter-user interference caused by NOMA technology. Therefore, Eve will be interfered by $x_2$ (or $x_1$) when it tries to decode $x_1$ (or $x_2$) since Eve can only wiretap the signal of $x_1$ from the BS, while wiretapping the signal of $x_2$ from both the BS and $U_1$. We assume that $n_{s,e}=n_{1,e}=n_e$ to simplify the analysis. Therefore, the wiretapped SINR at Eve for decoding $x_1$ and $x_2$ is given by

$${\gamma }_{S,E}^1=\frac{a_1{\rho }_{se}{\left|h_{se}\right|}^2}{a_2{\rho }_{se}{\left|h_{se}\right|}^2+1},$$

(13)

$${\gamma }_{S,E}^2=\frac{a_2{\rho }_{se}{\left|h_{se}\right|}^2}{a_1{\rho }_{se}{\left|h_{se}\right|}^2+1},$$

(14)

$${\gamma }_{1,E}^2=\frac{\xi {\rho }_{u_{1}e}{\left|h_{u_{1}e}\right|}^2}{(1-\xi ){\rho }_{u_{1}e}{\left|h_{u_{1}e}\right|}^2+1},$$

(15)

where ${\rho }_{se}=\frac{P_s}{N_e}$, ${\rho }_{u_{1}e}=\frac{P_1}{N_e}$. Under the constraints of low complexity, Eve applies SC to pick up the signal corresponding to the largest SINR as the effective signal. Therefore, the received SINR at Eve for decoding $x_2$ can be defined as

$${\gamma }_E^{SC}=max\left\{{\gamma }_{S,E}^2,{\gamma }_{1,E}^2\right\}.$$

(16)

2.2. Nco-NOMA

In the Nco-NOMA system, only two NOMA users can obtain information from the BS.Therefore, the received signal at $U_i$ and Eve can be expressed as

$$y_i\left(t\right)=h_i\left(\sqrt{a_1{P}_s}{x}_1\left(t\right)+\sqrt{a_2{P}_s}{x}_2\left(t\right)\right)+n_i\left(t\right),$$

(17)

$$y_e\left(t\right)=h_{se}\left(\sqrt{a_1{P}_s}{x}_1\left(t\right)+\sqrt{a_2{P}_s}{x}_2\left(t\right)\right)+n_e\left(t\right),$$

(18)

where $h_i$ is the channel gain from the BS to $U_i$. Therefore, the received SINR at $U_i$ to decode $x_i$ can be given by

$${\gamma }_{S,1}^2=\frac{a_2{\rho }_s{\left|h_1\right|}^2}{a_1{\rho }_s{\left|h_1\right|}^2+1},$$

(19)

$${\gamma }_{S,1}^1=a_1{\rho }_s{\left|h_1\right|}^2,$$

(20)

$${\gamma }_{S,2}^2=\frac{a_2{\rho }_s{\left|h_2\right|}^2}{a_1{\rho }_s{\left|h_2\right|}^2+1}.$$

(21)

Therefore, the wiretapped SINR of $x_i$ can be written as

$${\gamma }_{S,E}^1=\frac{a_1{\rho }_{se}{\left|h_{se}\right|}^2}{a_2{\rho }_{se}{\left|h_{se}\right|}^2+1},$$

(22)

$${\gamma }_{S,E}^2=\frac{a_2{\rho }_{se}{\left|h_{se}\right|}^2}{a_1{\rho }_{se}{\left|h_{se}\right|}^2+1}.$$

(23)

3. Performance Analysis

In this section, through deriving exact closed-form expressions for the outage and intercept probabilities, we evaluate the SRT performance under three scenarios: AN-aided Co-NOMA system, Co-NOMA system and Nco-NOMA system.

3.1. Co-NOMA with AN

3.1.1. Outage Probabilities

According to NOMA principle, the outage events at $U_1$ can be explained as follows: $U_1$ cannot decode $x_1$ or $x_2$. The outage events at $U_2$ can be explained as follows: the received SINR after SC at $U_2$ in one time slot is less than its target SINR. Based on the above description, the OP of $U_1$ and $U_2$ are defined as

$$P_{out}^{U_1}=1-P_r\left({\gamma }_{S,1}^2>{\gamma }_o^2,{\gamma }_{S,1}^1>{\gamma }_o^1\right),$$

(24)

$$P_{out}^{U_2}=P_r\left({\gamma }_{U_2}^{SC}\right.\left.<{\gamma }_o^2\right),$$

(25)

where ${\gamma }_o^i=2^{R_o^i}-1$, $R_o^i$ is the target transmission rate threshold to decode $x_i$. Noting that the channel coefficients $\left|h_{\theta }\right|$ follow independent and identically distributed Nakagami-m distribution, ${\left|h_{\theta }\right|}^2$ therefore obeys gamma distribution. Therefore, similar to [34,35], the probability density function (PDF) and cumulative distribution function of the square of the Nakagami random variable can be written as

$$f_{{\left|h_{\theta }\right|}^2}\left(x\right)={\left(\frac{m_{\theta }}{{\Omega }_{\theta }}\right)}^{m_{\theta }}\frac{x^{m_{\theta }-1}}{\Gamma \left(m_{\theta }\right)}exp\left(-x\frac{m_{\theta }}{{\Omega }_{\theta }}\right),$$

(26)

$$\begin{array}{c}\hfill F_{{\left|h_{\theta }\right|}^2}\left(x\right)=1-\frac{1}{\Gamma \left(m_{\theta }\right)}\Gamma \left(m_{\theta },x\frac{m_{\theta }}{{\Omega }_{\theta }}\right)=1-exp\left(-x\frac{m_{\theta }}{{\Omega }_{\theta }}\right)\sum _{k=0}^{m_{\theta }-1}{\left(x\frac{m_{\theta }}{{\Omega }_{\theta }}\right)}^k\frac{1}{k!},\end{array}$$

(27)

where $m_{\theta }$ is the shape parameter of the Nakagami-m distribution, which is the degree of signal fading caused by multipath interference and scattering processes, ${\Omega }_{\theta }$ denotes the average power of the multipath scattering component, $\Gamma \left(.\right),\Gamma \left(.,.\right)$ respectively represent the gamma function and incomplete gamma function similar to [36]. Referring to [34], we assume that LI is a variable with Rayleigh distribution $h_{LI}$, and ${\Omega }_{LI}$ is the corresponding mean power. Therefore, the ${\left|h_{LI}\right|}^2$ random variable follows independently exponentially distributed random variables, that is

$$f_{{\left|h_{LI}\right|}^2}\left(x\right)=\frac{1}{{\Omega }_{\mathrm{LI}}}exp\left(-\frac{x}{{\Omega }_{\mathrm{LI}}}\right).$$

(28)

After some mathematical calculations, the OP of $U_1$ and $U_2$ can be presented as follows:

Theorem 1.

The closed-form expression for the OP of $U_1$ is given by

$$\begin{array}{c}\hfill P_{out}^{U_1}=1-\frac{1}{{\Omega }_{\mathrm{LI}}}exp\left(\frac{1}{\varphi {\rho }_1{\Omega }_{\mathrm{LI}}}\right)\sum _{k=0}^{m_1-1}{\left(\frac{m_1{\phi }_1}{{\Omega }_1}\right)}^k\frac{1}{k!}\frac{{\left(\varphi {\rho }_1\right)}^k\Gamma \left(k+1,\frac{m_1{\phi }_1}{{\Omega }_1}+\frac{1}{\varphi {\rho }_1{\Omega }_{\mathrm{LI}}}\right)}{{\left(\frac{m_1{\phi }_1\varphi {\rho }_1}{{\Omega }_1}+\frac{1}{{\Omega }_{\mathrm{LI}}}\right)}^{k+1}},\end{array}$$

(29)

where ${\phi }_1=max\left({\alpha }_1,{\beta }_1\right)$, ${\alpha }_1=\frac{{\gamma }_o^2}{{\rho }_s\left(a_2-a_1{\gamma }_o^2\right)}$, ${\beta }_1=\frac{{\gamma }_o^1}{a_1{\rho }_s}$. It is easily noted that $P_{out}^{U_1}=1$ when ${\gamma }_o^2\ge \frac{a_2}{a_1}$. Therefore, in the rest of this section, we only investigate the performance on the condition of ${\gamma }_o^2<\frac{a_2}{a_1}$.

Proof. 

Please refer to Appendix A. □

Theorem 2.

The closed-form expression for OP of $U_2$ is written as

$$P_{out}^{U_2}=\left(1-Q_1\times Q_2\right)\times Q_3,$$

(30)

with

$$\begin{array}{c}\hfill Q_1=\frac{1}{{\Omega }_{\mathrm{LI}}}exp\left(\frac{1}{\varphi {\rho }_1{\Omega }_{\mathrm{LI}}}\right)\sum _{k=0}^{m_1-1}{\left(\frac{m_1{\alpha }_1}{{\Omega }_1}\right)}^k\frac{1}{k!}\frac{{\left(\varphi {\rho }_1\right)}^k\Gamma \left(k+1,\frac{m_1{\alpha }_1}{{\Omega }_1}+\frac{1}{\varphi {\rho }_1{\Omega }_{\mathrm{LI}}}\right)}{{\left(\frac{m_1{\alpha }_1\varphi {\rho }_1}{{\Omega }_1}+\frac{1}{{\Omega }_{\mathrm{LI}}}\right)}^{k+1}},\end{array}$$

(31)

$$\begin{array}{c}\hfill Q_2=\frac{1}{\Gamma \left(m_r\right)}\Gamma \left(m_r,\frac{m_r{\gamma }_o^2}{{\Omega }_r\left(\xi {\rho }_1-\left(1-\xi \right){\rho }_1{\gamma }_o^2\right)}\right),\end{array}$$

(32)

$$Q_3=1-\frac{1}{\Gamma \left(m_2\right)}\Gamma \left(m_2,\frac{m_2}{{\Omega }_2}{\alpha }_1\right).$$

(33)

In the rest of this section, we only analyze the performance on the condition of ${\gamma }_o^2<\frac{\xi }{1-\xi }$; else, $P_{out}^{U_2}=1$.

Proof. 

Please refer to Appendix B. □

3.1.2. Intercept Probabilities

Eve can wiretap the confidential messages transmitted on legitimate links when the capacity of the eavesdropping channel is greater than the rate difference $\left(R_o^i-R_s^i\right)$, where $R_s^i$ presents the secrecy rate that needs to be encoded by a secrecy encoder [37]. Therefore, the IP of $U_1$ and $U_2$ are given by

$$P_{int}^{U_1}=P_r\left({\gamma }_{S,E}^1>{\gamma }_E^1\right),$$

(34)

$$P_{int}^{U_2}=P_r\left({\gamma }_E^{SC}>{\gamma }_E^2\right),$$

(35)

where ${\gamma }_E^i=2^{R_o^i-R_s^i}-1$. Therefore, the closed-form expressions for IP of $U_1$ and $U_2$ can be expressed as

$$\begin{array}{cc}\hfill P_{int}^{U_1}& =P_r\left({\left|h_{se}\right|}^2>\frac{{\gamma }_E^1}{{\rho }_{se}\left(a_1-a_2{\gamma }_E^1\right)}\right)\hfill \\ \hfill & =\frac{1}{\Gamma \left(m_{se}\right)}\Gamma \left(m_{se},\frac{m_{se}{\gamma }_E^1}{{\Omega }_{se}{\rho }_{se}\left(a_1-a_2{\gamma }_E^1\right)}\right),\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill P_{int}^{U_2}& =1-P_r\left({\left|h_{se}\right|}^2\le \frac{{\gamma }_E^2}{{\rho }_{se}\left(a_2-a_1{\gamma }_E^2\right)}\right)\times P_r\left({\left|h_{u_{1}e}\right|}^2\le \frac{{\gamma }_E^2}{{\rho }_{u_{1}e}\left(\xi -\left(1-\xi \right){\gamma }_E^2\right)}\right)\hfill \\ \hfill & =1-\left(1-\frac{1}{\Gamma \left(m_{se}\right)}\Gamma \left(m_{se},\frac{m_{se}{\gamma }_E^2}{{\Omega }_{se}{\rho }_{se}\left(a_2-a_1{\gamma }_E^2\right)}\right)\right)\hfill \\ \hfill & \times \left(1-\frac{1}{\Gamma \left(m_{u_{1}e}\right)}\Gamma \left(m_{u_{1}e},\frac{m_{u_{1}e}{\gamma }_E^2}{{\Omega }_{u_{1}e}{\rho }_{u_{1}e}\left(\xi -\left(1-\xi \right){\gamma }_E^2\right)}\right)\right).\hfill \end{array}$$

(37)

In the remainder of this section, we only analyze the performance on the condition of ${\gamma }_E^1<\frac{a_1}{a_2}$, $\frac{{\gamma }_E^2}{1+{\gamma }_E^2}<\xi \le 1$ and ${\gamma }_E^2<\frac{a_2}{a_1}$, else $P_{int}^{U_1}=P_{int}^{U_2}=0$.

3.2. Nco-NOMA

3.2.1. Outage Probabilities

The OP of $U_1$ and $U_2$ can be given by

$$\begin{array}{c}\hfill P_{out}^{U_1}=1-P_r\left({\gamma }_{S,1}^2>{\gamma }_o^2,{\gamma }_{S,1}^1>{\gamma }_o^1\right)=1-\frac{1}{\Gamma \left(m_1\right)}\Gamma \left(m_1,\frac{m_1{\phi }_1}{{\Omega }_1}\right),\end{array}$$

(38)

$$\begin{array}{c}\hfill P_{out}^{U_2}=P_r\left({\gamma }_{S,2}^2<{\gamma }_o^2\right)=1-\frac{1}{\Gamma \left(m_2\right)}\Gamma \left(m_2,\frac{m_2{\alpha }_1}{{\Omega }_2}\right).\end{array}$$

(39)

3.2.2. Intercept Probabilities

The IPs of $U_1$ and $U_2$ are expressed by

$$\begin{array}{c}\hfill P_{int}^{U_1}=P_r\left({\gamma }_{S,E}^1>{\gamma }_E^1\right)=\frac{1}{\Gamma \left(m_{se}\right)}\Gamma \left(m_{se},\frac{m_{se}{\gamma }_E^1}{{\Omega }_{se}{\rho }_{se}\left(a_1-a_2{\gamma }_E^1\right)}\right),\end{array}$$

(40)

$$\begin{array}{c}\hfill P_{int}^{U_2}=P_r\left({\gamma }_{S,E}^2>{\gamma }_E^2\right)=\frac{1}{\Gamma \left(m_{se}\right)}\Gamma \left(m_{se},\frac{m_{se}{\gamma }_E^2}{{\Omega }_{se}{\rho }_{se}\left(a_2-a_1{\gamma }_E^2\right)}\right).\end{array}$$

(41)

In addition, in order to better describe the network trade-off performance of security and reliability, SRT is defined as the probability that the transmission of information in the network will neither be interrupted nor intercepted [38]. Therefore, the SRT of $U_2$ can be written as

$$\begin{array}{c}\hfill SR{T}^{U_2}=(1-P_{{}_{out}}^{U_2})(1-P_{{}_{int}}^{U_2}).\end{array}$$

(42)

Obviously, the larger the SRT, the better the trade-off performance of the user, while the smaller the SRT, the worse the trade-off performance of the user.

4. Numerical Results

Monte Carlo simulations are provided in this section to validate the accuracy of the numerical results under different settings. Referring to [25,39] and without loss of generality, we set $\rho =\frac{P}{N_0}$, $m_1=m_2=m_r=2$, ${\Omega }_1={\Omega }_r=2$, ${\Omega }_2=1$, ${\Omega }_{LI}=-15$ dB, $a_1=0.4$, $R_o^1=R_o^2=1$, $R_s^1=R_s^2=0.4$, ${\rho }_s={\rho }_1=0.5\rho$, ${\rho }_{se}={\rho }_{1e}=0.1\rho$, $\xi =0.6$, $\varphi =0.1$, unless otherwise stated.

Figure 2 investigates the SRT of $U_1$ under different parameters. The simulation values are the same as the corresponding analytical curves, which confirms the accuracy of the theoretical derivations. Since Eve can only eavesdrop on the signal of $x_1$ from the BS, adding the relay link will have no effect on IP. Therefore, we mainly evaluate the outage performance of $U_1$. As can be seen from Figure 2a, the trend of OP in the scenario of Co-NOMA with AN and Nco-NOMA is consistent with the increasing of $\rho$. As illustrated in Section 2, the effective SINR at $U_1$ decoding $x_1$ and $x_2$ is in direct proportion $\rho$, which affirms that the OP should decrease with the increasing $\rho$. At high SNR region, the effective SINR at $U_1$ for Nco-NOMA continues to increase with the increasing of $\rho$, while approaching constants in Co-NOMA with AN. As a result, the performance of the $U_1$ under the Nco-NOMA system continues to increase, while tending to be fixed under Co-NOMA with AN. Compared with Nco-NOMA, in the case of Co-NOMA with AN, the residual self-interference of $U_1$ brings about the decrease in efficient SINR, which increases the OP of $U_1$. With the values of LI decreasing, the performance gap between Co-NOMA with AN and Nco-NOMA diminishes gradually. Therefore, when the LI is suppressed to be sufficiently small, the performance penalty of $U_1$ caused by FD can be ignored.

Figure 2b plots the OP of $U_1$ versus the increasing power allocation factor $a_1$ in the case of Co-NOMA with AN. The increase in $a_1$ has an opposite effect on the effective SINR at $U_1$ decoding $x_1$ and $x_2$. At the lower region of $a_1$, the OP of $U_1$ is mainly dominated by ${\gamma }_{S,1}^1$, which results in a downward trend of OP. However, at a higher region of $a_1$, the OP of $U_1$ is mainly dominated by ${\gamma }_{S,1}^2$, which brings about the increasing of OP. Consequently, based on the nonconvex nature of OP, there exists an optimal power allocation to achieve the optimal outage performance of $U_1$. In addition, it is obvious that the increasing residual self-interference has a significant effect on the outage performance due to the FD operation of $U_1$.

Figure 2c depicts the OP of $U_1$ versus ${\rho }_s$ and $\varphi$. One can observe that as ${\rho }_s$ increases, the OP of $U_1$ declines gradually and eventually reaches saturation. As $\varphi$ decreases, the residual self-interference becomes weaker, which brings about the improvement of OP performance. Moreover, at the lower value of ${\rho }_s$, the transmission power of $U_1$ becomes stronger, and thus the change of $\varphi$ has a greater influence on OP. In Figure 2d, as the Nakagami-fade parameter $m_{se}$ increases, the channels between the BS to Eve and $U_1$ to Eve become stable, while performance improvements are more pronounced in areas with high transmit power. Furthermore, it is obvious that the IP of $U_1$ increases with the increasing of the given secrecy rate threshold $R_s^1$.

Figure 3 presents the SRT of $U_2$ versus with $\rho$ and $a_2$ for different values under three scenarios, respectively. The simulation values are consistent with the theoretical values, which validates the correctness of the theoretical derivations. In Figure 3a, it is obviously observed that compared with Nco-NOMA, the outage performance of $U_2$ is significantly improved with the assistance of the user relay, and thus Co-NOMA with AN and Co-NOMA schemes outperform those of the Nco-NOMA scheme. Furthermore, due to the decrease in the information-bearing ratio, the outage performance of Co-NOMA with AN is slightly inferior to Co-NOMA. In the case of Co-NOMA, Eve applies SC to eavesdrop $x_2$ from both the BS and $U_1$. At the same time, the effective SINR to decode $x_2$ at Eve becomes larger with the increasing of $\xi$. Therefore, compared to Co-NOMA, Co-NOMA with AN can notably improve security. As $\rho$ increases, the effective SINR to decode $x_2$ at $U_2$ or Eve continues to increase. Therefore, the strength of signal $x_2$ for three scenarios is enhanced simultaneously, and thus brings about the decrease in OP with the increase in $\rho$, while their corresponding IP increases. This result indicates that there exists a trade-off between reliability (OP) and security (IP) in the case of three scenarios. Therefore, SRT is analyzed in the following Figure 3b. It can be observed that when $\rho$ is less than 7 dB, the SRT of Co-NOMA is optimal. This is because the aid of the relay link greatly reduces the OP. With the increase in $\rho$, the security is sacrificed at the same time, which is necessary to exploit AN to improve security. When $\rho$ is greater than 11 dB, the case of Co-NOMA with AN is slightly inferior to Nco-NOMA due to the increase in the interference of AN. Therefore, the proposed scheme has great advantages in the low SNR region. When $\rho$ is 10, compared with Nco-NOMA and Co-NOMA, the SRP of Co-NOMA with AN is improved by 34% and is 2.22 times better, respectively.

Figure 3c,d evaluate the influence of various parameters on the performance of $U_2$ under the scenario of Co-NOMA with AN. Since the FD residual interference of $U_1$ will only affect OP of $U_2$, Figure 3c presents OP versus power distribution coefficient $a_2$ with different residual interference values. One observation is that the performance of $U_2$ improves with the increase in $a_2$. This can be attributed to the fact that the power allocated to $U_2$ becomes larger, resulting in the increase in effective SINR to decode $x_2$ at $U_2$. Another observation is that the residual self-interference at $U_1$ has little effect on the performance of $U_2$. Figure 3d describes the SRT versus $a_2$ and $\rho$. For the same $a_2$, the trend of SRT with $\rho$ is consistent with Figure 3b. In the region of low SNR, SRT increases as $a_2$ increases and eventually stabilizes. In the region of high SNR, SRT first increases then decreases with $a_2$, and finally tends to stabilize. This is because as $a_2$ increases, the OP of $U_2$ decreases, while the IP increases. For the low value of SNR, SRT is mainly determined by the OP, while for the high value of SNR, SRT is mainly determined by the IP.

In order to further illustrate the extent to which OP and IP are influenced by the information-bearing ratio and transmission power of the BS, Figure 4 plots the OP and IP of $U_2$ in the case of Co-NOMA with AN. Figure 4a shows the OP of $U_2$ versus ${\rho }_s$ and $\xi$; one can observe that as $\xi$ increases, the residual interference of $x_j$ becomes weaker, which brings about the improvement in outage performance. Moreover, at the higher value of $\xi$, as ${\rho }_s$ increases, the OP of $U_2$ is significantly improved and eventually reaches saturation with the assistance of the relay link. At the lower value of $\xi$, the outage performance initially improves gradually with the increasing ${\rho }_s$. As ${\rho }_s$ continues to increase, the residual interference of $x_j$ becomes stronger, resulting in a slight decrease in outage performance. Figure 4b presents the IP of $U_2$ versus $\rho$ and $\xi$. The IP of $U_2$ increases with the increasing value of $\rho$, which is consistent with that in Figure 3a. When $\xi$ is larger, $U_1$ allocates more power to broadcast the signal of $x_2$, Eve eavesdrops on $x_2$ by selecting the larger SINR from the BS and $U_1$ as an effective SINR. While $\xi$ is smaller, Eve primarily relies on the BS to decode the signal of $x_2$ by utilizing SC, and thus, along with $\xi$ increasing, IP increases. In addition, the change of $\xi$ has a greater impact on the IP at the low value of $\rho$.

5. Conclusions

This article investigated the PLS in a downlink Co-NOMA system. Specifically, the BS transmitted signals to the strong user directly, and to the weak user, with the assistance of the FD user relay in the presence of a passive Eve. FD and AN technologies were combined to improve the SRT performance. Analytical expressions of OP and IP were derived as performance metrics to investigate the SRT performance of the AN-aided Co-NOMA scheme. Additionally, the Co-NOMA scheme and Nco-NOMA scheme were also compared with the proposed scheme. Monte Carlo simulations showed the superiority of the proposed scheme in significantly enhancing the SRT performance of the weak user in the low SNR region, under the premise of minimizing the outage performance loss of the strong user. Additionally, the SRT performance of the two NOMA users can be further enhanced by utilizing the appropriate AN-bearing ratio and power allocation. For future work, the influence caused by imperfect SIC, imperfect CSI and estimation errors should be considered. At the same time, we would extend the considered system to the scenario in active eavesdropping scenarios. Furthermore, the application of multi-antenna technology and combining AN with the beamforming technique are other promising future directions to further strengthen the SRT performance.