Efficient Neural Network DPD Architecture for Hybrid Beamforming mMIMO

Abstract

This paper presents several different Neural Network based DPD architectures for hybrid beamforming (HBF) mMIMO applications. They are formulated, tested and compared based on their ability to compensate nonlinear distortion of power amplifiers in a single user (SU) and multiuser (MU) Fully-Connected (FC) HBF mMIMO transmitters. The proof-of-concept is provided with a 64 × 64 FC HBF mMIMO system, with 2 RF chains. The complexity of DPD solution is reduced by using a single Real-Valued Time-Delay Neural Network with two hidden layers (RVTDNN2L) instead of using as many different DPD blocks as there are RF chains in the HBF mMIMO transmitter and it is shown that the proposed architecture better compensates nonlinear distortion compared to the traditional memory polynomial DPD. Two RVTDNN2L DPD architectures are developed and tested for linearization of MU FC HBF mMIMO systems, and it is also shown that the proposed RVTDNN2L DPD architecture efficiently linearizes MU FC HBF mMIMO transmitters in terms of Normalized Mean-Squared Error (NMSE) and Error Vector Magnitude (EVM).

1. Introduction

Massive multiple-input multiple-output (mMIMO) combined with hybrid beamforming (HBF) has been widely accepted as one of the key technologies in fifth generation (5G) and beyond networks to improve the throughput and increase spectral efficiency [1,2,3,4,5,6,7]. Hybrid beamforming is a combination of analog beamforming in RF domain and digital beamforming in baseband. There are two types of HBF MIMO transmitter architectures: (a) fully connected (FC) architecture, when each antenna is connected to each of the P RF chains, and (b) partially connected or subarray architecture, when a set of L antennas is connected to one RF chain. On the other hand, in order to reduce power consumption, the power amplifiers (PAs), that are the main power consumers in the transmitters, must operate in a region close to saturation, causing increased nonlinear distortion and reduced spectral efficiency. Digital predistortion (DPD), as one of the most promising solutions and the most effective PA linearization technique, has been extensively studied in literature [8]. Conventional DPD requires dedicated predistorter for observing and linearizing the nonlinear distortion of each PA, which, in the case of HBF mMIMO systems, is completely unsuitable. Therefore, a lot of research work has been done to resolve DPD implementation problems in HBF mMIMO systems [9,10,11,12,13,14,15].

Most of these DPD techniques are proposed for subarray or reduced complexity HBF architectures, meaning that for the whole HBF mMIMO systems, as many DPD blocks as there are RF chains will be needed. In [9] DPD learning is based on the feedback from a single PA in the array. The decorrelation DPD model introduced in [10] uses a combined output signal of individual PAs in conjunction with a decorrelation-based learning rule. The same group of authors proposed a closed-loop (CL) DPD technique in [11] as a solution for multi-user hybrid MIMO that uses a single DPD block per RF transmission subarray. The Beam-Oriented DPD (BO-DPD) technique presented in [12] linearizes the signal of the main beam, not each PA individually, using the outputs of all PAs to obtain the feedback signal. Authors in [13] proposed a Power Scalable Beam-Oriented DPD (PSBO-DPD) that linearizes the signal of the main beam of a given subarray using feedback from only one PA. As a fully-connected (FC) HBF architecture allows higher spectral efficiency compared to a subarray HBF architecture, the authors in [15] proposed an approach for linearization of FC HBF mMIMO transmitters. However, the approach still uses as many DPD blocks as there are RF chains. Based on the type of feedback signal, over-the-air (OTA) DPD and direct transmit-end DPD are distinguished. Authors in [16] proposed a best linear approximation (BLA) based equalization strategy for OTA DPD in 5G Beamformer Array at 28 GHz and showed that using this approach enhances the linearization performance when a single and fixed set of DPD coefficients is used across different beam angles and receiver positions. Nevertheless, this paper considers the linearization of the main beam using direct transmit-end DPD and does not consider such equalization.

Neural networks (NNs) have high modelling accuracy as well as good distortion compensation performance in PA linearization [17,18,19,20,21,22,23,24]. Current investigations are based on the applications of different NN in DPD of a single PA. Authors in [20] proposed real-valued time-delay feedforward backpropagation-based NN for DPD in MIMO transmitters. However, as to the best of the authors knowledge, there have been no studies on the implementation of NNs DPD for HBF mMIMO yet.

In this paper, we propose NN based DPD for FC HBF mMIMO. Firstly, we present and analyze different NN configurations, and compare their predistortion capabilities in a single user (SU) FC HBF mMIMO transmitter, in terms of Error Vector Magnitude (EVM) and Normalized Mean-Squared Error (NMSE). It is shown that the proposed NN based DPD solution efficiently linearizes FC HBF mMIMO (with P RF chains) using a single NN instead of P different DPD blocks, thus reducing the complexity compared to the state-of-the-art DPD solutions for HBF mMIMO, which uses P different DPD blocks. Therefore, we proposed an advantageous Real-Valued Time-Delay Neural Network with two hidden layers (RVTDNN2L) and showed that this NN DPD solution has smaller NMSE and EVM compared to the state-of-the-art DPD solutions for SU HBF mMIMO. Secondly, for the proposed and selected RVTDNN2L DPD approach we investigated the influence of the number of RF branches on the linearization of the FC HBF mMIMO. We increased the number of RF branches to 4 and 8 and showed that, with the implementation of this approach, NMSE decreases by increasing the number of RF branches. At the end, we considered and examined a multiuser (MU) FC HBF mMIMO using two different DPD architectures with the RVTDNN2L approach. It is shown that the proposed approach better compensates for the nonlinear distortion of PAs in MU FC HBF mMIMO. One of the advantages of the RVTDNN2L DPD approach is also its adaptability, which means that with the change of environmental conditions, it can be relatively easily retrained and adapted to new conditions.

The rest of this paper is structured as follows. In Section 2, different NN based DPD models for PA linearization are presented and their performance and complexity are analyzed. Simulation results and comparisons between different NN DPD models for SU FC HBF mMIMO are given in Section 3 and RVTDNN2L DPD is proposed as the most efficient one. In Section 4, simulation results for the proposed RVTDNN2L DPD approach for MU FC HBF mMIMO are shown. Finally, Section 5 gives the conclusions.

2. NN DPD Models

A commonly used NN model has an input layer, one or more hidden layers and an output layer. Based on the number of hidden layers, NNs can be divided into two main categories: shallow and deep neural networks. While shallow NNs have one or two hidden layers, deep neural networks (DNNs) have at least three hidden layers with a high number of neurons in each layer. With the increase in the number of hidden layers, the network behavior is better modeled, while on the other hand, the requirements for signal processing increase. So, in this paper we analyze several different NN models such as, two shallow networks: real-valued time-delay neural network (RVTDNN) and augmented real-valued time-delay neural network (ARVTDNN), and one deep neural network (DNN), and propose the use of the RVTDNN network with two hidden layers (RVTDNN2L).

Input signals for the DNN, as well as for the RVTDNN and RVTDNN2L, are I and Q components of the signal. Beside I and Q components of the signal, ARVTDNN considers the envelope-dependent terms of the input signal, i.e., |x(n)|, |x(n)|² and |x(n)|³, where |x(n)| is amplitude of the input signal. The activation function used in these models is hyperbolic tangent (tanh), while for DNN model, sigmoid (logsig) activation function is used.

Complexity is one of the more important parameters that needs to be considered when choosing an appropriate NN DPD model. It can be expressed by the number of coefficients, as well as the number of floating-point operations (FLOPs). The total number of coefficients is equal to the sum of the used coefficients in every layer, and can be computed as follows:

$$N_{coef}=\left(N_i+1\right)N_1+{{\displaystyle \sum }}_{f=2}^F\left(N_{f-1}+1\right)N_f+\left(N_F+1\right)N_O$$

(1)

where Ni is the number of neurons in the input layer, F is the number of hidden layers, N_f is the number of neurons in f-th hidden layer and No is the number of neurons in the output layer.

The number of FLOPs is used for estimation of the computational complexity of the model and is different for every particular operation performed [24,25]. For the NNs with hyperbolic tangent activation function, number of FLOPs can be computed as follows:

$$\mathrm{FLOPs}=2N_i{N}_1+2N_F{N}_O+2{{\displaystyle \sum }}_{f=2}^F{N}_{f-1}{N}_f+15{{\displaystyle \sum }}_{f=1}^F{N}_f$$

(2)

For the NNs with sigmoid activation function, number of FLOPs can be computed as follows:

$$\mathrm{FLOPs}=2N_i{N}_1+2N_F{N}_O+2{{\displaystyle \sum }}_{f=2}^F{N}_{f-1}{N}_f+13{{\displaystyle \sum }}_{f=1}^F{N}_f$$

(3)

Due to its simple implementation, the indirect learning architecture (ILA) was used to form the DPD architecture based on single NN for the whole HBF mMIMO system, as shown in Figure 1. First, the appropriate NN model for DPD was formed and was trained using combined output data signals from all antennas and input data to the HBF MIMO transmitter. Then the trained NN model was used to update the input signal to HBF MIMO transmitter.

The algorithm used for training NN models is Levenberg-Marquardt optimization algorithm, a well-known and widely used type of back-propagation algorithm [18,21,22,24]. The Levenberg–Marquardt algorithm has been shown to perform exceptionally well because of its fastest convergence with lower mean square errors for training feedforward neural networks [26]. It also has an efficient implementation in Matlab ANN Toolbox. To update the values of the synaptic weights and biases in neural network, the Levenberg-Marquardt algorithm uses the following approximation:

$$x_{k+1}=x_k-{\left[J^{T}J+\mu I\right]}^{-1}{J}^{T}e$$

(4)

where $J$ is the Jacobian matrix that contains first derivatives of the network errors with respect to the weights and biases, and e is a vector of network errors. Initial $\mu$ is set equal to 0.001 and increase factor for $\mu$ is 10.

The optimal number of neurons in hidden layers are determined during training of the network and are set to give the best performance. As a result of extensive simulations, we determine following characteristics of NN models: DNN with 3 hidden layers, logsig activation function and 12 neurons in each hidden layer; RVTDNN with one hidden layer, tansig activation function and 35 neurons in hidden layer; ARVTDNN with one hidden layer, tansig activation function and 28 neurons in hidden layer; and RVTDNN2L with two hidden layers, tansig activation function, 18 neurons in the first hidden layer and 9 neurons in the second hidden layer. Block diagrams of these NN models are shown in Figure 2. As well, we propose the use of RVTDNN2L network for digital predistortion of FC HBF mMIMO transmitter. The performance and complexity of different analyzed NN models are listed in

Table 1. Performance and complexity of different NNs.

Type of NN DPD	Input Data	Number of Neurons in Input Layer	Number of Neurons in Hidden Layers	Activation Function	Number of Neurons in Output Layer	Ncoef	FLOPs
DNN	I/Q	2	12 12 12	logsig	2	374	852
RVTDNN	I/Q	2	35	tansig	2	177	805
ARVTDNN	I/Q, |x(n)|, |x(n)|2,|x(n)|3	5	28	tansig	2	226	812
RVTDNN2L	I/Q	2	18 9	tansig	2	245	837

. A large number of simulations were performed, varying the number of neurons per layer from 1 to 30 for the first hidden layer and from 1 to 15 for the second hidden layer, for RVTDNN2L network. For different combinations of neurons per layer, the number of coefficients according to formula (1) was calculated, as well as the number of FLOPs according to formula (2) and values of NMSE and EVM. Observing the obtained results, the optimal numbers of neurons were chosen so that NMSE and EVM values are comparable to the values existing in literature in the literature for HBF mMIMO transmitters, but also that the number of coefficients and FLOPs are as small as possible, in order to reduce the complexity of the system. The numbers of neurons per layer for other types of networks were chosen in a similar way, ensuring that their complexity is similar to that of the RVTDNN2L network.

3. Results

In this section, some DPD simulation results of the SU FC HBF mMIMO transmitter are presented. Individual PA units in transmitter are modeled using NN, based on measurement data from an actual PA. A two-stage PA (CFH 2162-P3 PA) with a 14 dB gain, and P1dB of 37 dBm driven in a hard compression region was used. The modulated waveform with channel bandwidth of 20 MHz, was fed into this power amplifier model at 2140 MHz. The process of PA modeling is shown in Algorithm 1 in Appendix A. Single PA has been modeled using one RVTDNN2L with two hidden layers, tansig activation function, 32 neurons in the first hidden layer and 15 neurons in the second hidden layer. To consider nonlinear crosstalk that occurs before PA, the output of each PA is modeled as:

$$y_i=f_i\left(x_i+{{\displaystyle \sum }}_{\begin{array}{c}n=1\\ n\ne i\end{array}}^{Nt}{\alpha }_{i,n}·x_n\right)$$

(5)

where $x_i$ is the input to i-th PA, $f_i$ is the PA response function, ${\alpha }_{i,n}$ is the crosstalk factor reflecting the impact of n-th PA to i-th PA [27]. Then NN based DPD model was created, using different NN architectures, as an integral solution to jointly compensate all PAs nonlinearity, crosstalk, I/Q imbalance and dc offset imperfections, similarly to the authors work presented in [22]. Considered DPD architecture is shown in Figure 1. To obtain the feedback signal, used for the DPD learning, the outputs of the individual PAs are firstly extracted, then co-phased and combined in the analog domain before being applied, meaning that direct transmit-end feedback is considered.

The basic concept of hybrid beamforming is to split both the precoding and combining weights to baseband digital weights and RF band analog weights. The problem of determining these weights is quite a large topic and is not considered here. In this paper, the orthogonal matching pursuit algorithm which is implemented in Matlab and proposed in [28] is used to obtain hybrid precoding and combining weights. Next, a clustered Saleh-Valenzuela channel model is adopted, with impulse response h(t) defined as follows [29]:

$$h\left(t\right)={\displaystyle \sum }_{l=0}^{N_{cl}-1}{\displaystyle \sum }_{k=0}^{N_{ray}-1}{\beta }_{kl}·e^{j{\theta }_{kl}}·\delta \left(t-T_l-{\tau }_{kl}\right)$$

(6)

where: N_cl and N_ray are maximum number of clusters and multipath components inside clusters, respectively, ${\beta }_{kl}$—the amplitude of k-th ray in l-th cluster, ${\theta }_{kl}$—the phase of the kl-th ray, T_l—time arrival of the l-th cluster, ${\tau }_{kl}$—arrival delay of k-th ray with respect to first ray of the l-th cluster. The model contains N_cl = 6 clusters, with N_ray = 5 rays in each cluster. It should be noted that a similar channel modeling approach is utilized in 3GPP standardization [30].

A 64-QAM modulated OFDM signal with a bandwidth of 20 MHz, generated in Matlab is used in DPD simulations. Subcarrier spacing is 15 kHz, and the number of used subcarriers is 1200. Matlab code for generating OFDM signal is given in Appendix B. Simulations of a 64 × 64 fully-connected hybrid beamforming MIMO system, with 2 RF chains were performed. Four different NN models for DPD are considered: DNN, RVTDNN, ARVTDNN and RVTDNN2L, and compared with each other and with the memory polynomial (MP) DPD approach based on following formula [31]:

$$x_{out}\left(n\right)={\displaystyle \sum }_{m=0}^M{\displaystyle \sum }_{k=1}^K{a}_{mk}·x_{in}\left(n-m\right)·{\left|x_{in}\left(n-m\right)\right|}^{k-1}$$

(7)

where: $a_{mk}$—model coefficients, M—memory depth, K—nonlinearity order, $x_{in}$ and $x_{out}$—input and output signal, respectively. Simulations were performed for MP model used in [32] with memory depth M = 3 and order of nonlinearity K = 11. Block diagram of used MP model is shown in Figure 3.

Figure 4 shows obtained power spectrum densities (PSDs) without DPD, with MP DPD model and with analyzed NN DPD models.

Table 2. Performance of different NN DPD architectures for 64 × 64 FC HBF MIMO system.

Type of DPD	EVM [%]	NMSE [dB]
Without DPD	22.09	-
MP	2.04	−33.79
DNN	1.37	−37.25
RVTDNN	1.99	−33.98
ARVTDNN	1.70	−35.39
RVTDNN2L	1.23	−38.19

summarizes the performance of MP DPD model, different NN DPD models and the proposed RVTDNN2L DPD model, in terms of EVMs and NMSEs. As it can been seen, PA linearization is significantly improved with NN DPD models compared to MP DPD model, especially with the proposed RVTDNN2L DPD model. It should be noted that observed NN DPD models are similar in complexity, i.e., in number of FLOPs. As it can be seen from the obtained results, the proposed RVTDNN2L DPD outperforms other analyzed DPD models.

Further, the proposed RVTDNN2L DPD technique has been applied on FC HBF mMIMO system with 4 and 8 RF branches. Simulations were performed in Matlab, with the same input signal and same parameters of RVTDNN2L DPD as in the previous analysis. The same orthogonal matching pursuit algorithm and Saleh-Valenzuela channel model were used to generate the hybrid beamforming coefficients for system with 4 and 8 RF branches. Figure 5 shows the power spectrum densities (PSDs) for FC HBF mMIMO system with 2, 4 and 8 RF branches with the proposed RVTDNN2L DPD model.

Table 3. Performance of RVTDNN2L DPD architecture for 64 × 64 FC HBF MIMO system with 2, 4 and 8 RF branches.

	2 RF Branches	4 RF Branches	8 RF Branches
EVM without DPD [%]	22.09	22.39	23.39
EVM with RVTDNN2L DPD [%]	1.23	1.04	0.74
NMSE [dB]	−38.19	−39.63	−42.60

summarizes the performance of the proposed RVTDNN2L DPD model for system with 2, 4 and 8 RF branches, in terms of EVMs and NMSEs. It can be seen that EVM without DPD increases with increasing number of RF branches, meaning that nonlinear distortion increases with increasing number of RF branches, which was expected. The obtained simulation results showed that EVM with the proposed RVTDNN2L DPD model decreases with increasing number of RF branches, which means that the proposed model better compensates nonlinear distortion if the number of RF branches increases, since the RF signals are more correlated with each other.

Although the use of more RF branches achieves better results in EVM and NMSE with the application of digital predistortion, due to the cost of implementing a physical system with more RF branches, in practice HBF MIMO transmitters with 2 RF branches are used. Consequently, FC HBF mMIMO with 2 RF branches will be analyzed in further research of RVTDNN2L DPD for MU case.

4. RVTDNN2L DPD Approach for MU Case

In MU FC HBF MIMO transmitter it is necessary to take into account the signals from all users during the construction of the DPD module, due to the fact that during HBF the data to be sent are formed on the basis of data from all users. In this paper we considered a FC HBF MIMO system with only two users, but with certain minor changes it can be adapted to the case of more than two users.

We proposed and analyzed two architectures for a multi-user system:

• DPD system with one RVTDNNL network with inputs from both users and outputs for both users, shown in Figure 6 (arch1),
• DPD system with two RVTDNNL networks with inputs from both users and output for one user, shown in Figure 7 (arch2).

Levenberg-Marquardt optimization algorithm was applied for training RVTDNN2L. Tansig activation function was used in both hidden layers. We tested model with different number of neurons in hidden layers of network and proposed the optimal number to obtain the best performance. Graphics and tables are given for three combinations of numbers of neurons in hidden layers, i.e., for I = 18 neurons in the first hidden layer and II = 9 neurons in the second hidden layer; I = 32 neurons in the first hidden layer and II = 15 neurons in the second hidden layer; and I = 48 neurons in the first hidden layer and II = 21 neurons in the second hidden layer. As for SU FC HBF MIMO, a clustered Saleh-Valenzuela channel model is adopted. For MU FC HBF MIMO, the model contains N_cl = 6 clusters, with N_ray = 10 rays in each cluster. For the scheme in Figure 6, the proposed RVTDNN2L DPD model with 4 inputs and 4 outputs was used, i.e., we considered I and Q components of signals from both users, both inputs and outputs. While for the scheme in Figure 7, the proposed RVTDNN2L DPD model has two NNs, one for each user, both with 4 inputs and 2 outputs, i.e., we considered I and Q components of output signals from both users, and I and Q components of input signals for specific user.

The same PA model as for the SU case was used. Two 64 QAM modulate OFDM signals with a bandwidth of 20 MHz were generated in Matlab. Simulations of a 64 × 64 FC HBF MIMO system, with 2 users and 2 RF chains were performed, for both architectures shown in Figure 6 and Figure 7.

Figure 8 and Figure 9 show obtained power spectrum densities (PSDs) without DPD and with proposed RVTDNN2L DPD model for both types of architecture. While

Table 4. Performance of RVTDNN2L DPD for 64 × 64 MU FC HBF MIMO system—user 1.

RVTDNN2L	Arch1		Arch2
	EVM [%]	NMSE [dB]	EVM [%]	NMSE [dB]
I = 18, II = 9	4.07	−27.81	3.16	−30.02
I = 32, II = 15	3.08	−30.22	2.13	−33.45
I = 48, II = 21	2.48	32.11	1.78	−34.97

EVM without DPD is 17.43 %.

and

Table 5. Performance of RVTDNN2L DPD for 64 × 64 MU FC HBF MIMO system—user 2.

RVTDNN2L	Arch1		Arch2
	EVM [%]	NMSE [dB]	EVM [%]	NMSE [dB]
I = 18, II = 9	4.56	−26.81	3.47	−29.19
I = 32, II = 15	3.34	−29.52	2.24	−32.98
I = 48, II = 21	2.69	−31.40	1.97	−34.11

EVM without DPD is 18.58 %.

summarize the performance of the proposed RVTDNN2L DPD models for each user separately, in terms of EVMs and NMSEs. As can been seen, PA linearization is slightly better with RVTDNN2L DPD model with two NNs compared to RVTDNN2L DPD model with one NN. On the other hand, it should be noted that the architecture with two NNs is more complex, in terms of number of coefficients and FLOPs, compared to architecture with one NN, which is shown in

Table 6. Performance and complexity of different RVTDNN2L DPD models.

RVTDNN2L	Arch1		Arch2
	Ncoef	FLOPs	Ncoef	FLOPs
I = 18, II = 9	301	945	562	1818
I = 32, II = 15	719	2041	1374	3962
I = 48, II = 21	1357	3603	2626	7038

.

From the above results, we conclude that compared to SU case it is necessary to further increase the complexity of the RVTDNN2L DPD system for MU case in order to efficiently linearize PAs, either by increasing the number of neurons in each layer, or by introducing another RVTDNN2L DPD for another user. Based on the results of extensive simulations, taking into account the complexity of the system, the best results were shown by the architecture with 2 RVTDNN2L networks, with 32 and 15 neurons in the first and the second hidden layer, respectively.

5. Conclusions

In this paper, the different NN based DPD architectures are formulated and tested. The RVTDNN2L DPD architecture in particular, which has been developed and implemented for the first time, best compensates nonlinear distortion of PAs in FC HBF mMIMO transmitters.

The developed RVTDNN2L DPD architecture was tested for linearization of FC HBF mMIMO system with more than 2 RF branches, and it was shown that, even though EVM without DPD increases with an increasing number of RF branches, the application of the proposed RVTDNN2L DPD model decreases total NMSE. Both SU and MU FC HBF mMIMO were analyzed and comprehensive simulations have been performed. It was shown that the proposed RVTDNN2L DPD model efficiently linearizes SU, as well as MU FC HBF mMIMO transmitters, where NMSE was of the same order of magnitude as in the existing DPD solutions for HBF mMIMO. As a result of all the above contributions, this study has shown that the developed RVTDNN2L DPD solution has very good PA modeling and linearization performance, in terms of NMSE and EVM, for FC HBF mMIMO transmitters.