Game Algorithm for Resource Allocation Based on Intelligent Gradient in HetNet

Abstract

In order to improve system performance such as throughput, heterogeneous network (HetNet) has become an effective solution in Long Term Evolution-Advanced (LET-A). However, co-channel interference leads to degradation of the HetNet throughput, because femtocells are always arranged to share the spectrum with the macro base station. In this paper, in view of the serious cross-layer interference in double layer HetNet, the Stackelberg game model is adopted to analyze the resource allocation methods of the network. Unlike the traditional system models only focusing on macro base station performance improvement, we take into account the overall system performance and build a revenue function with convexity. System utility functions are defined as the average throughput, which does not adopt frequency spectrum trading method, so as to avoid excessive signaling overhead. Due to the value scope of continuous Nash equilibrium of the built game model, the gradient iterative algorithm is introduced to reduce the computational complexity. As for the solution of Nash equilibrium, one kind of gradient iterative algorithm is proposed, which is able to intelligently choose adjustment factors. The Nash equilibrium can be quickly solved; meanwhile, the step of presetting adjustment factors is avoided according to network parameters in traditional linear iterative model. Simulation results show that the proposed algorithm enhances the overall performance of the system.

1. Introduction

With the constant evolution of cellular mobile communication systems, traditional cellular monolayer networks have failed to satisfy the requirement of transmission speed, and are beset by such issues as imperfect indoor coverage effect and insufficient capacity for outdoor hotspot areas [1]. Hence, the heterogeneous network (HetNet) has become a key technology for the next generation of communication networks. Given that over 70% mobile data is produced indoors at present [2,3], the femtocell can effectively solve relevant issues about indoor communication due to its flexibility of deployment. Therefore, deploying femtocells in current macro-cellular networks to form macro-femto bilayer HetNet can transfer the load of the macro base station to femtocells, thus improving system performance. However, the introduction of femtocells will bring cross-layer interference between macro base stations and femtocells. The macro users located around femtocells tend to be highly subject to the strong interference of femtocells, thereby lowering the overall performance of system, especially the throughput [4]. In order to solve the above problems within academia, there are some cases using resources allocation method.

In recent years, most research concerning resource allocation algorithms have introduced game theory [5,6]. A game of resource allocation problem in HetNets involves a set of players, strategies, and payoffs. The players can be defined as base stations and users. Strategies can be the choice of transmission power level or subcarrier allocation. The payoff is the evaluation for a player of all possible outcomes represented by the utility function. Based on these settings, the interferences in wireless networks can then be considered as a result of the frequency resources among stations and users. Due to the strong transmitting power and wide coverage scope, the macro base station plays a more important role than femtocells in the formation of cross-layer interference, so it should first satisfy the requirement of frequency spectrum of macro users. Moreover, players cannot know strategy at the current moment, which means that it is a game with incomplete information. In previous literature, games with incomplete information—known as Bayesian games—are employed to address these kinds of issues. These games often assume that the players act independently according to some complex strategies for optimizing the setting of various resources of the network. With respect to HetNet, the macro plays a centralized role and can communicate with the low power node (LPNs) through the X2 interface. Thus, the topology matches a game with the macro base station being leader and the femtocells acting as followers. With this understanding, the Stackelberg game is a strategic game in economics in which the leader firm moves first and then the follower firms move sequentially. In the Stackelberg model, the leader can make prior decisions, and followers will choose a reasonable resources allocation scheme based on these decisions. Therefore, the Stackelberg game theory is more appropriate to analyze the resources allocation of HetNet based on cross-layer interference suppression.

There are already some researchers studying this area. The Stackelberg game theory was used to solve the resource allocation problem both in power and in spectrum [7,8,9,10,11,12,13,14,15,16,17,18,19,20]. In the two-tier network scenario of single macro community, reference [8] proposed a supermodel game-based femtocell downlink power control algorithm in the double network of single community. The proposed algorithm maximized the network capacity through reducing femtocells’ cross-layer interference upon neighboring macro users, yet without considering the interference of the macro base station upon femtocell users or the macro base station in game process. In the same network scenario, [10] proposed a power control algorithm with self-adapting utility based on Signal to Interference plus Noise Ratio (SINR). This algorithm could be realized through distributed ways and gradually reduce femtocell transmitting power causing strong interference, so as to effectively hinder cross-layer interference. The Stackelberg game is introduced to address numerous problems such as power control [8,10], but there are only a few algorithms attempted to optimize the spectrum sharing between macro and femtocells. In particular, reference [11] proposed a price based resource allocation strategy to handle the spectrum sharing problem, where the macro base station acts as a leader and protects itself by pricing the interference from femtocell users. In the system model given by [12], the leaders in the Stackelberg game could make priority decisions, while the followers should choose a reasonable resource allocation scheme based on the decisions. Considering both macro base station utility and femtocell utility, reference [14] proposed the Stackelberg game based on an uplink power distribution framework. Under the condition of ensuring Quality of Service (QoS) of macro users, it did not take into account the uplink interference of macro users upon femtocell. Besides, when transmitting power of macro base station was given, the macro base station had enough information to predict response of base station, which would bring large amount of signaling overhead to network. A kind of spectrum leasing framework with one substep to obtain the optimal algorithm of Stackelberg equilibrium was proposed in [16]. Based on the above analysis, a kind of spectrum leasing framework was also proposed in [18], where femtocells lease spectrum from their co-existing macro base stations to serve femtocell users and allow the dynamic access of macro base station users. The above references all assume that each base station in HetNet is an independent individual in game model that attempts to seek maximal energy self-sufficiency without considering the feelings of other individuals in the network, so the optimal overall performance of the community cannot be realized through adjusting self-possessed spectrum resources. Pricing policy considering both economic income and spectrum income was proposed in [19,20]. When femtocells bought spectrum from a macro base station, the price would be determined by the improvement brought by system throughput. Meanwhile, if femtocells can provide macro base station users under severe interference with spectrum and access service, the price will be reduced to certain degree. However, setting a spectrum price will lead to high signaling overhead. At the same time, none of the above methods has provided the solution scheme when the value assignment scope for the Nash equilibrium is continuous. The Nash equilibrium is a solution concept of a game involving two or more players in which each player is assumed to know the equilibrium strategies of others, and a player has nothing to gain by just changing their own strategy. Summarizing the results of the above research, some problems exist in current algorithm studies. On the one hand, most algorithms focus on the performance of the macro base station but neglect the performance of the overall system; on the other hand, most of the existing algorithms adopt utility function using earnings to define both game playing parties. Pecuniary transactions are not only impractical but also the cause of high signaling overheads [21]. Moreover, most of the algorithms are Stackelberg game featuring finite discrete perfect information, whose Nash equilibrium solution can be obtained through simple backward induction. However, when the range of participant features have continuum value, game playing will become continuously expansive and dynamic. Equilibrium is never actually reached since the equilibrium they could pick from was discrete. It would drop to the Bertrand equilibrium. Letting the players learn an optimal payoff function, it can be found that automated learning gives consistent results. The optimum for the payoff function corresponds to the optimal price in an equilibrium. Therefore, an effective and intelligent Nash equilibrium solution algorithm is needed.

In this paper, the Stackelberg game model is adopted to cope with modeling resources allocation. First, we build a spectrum allocation scheme with the concept similar to community expansion, where macro base station pushes partial users under severe cross-layer interference to femtocells nearby and contributes partial spectrum as return. Then considering the interest of both the macro base station and femtocells, one kind of utility function without using pricing strategy is built to improve the overall system throughput capacity. Finally, given that optimal strategy value range is continuous, which renders the existing algorithms impossible to work out Nash equilibrium, a gradient algorithm is introduced for the first time to solve Nash equilibrium, based on which an effective and intelligent gradient algorithm is proposed. Simulation results show that this scheme has improved the performance of the system to a certain extent.

The rest of the paper is organized as follows. Section 2 presents the system model. Section 3 explains the utility functions and optimization objects analysis. Section 4 introduces the iterative algorithm by adopting the gradient descent algorithm. In Section 5, the simulation is given to evaluate the improved performance of the proposed algorithm. Finally, we show the conclusion of the paper.

2. System Model

The paper adopts a downlink bilayer HetNet model consisting of a macro base station and femtocells. Figure 1 schematically shows a downlink communication system. Downlink transmission links in the HetNet model consist of five parts. The solid line represents the useful signal, while the dash line represents the interference signal. The community contains one macro base station m and several femtocells $F_j,j\in \{1,2,\dots ,N\}$. Base stations of the same type are in the same layer and enjoy the same transmitting power. In previous system models, femtocells are normally randomly and evenly distributed. However, in actual environment, different regions have different requirements for capacity. Thus, the actual network deployment will display a certain feature of randomness. Hence, we build a commonly applicable HetNet model with consideration of random and independent distribution of base stations. Femtocell is assumed to be subject to an independent Poisson point process with a density of ${\lambda }_j$, and this process is only related with the parameter of density. Macro users and femtocell users are assumed to be randomly distributed, with a number of S_m and S_Fj, respectively [22,23]. Then m_i refers to the ith macrocell user, and F_j,i refers to the ith user of S_Fj.

Considering spectrum scarcity and increasing capacity demand, full frequency multiplex has become a main feature of a multilayer HetNet [24], but it will lead to severe cross-layer interference, especially when macro users are close to a femtocell. The channel transmission model is a mixed one consisting of path loss and shadow fading with logarithmic normal distribution. The overlapping of the two can reflect the decrease of signal strength with distance and random decline of path loss caused by obstacles (indoor walls) in communication link, hence restoring the actual communication link to the largest degree [25]. Based on base station and location information of users, modeling is conducted of SINR of different users for analysis. For macro users, they undergo interference of all femtocells in the community. It is assumed that the macro base station has a transmitting power of $P_m$ located at (0, 0) and, for each j in $\{1,2,\dots ,N\}$, a femtocell F_j located at $\left(x_j,y_j\right)$ and with a transmitting power of $P_{F_j}$, namely $FB{S}_j$. When macro base station user $m_i$ is located at $\left(x,y\right)$, urban space transmission formula $128+37.6\times {\log }_{10}(dist)$ [26] is adopted, where $dist$ means the distance between the user and base station. The SINR of the user can be calculated as follows:

$$SIN{R}_{m_i}=\frac{P_{m,m_i}}{{\displaystyle {\sum }_{j=1}^N{P}_{F_j,m_i}}+N_0}$$

(1)

where $P_{m,m_i}$ and $P_{F_j,m_i}$ are the received signal power from m and F_j to the user m_i respectively, which are calculated as $P_{m,m_i}=P_m\cdot {10}^{-\frac{128+37.6\cdot {\log }_{10}{\left(x^2+y^2\right)}^{1/2}}{10}}$ and $P_{F_j,m_i}=P_{F_j}\cdot {10}^{-\frac{128+37.6\cdot {\log }_{10}{\left({\left(x-x_j\right)}^2+{\left(y-y_j\right)}^2\right)}^{1/2}}{10}}$. $N_0$ is the noise power.

For femtocell user F_i,j, they will undergo interference from macro base station and other femtocells. Each k in $\{1,2,\dots ,N\}$, a femtocell F_k located at $\left(x_k,y_k\right)$ and with transmitting power of $P_{F_k}$, namely $FB{S}_k$. When the femtocell user F_i,j is located at$\left(x_f,y_f\right)$, the received signal power mentioned in the Equation (2) will adopt indoor space transmission formula $127+30\times {\log }_{10}(dist)$ [27]. The SINR can be calculated as follows:

$$SIN{R}_{F_{j,i}}=\frac{P_{F_j,F_{j,i}}}{P_{m,F_{j,i}}+{\displaystyle {\sum }_{k\ne j}{P}_{F_k,F_{j,i}}+N_0}}$$

(2)

where $P_{F_j,F_{j,i}}$ is the received signal power from F_j to user F_j,i which is calculated as $P_{F_j,F_{j,i}}=P_{F_j}\cdot {10}^{-\frac{127+30\cdot {\log }_{10}{\left({\left(x_f-x_j\right)}^2+{\left(y_f-y_j\right)}^2\right)}^{1/2}}{10}}$. Similar to the calculation of $P_{m,m_i}$, $P_{m,F_{j,i}}$, and $P_{F_k,F_{j,i}}$ are the received signal power from m and F_j to user F_j,i. which are calculated as $P_{m,F_{j,i}}=P_m\cdot {10}^{-\frac{128+37.6\cdot {\log }_{10}{\left({x_f}^2+{y_f}^2\right)}^{1/2}}{10}}$ and $P_{F_k,F_{j,i}}=P_{F_k}\cdot {10}^{-\frac{128+37.6\cdot {\log }_{10}{\left({\left(x_f-x_k\right)}^2+{\left(y_f-y_k\right)}^2\right)}^{1/2}}{10}}$.

When SINR of users (calculated as Equation (1) or (2)) is lower than the SINR threshold $c_{th}$, where $c_{th}$ is a predefined threshold which has been discussed in [28], it is held that femtocell has sufficient interference on macro base station users, with strong interference link existing. At the moment, to ensure the improvement of overall system performance, we adopt the thought similar to community expansion, i.e., commanding the macro base station to allocate its users under severe interference to femtocells nearby and transferring partial spectrum resources to femtocells to serve the users allocated. The diagram is as follows.

The gray area in Figure 2 is the existing service area of femtocell, and MUE4 is a macro user under serious cross-layer interference. To eliminate cross-layer interference there, the macro base station divides frequency spectrum with $\alpha \cdot W$ bands for use of the femtocell, so as to expand its existing service area with a factor of proportionality of $\beta$, where $\alpha$ is spectrum bias transferred by macro base station. Femtocell will include users under serious interference into family users and uninstall the partial load of macro base station to uplift the overall performance of the system. Expanded service area is indicated in blue as in the Figure 2. In this way, user F_j,i served by $\alpha \cdot W$ spectrum will no longer be subject to cross-layer interference. The computation formula for SINR is calculated as follows:

$$SIN{R^{*}}_{F_{j,i}}=\frac{P_{F_j,F_{j,i}}}{{\displaystyle {\sum }_{k\ne j}{P}_{F_k,F_{j,i}}+N_0}}$$

(3)

This scheme encourages femtocells to unload the load of macro base station. Substantially, it means expansion of femtocell community service area, so as to improve the overall system performance.

3. Stackelberg Game Model and Problem Statement

3.1. Common Algorithm

Most algorithms based on existing Stackelberg game theory adopt utility function using earnings to define macro base station, but in reality, spectrum pricing will bring large signaling overhead. Based on this, references [29,30] made improvements by defining the utility function as average throughput capacity of macro base station and standard distortion rate of femtocells, respectively. However, each base station in the heterogeneous network was assumed as an independent individual in-game model, which attempted to seek maximal energy efficiency without considering feelings of other individuals in the network or the overall performance of system. To consider the overall performance of the system, and to improve the performance of the system to reduce the signaling overheads has become our primary concern. Meanwhile, when applying game models to solving the problem of cross-layer interference, adopting backward induction [13] to obtain Nash equilibrium can only be applied to the special case of a finite perfect information discrete game. The Nash equilibrium range of most games should be continued. In terms of our research, there has not been a solution for Nash equilibrium with a continuous value range.

In the following parts, we will discuss one kind of optimal algorithm for solving the above problems. The game flow in the following part can be summarily described. The femtocells describe the amount of parameter $\beta$, which is the rate between the final user number included in the femtocell service scope and the initial user number of the femtocell. The macro base station continues to choose $\alpha$ as the strategy, where $\alpha$ expresses the spectrum retained by the macro base station.

3.2. Utility Functions

Femtocells in the system model of this paper are randomly deployed, which will make macro users close to femtocells susceptible to severe cross-layer interference and ultimately influence the throughput capacity of the system. To resist cross-layer interference, the macro base station will choose some macro users that cannot receive good service from it and allocate them to femtocells close by encouraging femtocell to expand its service area. The cost for macro base station is that some originally available spectrum will be used by femtocell to serve those macro users separated out, thus realizing optimal community energy efficiency through adjusting self-possessed spectrum resources. To ensure the overall system performance, we attempt to define the utility function of participants as average throughput capacity. Firstly, for users of the macro base station, the average throughput capacity of macro users can be defined as follows:

$$r_{m_i}=\frac{1}{S_m}(1-\alpha )W{\log }_2(1+SIN{R}_{m_i})$$

(4)

where W is the overall bandwidth that can be provided by the system. As for femtocell, the average throughput capacity of family users before the service area is expanded can be defined as follows:

$$r_{j,i}=\frac{1}{S_{F_j}}(1-\alpha )W_j{\log }_2(1+SIN{R}_{j,i})$$

(5)

where W_j is the bandwidth occupied by femtocell F_j. After the expansion of service area, partial macro users will be served by a femtocell using $\alpha \times W$ frequency band, and its average throughput capacity of users can be defined as follows:

$$r_{j,i}^c=\frac{1}{S_{F_j}^c}\alpha W_j{\log }_2(1+\frac{SIN{R}_{j,i}^{*}}{{\beta }_j})$$

(6)

where ${\beta }_j$ is community expansion coefficient and $S_{F_j}^c$ is the number of extra users served by femtocell after uninstalling load of macro base station, which is also positively correlated with ${\beta }_j$.

Based on the average throughput capacity defined above, we can define the utility function of game players. The users defined above are randomly distributed. Therefore, for any random users, the minimal average throughput capacity r can be obtained through Equations (4)–(6). Assuming that $r_0$ is the minimal average throughput capacity of system users, the utility function of a base station can be defined as follows:

$$U=N\log (\frac{r}{r_0})$$

(7)

This utility function ensures the proportional fairness among the users. Meanwhile, ${{\partial }^{2}U/\partial r}^2\le 0$, based on which it can be judged that this utility function is a concave function, i.e., Nash equilibrium of the system can be obtained through a linear optimization algorithm. Thus, we can establish a Stackelberg game model in which the macro base station acts as the leader, and the femtocells act as the followers. In the following, we will discuss specific revenue function expressions for femtocells and the macro base station.

Based on the spectrum reuse scheme, the reward to be obtained by femtocells is the partial spectrum from the macro base station. The rate of these spectrum accounts for $\left(1-\alpha \right)\cdot W$. If the transferred spectrum of the macro base station exceeds its own capacity, the average throughput capacity will decrease. Hence, the rate of the transferred spectrum shall be prudently chosen. Similarly, to ensure the throughput of the overall system, the bias of numbers to be accepted by femtocells shall not exceed its own capacity. This means that there is a compromise of two conditions between leaders and followers. Only in this way, can the overall performance of system be ensured.

For the femtocells, a plan is made for each femtocell according to the obtained rate parameters, and some surrounding macro users are chosen to provide service for them. In other words, cell expansion bias ${\beta }_j$ expresses the rate between the final user number included in the femtocell service scope and the initial user number of the femtocell. Given that leaders can make priority decisions, we need to first analyze the non-cooperative decision making process of femtocells. As for given parameters, each femtocell chooses service objects according to the descending order of signal strength of macro users to be serviced. Each femtocell chooses the parameters that will maximize its utility function. This can be expressed as follows:

$$\begin{array}{l}\max U_j=S_{F_j}\log (\frac{r_{j,i}}{r_0})+S_{F_j}^c\log (\frac{r_{j,i}^c}{r_0})-\left(1-\alpha \right)\omega W_j\\ s.t.\hspace{1em}{\beta }_j\ge 1,j=1,2,\dots ,N\\ 0\le W_j\le W\end{array}$$

(8)

where $S_{F_j}^c$ represents the macro users served by a femtocell, which can be expressed as $\lceil ({\beta }_j-1)\cdot S_{F_j}\rceil$ since $\beta$ is not an integer. $\omega$ is a positive constant. From the above expressions, it can be seen that the load taken by a femtocell for the macro base station becomes the revenue of the former. Assuming that each femtocell cannot get the information from one other, and the Equation (8) is an independent process of non-cooperation, the optimal strategy for F_j can be defined as follows:

$$\begin{array}{l}{\beta }_j^{*}=\arg \underset{{\beta }_j}{\max }{U}_{F_j}\\ s.t.0\le W_j\le W\end{array}$$

(9)

Meanwhile, the strategy set of femtocells can be defined as $\beta =[{\beta }_1,{\beta }_2,\dots {\beta }_N]$.

For the macro base station, although it sacrifices partial spectrum, users receiving its service still enjoy sound performance with uplifted system gain. The revenue function of macro base station can be defined as follows:

$$\begin{array}{l}\max U_m=S_m^{*}\log (\frac{r_{m_i}}{r_0})\\ s.t.0\le \alpha \le 1\\ {\beta }_j\ge 1,j=1,2,\dots ,N\end{array}$$

(10)

where $S_m^{*}$ is the actual number of users served by final macro base station, and can be obtained through $S_m^{*}=S_m-{\displaystyle \sum _{j=1}^N{S}_{F_j}^c}$. In this way, the optimal strategy for macro base station m can be defined as follows:

$${\alpha }^{*}=\arg \underset{\alpha }{{\displaystyle \max }}{U}_m$$

(11)

In this part, system utility function and revenue function are given. Next, we will present the solutions to obtain the optimal strategy.

3.3. Optimization Problems

In the above part, we have determined the optimized results reached by femtocells and a macro base station. In this part, we will elaborate on the specific methods.

According to the Stackelberg game model, the leaders are normally the first one to make response, and the followers make a corresponding strategy choice based on it. Due to the fact that the Stackelberg model is a double oligarchies game model, game process has no obvious primary and secondary classes. In the paper, we will firstly determine the optimal strategy of macro base station. The femtocells serve a part of the macro users because the macro base station is at the expense of the spectrum. This process is the one by which the femtocell offloads load. Given that final user number of the macro base station is the result of femtocells uninstalling the load, user number of a macro base station is the linear function related with community expansion bias $\beta$. Therefore, revenue expression of a macro base station can be rewritten with the following equation:

$$\begin{array}{l}\max U_m=f(\beta )\log (\frac{r_m}{r_0})\\ s.t.0\le \alpha \le 1\\ {\beta }_j\ge 1,j=1,2,\dots ,N\end{array}$$

(12)

Moreover, $U_m$ and $W_j$ are relevant, so we will first introduce the optimal solution $W_j^{*}$ of system bandwidth obtained by a femtocell. The second partial derivative of $W_j$ obtained through $U_j$ is $\frac{{\partial }^2{U}_j}{\partial W_j^2}=-\frac{S_{F_j}}{W_j^2}-\frac{S_{F_j}^c}{W_j^2}<0$. $U_j$ as a function of $W_j$ is a concave function. That is to say, optimal solution $W_j^{*}$ can be obtained through one-order function.

$$\frac{\partial U_j}{\partial W_j}=\frac{S_{F_j}}{W_j}+\frac{S_{F_j}^c}{W_j}-(1-\alpha )\omega$$

(13)

Optimal solution for femtocell spectrum distribution can be obtained as shown in Equation (14):

$$W_j=\{\begin{array}{l}\frac{\lceil \beta S_{F_j}\rceil }{(1-\alpha )\omega }\hspace{1em}0\le \alpha \le 1-\frac{\lceil \beta S_{F_j}\rceil }{\omega \theta }\\ W1-\frac{\lceil \beta S_{F_j}\rceil }{\omega \theta }\le \alpha \le 1\end{array}$$

(14)

When substituting Equation (14) into Equation (12), we can get the following results:

$$\begin{array}{l}\max U_m=f(\beta )\log [\frac{\beta (1-\alpha )W\log (1+SIN{R}_{m_i})}{f(\beta )r_0}]-{\displaystyle \sum _{j=1}^{N}f({\beta }_j)\log [\frac{{\beta }_j\alpha S_{F_j}{W}_j\log (1+\frac{SIN{R}_{j,i}^{*}}{{\beta }_j})}{(1-\alpha )\omega f({\beta }_j)r_0}]}\\ s.t.0\le \alpha \le 1\\ {\beta }_j\ge 1,j=1,2,\dots ,N\end{array}$$

(15)

For the optimization issues with constraints, normally the Lagrange function is used to build this function. To meet the demand of optimization results of constraint, the Lagrange function is used to build a modified target function $\varphi (\alpha )=U_m(\alpha )-\mu (\alpha -1)$, where $\mu$ is a constant. The second partial derivative $\varphi (\alpha )$, $\frac{{\partial }^2\varphi (\alpha )}{\partial {\alpha }^2}=-\frac{S_m}{{(1-\alpha )}^2}-{\displaystyle {\sum }_{j=1}^N(\frac{S_{F_j}^c}{{\alpha }^2}-\frac{S_{F_j}^c}{{(1-\alpha )}^2})}$, related with $\alpha$ is less than 0. It can be understood that this optimization issue is a concave function, the optimization for which can be obtained through the one-order function. To obtain the optimal spectrum distribution factor ${\alpha }^{*}$ of Nash equilibrium, assume $\partial \varphi (\alpha )/\partial \alpha =0$, where:

$$\frac{\partial \varphi (\alpha )}{\partial \alpha }=-\frac{S_m}{1-\alpha }+\frac{{\displaystyle {\sum }_{j=1}^N{S}_{F_j}^c}}{(1-\alpha )\alpha }-\mu$$

(16)

Then we can obtain Nash equilibrium ${\alpha }^{*}$, with the expression shown as below:

$${\alpha }^{*}=\sqrt{\frac{{(\mu +S_m)}^2}{4{\mu }^2}-\frac{{\displaystyle {\sum }_{j=1}^N{S}_{F_j}^c}}{\mu }}+\frac{\mu +S_m}{2\mu }$$

(17)

After determining the optimal strategy of the macro base station, we will introduce an optimal strategy solution for femtocells. Previously, we have introduced a revenue function for femtocells, which includes two parts. Namely, the revenue for users and the revenue obtained through macro base station uninstalling its partial load on partial spectrum. One-order derivative of ${\beta }_j$ is solved for Equation (18).

$$\frac{\partial U_j}{\partial \beta }=\log \left[\frac{{\alpha }^{*}{W}_j\log \left(1+SIN{R}_{j,i}^{*}/{\beta }_j\right)}{\left(\beta -1\right)S_{F_j}{r}_0}\right]-\frac{\left(\beta -1\right)}{\beta SIN{R}_{j,i}^{*}\log (1+SIN{R}_{j,i}^{*}/{\beta }_j)}-1$$

(18)

Further obtaining the second order derivative.

$$\frac{{\partial }^2{U}_j}{\partial {\beta }^2}=-\frac{1}{\beta SIN{R}_{j,i}^{*}\log \left(1+SIN{R}_{j,i}^{*}/{\beta }_j\right)}-\frac{\beta /SIN{R}_{j,i}^{*}+\log \left(1+SIN{R}_{j,i}^{*}/{\beta }_j\right)}{SIN{R}_{j,i}^{*}{\beta }^{*}{\log }^2\left(1+SIN{R}_{j,i}^{*}/{\beta }_j\right)}-1$$

(19)

Because $\frac{{\partial }^2{U}_j}{\partial {\beta }^2}<0$, $U_j$ is a concave function, with the presence of optimal solution ${\beta }^{*}$. The optimal solution can be obtained through Equation (18).

In the actual heterogeneous wireless network spectrum sharing model based on game model, players generally cannot know the strategy of other players or users at the moment, hence being unable to accurately solve Nash equilibrium according to linear equation set in Formula (19). Because of this, we need to further probe into the optimization solution seeking method instead of depending on other players’ decisions at the moment. Assuming that players cannot know the strategy at the moment but know the strategy of players at the previous moment, then Nash equilibrium strategy with maximal system revenue can be obtained through iterative methods. In the next section, we will introduce one kind of intelligent gradient algorithm for completing optimization solution.

3.4. Intelligent Method Based on Gradient Algorithm

In adopting game theory to solve the issue of HetNet resource allocation, the process of obtaining strategy is dynamic. Moreover, the Nash equilibrium [31] value scope of the Stackelberg model built in the above is continuous instead of disperse, so backward induction cannot be used for obtaining solutions. In actual situations, players cannot know strategy of current moment. Therefore, we need to further discuss the intelligent algorithm without the need to know the strategy of other simultaneous players. The gradient algorithm can approach the optimal solution gradually through iteration, which is also focused. Nash equilibrium with maximal system revenue can be realized iteratively with optimization algorithm [32,33].

In solving target spectrum strategy with linear iterative method, choosing a suitable adjustment factor has been the main research direction. For authorized users, system revenue and optimal frequency band distribution factor display a concave function relationship. Assuming that ${\beta }_j^k$ is the strategy of femtocell j at the moment of k, then $U_j^k(\beta )$ is the revenue obtained by femtocell j using strategy ${\beta }_j^k$ for sharing spectrum at the moment of k. Before obtaining the maximum revenue function, the system revenue function increases with $\beta$. At the moment, $\partial U_j/\partial \beta$ is positive. Through iteration in Equation (20), strategy ${\beta }_j^k$ can be increased until system revenue function reaches the maximum. When strategy is excessively big and because the coverage of femtocells is limited, revenue function decreases with the increase of strategy value, and $\partial U_j/\partial \beta$ is negative. Iteration in Equation (20) makes strategy ${\beta }_j^k$ gradually decrease until reaching equilibrium strategy.

$${\beta }_i^{k+1}={\beta }_i^k+\mathsf{\lambda }\frac{\partial \left(U_i^k(\beta )\right)}{\partial {\beta }_i^t}$$

(20)

where, $\lambda$ is a fixed non-negative adjustment factor.

In addition, we also adopt a steepest descent approach in the optimization algorithm and choose iterations with the highest speed in making macro base station revenue function change to reach the minimum of target function as soon as possible. To solve the Nash equilibrium strategy, adjustment factors are self-adaptingly solved, so as to allow the shared spectrum price to approach Nash equilibrium price strategy along the gradient direction of authorized user system revenue function. Assuming that target function is $f(\beta )=U_j(\beta )$, then the directional function ${\Delta }^k$ of target function at strategy ${\beta }^k$ is:

$${\Delta }_i^k=\partial f({\beta }_i)/\partial {\beta }_i^k$$

(21)

Then conduct a one-dimensional search of strategy along the direction of ${\Delta }^k$, and choose target function with the steepest changes to solve adjustment factor ${\lambda }^k$.

$${\lambda }^k=\arg \underset{\lambda }{\max }f({\beta }^k+\lambda {\Delta }^k)$$

(22)

Finally, solve the femtocell strategy according to Formula (23), until the solved strategy meets the accuracy requirement.

$${\beta }_i^{k+1}={\beta }_i^k+{\mathsf{\lambda }}^k{\Delta }_i^k$$

(23)

This method can solve and obtain Nash equilibrium strategy with shorter iteration cycle. Utilizing the steepest descent thought and choosing the fastest descending direction of target function value are helpful for target function to reach the minimum of target function as soon as possible. In other words, the system revenue function solves the Nash equilibrium strategy along the route with the fastest changes, and self-adaptively solves adjustment factors and enables strategy to approach the Nash equilibrium price strategy along a gradient direction of system revenue function.

4. Simulation Results and Analyze

In this part, we conduct simulation analysis of the above algorithm performance. For the sake of convenience, we first consider a simple scenario containing one macro base station and one femtocell. Then we consider the complex network environment, where the system contains N femtocells whose distribution conforms to Poisson Point Process. The rest of the simulation parameters are shown in

Table 1. Simulation parameter. SCME: Spatial Channel Model extension.

Parameters	Macrocell	Femtocell
System bandwidth $W$	20 MHz	20 MHz
User density	100/macro	2–15/femto
Cell radius	500 m	15 m
Max transmit power of base stations $P_m(P_{F_j})$	43 dBm	20 dBm
Fast fading	SCME	SCME
Noise level $N_0$	−174 dBm/Hz	−174 dBm/Hz
$c_{th}$	10 dB	10 dB

.

Figure 3 presents the simulation analysis of the Nash equilibrium strategy under different adjustment factors with the use of a linear gradient iteration algorithm and one that uses the intelligent gradient algorithm proposed in the paper. $\alpha$ represents the optimal released spectrum ratio, and $\beta$ refers to the optimal biasing factor of femtocells. It can be seen that selecting adjustment factors plays an important role in the convergence rate of iteration solution of the Nash equilibrium price strategy. When the adjustment factor is small, strategy adjustment has a slow convergence rate; when the adjustment factor is big, fluctuation in using iteration to solve Nash equilibrium strategy is a big influence on the convergence rate. The players’ strategy of the proposed gradient algorithm is the same as the strategy of the linear gradient algorithm. Through choosing the fastest changing direction of the system revenue function, we can rapidly obtain the Nash equilibrium strategy.

The simulation result in Figure 3 has shown that the algorithm proposed in the paper can get the Nash equilibrium as same as it is obtained through the linear gradient algorithm, and we can rapidly obtain Nash equilibrium strategy. Then we consider the complex network environment, where the system contains one macro base station and nine femtocells whose distribution conforms to Poisson Point Process. Figure 4 presents the simulation analysis of the Nash equilibrium strategy. The red lines show the Nash equilibrium of femtocells that act as the followers, and the blue line shows the Nash equilibrium of the macro base station that acts as the leader. It can be seen that the Nash equilibrium strategy can be obtained through the proposed algorithm and the iteration period is short.

Figure 5 presents the error performance analysis of the Nash equilibrium strategy obtained between the proposed intelligent gradient iteration algorithm and linear iteration algorithm, from which we can see that when adjustment factor is small, the price strategy error of linear iteration algorithm becomes small as well; with increase of adjustment factor, the error between linear iteration algorithm and intelligent gradient iteration algorithm proposed in the paper is close.

Figure 6 and Figure 7 show the performance of the proposed scheme and the existing scheme. The existing scheme is the one introduced by the reference [29]. Figure 6 shows the femto users’ (FUEs’) cumulative distribution function (CDF) of the SINR. It obviously shows that the femto users’ SINR performance is better. It is because the existing scheme made improvements by defining the utility function as average throughput capacity of the macro base station and standard distortion rate of femtocells, respectively. However, each base station in HetNet was assumed as an independent individual in the game model, which attempted to seek the maximal energy efficiency without considering the feelings of other individuals in the network or the overall performance of the system. The definition method of the utility function is improved in our algorithm. The proposed scheme considers the overall performance of the system and improves the performance of the system. Moreover, the proposed algorithm especially improves the average throughput of the users when the users’ SINR is low. The proportion of users who have a lower SINR is lower. The proportion of users with less than 20 dB is only 0.05.

Figure 7 shows the average throughput of all users with an increasing number of macro users (MUEs). Figure 8 shows the average throughput of macro users with an increasing number of MUEs. The result shows that the model can effectively avoid the cross layer interference of the cell. Compared with the existing schemes, the overall performance of the system is considered on the basis of reducing the signaling overhead, and the average throughput is better than the existing schemes. The macro base station ensures the good quality of each user through the proposed scheme. With the increase in the number of macro users, the average throughput of all users decreased to a certain extent.

5. Conclusions

This paper uses the Stackelberg game model to analyze the resources allocation of double layer HetNet, and utilizes the average throughput capacity of users to build utility function with concavity. Unlike the allocation of traditional Stackelberg game models, players determine the optimal strategy for the sharing spectrum throughout the game, so as to maximize the overall system revenue. However, the strategy value scope for built system revenue function is continuous and in reality, users generally cannot know the strategy of other authorized users at the moment, the existing backward induction methods cannot be used to solve the Nash equilibrium strategy. Therefore, it is of more significance to obtain iterative solution methods without depending on other user strategies at present. The intelligent gradient algorithm proposed in the paper only utilizes player strategy at the previous moment and solves the Nash equilibrium strategy through iteration along the fastest changing direction of revenue function, hence solving Nash equilibrium in-game with a continuous value scope. Compared with the linear iteration algorithm, thanks to its dynamic and flexible choice of adjustment factors, it enjoys a shorter iteration cycle under the same conditions and avoids the failure to obtain a stable Nash equilibrium caused by the unreasonable choice of adjustment factors in the linear iteration algorithm.