A Real-Time Pricing Scheme for Energy Management in Integrated Energy Systems: A Stackelberg Game Approach

Abstract

This paper proposes a real-time pricing scheme for the demand response management between one energy provider and multiple energy hub operators. A promising energy trading scenario has been designed for the near future integrated energy system. The Stackelberg game approach was employed to capture the interactions between the energy provider (leader) and energy consumers (follower). A distributed algorithm was proposed to derive the Stackelberg equilibrium, then, the best strategies for the energy provider and each energy hub operator were explored in order to maximize their benefits. Simulation results showed that the proposed method can balance the energy supply and demand, improve the payoffs for all players, as well as smooth the aggregated load profiles of all energy consumers.

1. Introduction

Conventional energy infrastructures such as electricity, natural gas, and heat networks, are mostly planned and operated independently. However, with the developments of on-site generation technologies, energy storage, combined heating and power (CHP) technologies, and demand side management, as well as information and communication technologies, modern power systems are becoming smart integrated energy systems [1,2]. The integrated energy system can be described as an energy hub where multi energy carriers can be converted, stored and distributed to meet various the energy demands of end users [3]. Given this environment, innovative business models arise and the conventional power provider evolves into an integrated energy service provider, which is able to supply electric power, natural gas, heat, cooling, etc. to energy consumers, including energy hubs [4].

Traditionally, utility companies increase the power supply level to maintain a balance of electricity supply and demand when the load demand is high. With the help of advanced metering and bi-directional communication infrastructures, the demand response (DR) is a promising method to reshape the load profiles and balance the electricity supply and demand [5,6,7,8]. However, in the context of smart integrated energy systems, the energy demands of consumers becomes diversified. For example, energy hubs need to purchase electricity and natural gas from the integrated energy provider (IEP) simultaneously. The IEP can develop optimal electricity and natural gas price strategies to balance the supply and demand, as well as take advantage of the synergies between power and natural gas. This is called an integrated demand response, in which energy customers not only can shift their loads but also switch the energy types they consume [9].

However, energy pricing schemes are still an open issue for integrated energy providers. Most of the existing research focuses on the electricity pricing mechanism and power demand response. For example, in [10], a Stackelberg game method is used to describe the power demand response and electricity trading between one power provider and multiple consumers. In [11], the demand side management in a smart grid is formulated as two non-cooperative games to investigate the interactions between electricity providers and customers. A game-theoretic approach to optimize the time-of-use electricity pricing is proposed in [12]. Furthermore, an efficient power pricing method is proposed in [13] which can prevent users’ cheating. An autonomous and distributed demand side energy management system among electricity users based on game theory is presented in [14]. In [15], a Stackelberg game between utilities and end users to maximize their own payoffs is proposed in the smart grid. In [16], a real-time pricing-based management strategy in a smart grid is proposed with a hierarchical game method. A new real-time power pricing algorithm to minimize the peak-to-average ratio (PAR) in aggregate load demand is proposed in [17]. In [18], the demand side management of multiple electricity providers and multiple electricity consumers in future smart grids is discussed based on non-cooperative game. Additionally, a flexible management strategy of wind power heating is proposed in [19] to facilitate the consumption of curtailed wind based on game theory. In [20], two energy strategies are proposed to maximize the social welfare of generators and energy consumers in the smart grid based on a Stackelberg game.

Since integrated energy systems have both better environmental and economic benefits, extensive research has focused on these systems along with the concept of the energy hub, and have mainly included energy management [21,22,23], optimal energy flow [24,25], optimal configuration and planning [26,27]. Furthermore, there are several studies which discuss demand response programs for integrated energy systems with multi energy hubs. In [28], an autonomous demand response program is discussed in the smart energy hub framework, and the interactions between smart energy hubs are formulated as a non-cooperation game. In [29], the interactions among smart energy hubs are also formulated as a non-cooperation game, and the potential game method is employed to optimize the strategies for each energy hub. In [30], the strategic operation of the energy hubs in a competitive electricity market is investigated, time of use and dynamic power pricing schemes are compared, and an efficient algorithm for the energy management of energy hubs is proposed. Also, in [11], a demand side management game among a group of smart energy hubs is proposed. However, these studies are mainly focused on the interactions between energy hubs, most of which are formulated as non-cooperative games. The interactions between energy providers and energy hubs are far from fully investigated.

In this paper, a multi energy pricing scheme is proposed to describe the interactions between an integrated energy provider and multiple integrated energy consumers. The main contributions are summarized as follows.

• A multi energy trading framework including one integrated energy provider and multiple integrated energy consumers, is proposed.
• A price-based energy management strategy is proposed to manage the electricity and gas trading between the integrated energy provider and smart energy hub operators.
• A Stackelberg game is proposed to capture the interactions between the integrated energy provider (leader) and smart energy hub operators (followers).
• A distributed algorithm between the integrated energy provider and energy hubs is proposed to derive the Stackelberg equilibrium, through which the optimal strategies for each player can be determined and the balance of energy supply and demand can be kept.

The rest of this paper is organized as follows. Section 2 presents the model of a smart energy hub. In Section 3, the system model is modeled in detail including the formulation of the Stackelberg game and the distributed interactive algorithm is introduced. In Section 4, an illustrative example is examined to analyze the proposed method. Conclusions are drawn in Section 5.

2. Smart Energy Hubs

An energy hub is a unit where different energy resources can be converted, conditioned and stored. By integrating modern communication, information and control technologies, the energy hub evolves into a smart energy hub (SEH) [11]. Bi-directional communication infrastructure, smart energy meters and a smart energy management system (SEMS) are embedded in the SEH. The SEMS is employed to exchange real time data with the energy provider and energy consumers via the bi-directional communication infrastructure. Furthermore, the SEMS is responsible for collecting information about electricity and natural gas prices, the device status, the power and heating loads. Based on these data, the SEMS makes optimal decisions and sends control signals to coordinate the operation of the whole system [9].

Figure 1 illustrates the architecture of two typical types of smart energy hub. The type I SEH is composed of a gas turbine, electric heat pump, and transformer. The type II SEH is comprised of a gas turbine, gas boiler and transformer. The input ports of the two types of SEH are connected to the electricity and natural gas grids. The output ports can simultaneously provide power and heating energy services to end users. Coupled with power and gas networks and integrating various energy converters, the SEH can consume electricity or natural gas flexibly to satisfy the diversified energy demands of end consumers.

From the perspective of integrated energy suppliers, the energy hubs are immediate electricity and natural gas consumers. When the electricity tariff is high, the energy hubs tend to consume more natural gas, and vice versa. Thus, the electricity load demand from the electricity utility can be reduced in peak periods and increased in valley hours by optimizing the multiple input energy carriers. For the end energy consumers, they are fed by the output energy of energy hubs immediately. Thus, their thermal demands can be provided by the energy hubs instead of their air conditioners. Also, their hot water and heating loads can be supplied by the energy hubs without working their water heaters and heat pumps. Equivalently, the electricity demands are effectively reduced. Thus, from the viewpoint of end energy users, their power and heating demands can be satisfied without changing their electricity consumption behaviors and violating their comfort levels. Of course, the conventional demand response programs (e.g., energy shifting) still work. Thus, in an integrated demand response (IDR) program, energy consumers cannot only shift or curtail their energy consumption, but can also switch the consumed energy type. By taking advantages of complementarities of multi-energy carriers, the IDR becomes more flexible, economic and potential.

Take the input power of each energy converter as variables, then energy flows of the type I SEH can be described by (1) and (2), and the type II SHE can be formulated as (3) and (4).

$$\left[\begin{array}{c}{L}_{n,t}^e\\ L_{n,t}^h\end{array}\right]=\left[\begin{array}{ccc}{\eta }_n^T& 0& {\eta }_n^{GT,e}\\ 0& CO{P}_n^{EHP}& {\eta }_n^{GT,h}\end{array}\right]\left[\begin{array}{c}{E}_{n,t}^T\\ E_{n,t}^{EHP}\\ G_{n,t}^{GT}\end{array}\right]$$

(1)

$$\{\begin{array}{l}{E}_{n,t}^{in}=E_{n,t}^T+E_{n,t}^{EHP},G_{n,t}^{in}=G_{n,t}^{GT}\\ 0\le E_{n,t}^{in}\le \overline{E_n^{in}},0\le G_{n,t}^{in}\le \overline{G_n^{in}}\\ 0\le E_{n,t}^T\le \overline{E_n^T},0\le E_{n,t}^{EHP}\le \overline{E_n^{EHP}}\\ 0\le G_{n,t}^{GT}\le \overline{G_n^{GT}}\end{array}$$

(2)

$$\left[\begin{array}{c}{L}_{n,t}^e\\ L_{n,t}^h\end{array}\right]=\left[\begin{array}{ccc}{\eta }_n^T& {\eta }_n^{GT,e}& 0\\ 0& {\eta }_n^{GT,h}& {\eta }_n^{GB}\end{array}\right]\left[\begin{array}{c}{E}_{n,t}^T\\ G_{n,t}^{GT}\\ G_{n,t}^{GB}\end{array}\right]$$

(3)

$$\{\begin{array}{l}{E}_{n,t}^{in}=E_{n,t}^T,G_{n,t}^{in}=G_{n,t}^{GT}+G_{n,t}^{GB}\\ 0\le E_{n,t}^{in}\le \overline{E_n^{in}},0\le G_{n,t}^{in}\le \overline{G_n^{in}}\\ 0\le E_{n,t}^T\le \overline{E_n^T},0\le G_{n,t}^{GT}\le \overline{G_n^{GT}}\\ 0\le G_{n,t}^{GB}\le \overline{G_n^{GB}}\end{array}$$

(4)

A part of the power and heating loads of end users is flexible, thus, it can be shifted or adjusted according to demand response signals. In this paper, the energy shifting of power and heating loads are considered and can be expressed as follows [31].

$$\{\begin{array}{l}(1-R_x)L_{n,x}^{0,t}\le L_{n,x}^{dr,t}\le (1+R_x)L_{n,x}^{0,t}\\ {\displaystyle \sum _{t=0}^T{L}_{n,x}^{dr,t}}={\displaystyle \sum _{t=0}^T{L}_{n,x}^{0,t}}\end{array}$$

(5)

where $x\in \left\{e,h\right\}$ is the index of load type, $e$ and $h$ denote electricity and heating load, respectively, $R_x$ denotes the load shifting ratio; the first term of Equation (5) denotes that the load shift should keep within its limits, the second term of Equation (5) denotes the sum of load demands that are not changed after the demand response, $L_{n,x}^{dr,t}$ and $L_{n,x}^{0,t}$ are the load x after and before demand response, respectively.

3. The System Model

Figure 2 illustrates a smart integrated energy system consisting of one integrated energy provider and N smart energy hub operators (SEHO). The energy provider might be an energy retailer or an alliance of electricity and natural gas utility companies. The energy provider purchases electricity and natural gas from the wholesale market and sells them to the smart energy hub operators in a retail market. Each smart energy hub is equipped with two-way communication infrastructure, smart energy meters and smart energy management system. Bi-directional communication is available between the energy provider and a cluster of SEHOs through a local area network (LAN).

In general, the SEHO is responsible for the normal operation of the SEH and the energy consumption management of end users. One of the most important tasks for the SEHOs is to accurately forecast the power and heating aggregate load demands. Once information has been received, such as the aggregate load demand, the electricity and natural gas prices broadcasted by the energy supplier, then the SEHOs will decide the amount of purchased electricity and natural gas from the energy provider, and coordinate the operation of energy conversion devices to meet the load demands of end users and maximize their revenues.

An integrated demand response program is implemented in this integrated energy system, the real-time electricity and natural gas prices are broadcast to all the SEHOs by the IEP to encourage them to optimize energy consumptions. In the following sections, a more detailed interaction model between one energy provider and a cluster of smart energy hubs will be introduced.

3.1. Integrated Energy Provider Model

The integrated energy provider is assumed to be an alliance of ab electricity utility company and a natural gas utility company. It can supply the electricity and natural gas to energy hubs simultaneously. In general, the energy supply cost is a monotonically increasing function of the amount of supplied energy. Without losing generality, a quadratic function is taken to formulate the energy cost at time slot t [9,10].

$$C_t(g_t)=a_t^X\cdot {(g_t)}^2+b_t^X\cdot g_t+c_t^X\begin{array}{cc},& a_t^X>0\end{array}$$

(6)

where $a_t^X$, $b_t^X$ and $c_t^X$ are cost coefficients, which can be predetermined by the energy provider. The $g_t$ is the total electricity $E_t^{to}$ or the total natural gas $G_t^{to}$ supplied by the energy provider at time slot $t$. It is obvious that the energy cost function is strictly convex.

In the IDR program, the real-time electricity price $p_{e,t}$ and the real-time gas price $p_{g,t}$ are determined by the difference in supply and demand, which are broadcast to the smart energy hub operators by the energy provider to encourage them to adjust their energy consumption behaviors.

The payoff ${\mathit{U}}_{EP}^{\mathit{t}\mathit{o}}(E_t^{to},G_t^{to})$ of energy provider during a day time can be defined as a function of $E_t^{to}$ and $G_t^{to}$:

$${\mathit{U}}_{EP}^{\mathit{t}\mathit{o}}(E_t^{to},G_t^{to})=E_t^{to}{p}_{e,t}+G_t^{to}{p}_{g,t}-C_t(E_t^{to})-C_t(G_t^{to})$$

(7)

where $E_t^{to}{p}_{e,t}$ and $G_t^{to}{p}_{g,t}$ are the profits from selling electricity and natural gas to the energy hub operators at time slot $t$, respectively. $C_t(E_t^{to})$ and $C_t(G_t^{to})$ are the electricity supply cost and natural gas supply cost at time slot $t$, respectively.

Once the real-time energy prices $p_{e,t}$ and $p_{g,t}$ are determined, the energy provider aims to supply the smart energy hub operators a certain amount of electricity and natural gas to maximize his profits. The optimal strategy profiles of the energy provider can be obtained by solving the following optimization problem ((8)–(9)):

$${(E_t^{to},G_t^{to})}^{*}=\arg \max ({\mathit{U}}_{EP}^{\mathit{t}\mathit{o}}(E_t^{to},G_t^{to}))$$

(8)

$$s.t.\{\begin{array}{l}{\displaystyle \sum _{n\in N}{E}_{n,t}^{in}}\le E_t^{to}\le E_{t,\max }^{to}\\ {\displaystyle \sum _{n\in N}{G}_{n,t}^{in}}\le G_t^{to}\le G_{t,\max }^{to}\end{array}$$

(9)

The constraint (9) guarantees that the electricity and gas supplied by the energy provider can meet the demands of all the energy hub operators. $\mathit{N}=[1,\cdots ,n,\cdots ,N]$ denotes the set of smart energy hub operators. The energy pricing mechanism based on the supply and demand will be discussed in detail in Section 3.3.

3.2. The Smart Energy Hub Operator Model

The utility function of each smart energy hub operator $n$ can be expressed as:

$${\mathit{U}}_{\mathit{n}}(E_{n,t}^{in},G_{n,t}^{in})={\omega }_{n,t}(E_{n,t}^{in})-E_{n,t}^{in}{p}_{e,t}{\omega }_{n,t}(G_{n,t}^{in})-G_{n,t}^{in}{p}_{g,t}$$

(10)

where $E_{n,t}^{in}{p}_{e,t}$ and $G_{n,t}^{in}{p}_{g,t}$ are the payments of operator $n$ for consuming electricity $E_{n,t}^{in}$ and natural gas $G_{n,t}^{in}$ at time $t$, respectively. ${\omega }_{n,t}(E_{n,t}^{in})$ and ${\omega }_{n,t}(G_{n,t}^{in})$ denote the benefits for the satisfaction of energy consumption. Without losing generality, a quadratic function can describe the benefits gained by consuming electricity or natural gas [9,10].

$$\begin{array}{ccc}{\omega }_{n,t}(X_{n,t}^{in})={\alpha }_{n,t}^X{X}_{n,t}^{in}-\frac{{\beta }_n^X}{2}{(X_{n,t}^{in})}^2& {\alpha }_{n,t}^X>0& {\beta }_n^X>0\end{array}$$

(11)

where $X_{n,t}^{in}$ can be $E_{n,t}^{in}$ or $G_{n,t}^{in}$. Besides, ${\alpha }_{n,t}^X$ is a consumer preference parameter characterizing customer types, which may vary with different consumers and time slots. It can be seen that the consumer with a greater ${\alpha }_{n,t}^X$ prefers to consume more $X_{n,t}^{in}$ to improve the satisfaction level of energy consumption, and ${\beta }_n^X$ is a predetermined constant.

Each smart energy hub operator optimizes their energy consumption strategies $E_{n,t}^{in}$ and $G_{n,t}^{in}$ to maximize their utility and meet the end-use energy demands, which can be obtained by solving the following optimization problems.

$$\{\begin{array}{l}(E_{n,t}^{in},G_{n,t}^{in}{)}^{*}=\arg \max [{\mathit{U}}_{\mathit{n}}(E_{n,t}^{in},G_{n,t}^{in})]\\ s.t.(1),(2),(5)or(3),(4),(5)\end{array}$$

(12)

3.3. The Energy Pricing Mechanism: Stackelberg Game Approach

With the further expansion of business, the traditional utility company has become an integrated energy provider, which can provide multi energy resources such as electricity, natural gas, fresh water, etc. to energy consumers. As illustrated in Figure 2, the energy provider simultaneously provides electricity and natural gas to N smart energy hub operators. Meanwhile, the energy provider is the electricity and natural gas price setter. Through setting the electricity and gas prices, the energy provider aims to balance the supply and demand as well as maximize their profits. When the energy prices are broadcast to the SEHOs, the SEHOs will optimize their energy consumption and determine the electricity and natural gas demands to maximize their energy consumption satisfaction. In contrast, the constraint (9) shows that the adjusted energy demands of SEHOs will affect the amount of electricity and gas supplies of the energy provider. In other words, the energy provider can use the energy price as a weapon to induce the smart energy hub operators to participate in the IDR programs as well as keep the energy supply and demand balance.

The interactions between the IEP and SEHOs can be described with the Stackelberg game, where the IEP acts as the leader, and the SEHOs are the followers. A classic game has three elements, the players, the strategy sets and the utility or payoff functions. The one-leader, and N-followers Stackelberg game $G_S$ captures the interactions between the IEP and SEHOs and can be formally identified as follows.

$$G_S=\left\{EP\cup \mathit{N};{\mathsf{\Omega }}_{EP},{\mathsf{\Omega }}_{n\in N};{\mathit{U}}_{EP}^{\mathit{t}\mathit{o}},{\mathit{U}}_{\mathit{n}}\right\}$$

(13)

• Players $EP\cup \mathit{N}$:

The energy provider (EP) acts as the leader and the SEHOs in set $\mathit{N}=[1,\cdots ,N]$ are followers in response to the strategies of the energy provider.

• Strategy sets ${\mathsf{\Omega }}_{EP}$ and ${\mathsf{\Omega }}_{n\in N}$:

${\mathsf{\Omega }}_{EP}=\left\{(E_t^{to},G_t^{to})|{\displaystyle \sum _{n\in N}{E}_{n,t}^{in}}\le E_t^{to}\le E_{t,\max }^{to},{\displaystyle \sum _{n\in N}{G}_{n,t}^{in}}\le G_t^{to}\le G_{t,\max }^{to}\right\}$ denotes the feasible strategy set of the energy provider referring to (9), from which the energy provider determines their energy supply strategy $(E_t^{to},G_t^{to})$, namely, the daily electricity and natural gas supply vectors. Each SEHO will select their strategies $(E_{n,t}^{in},G_{n,t}^{in})$ from the feasible strategy set ${\mathsf{\Omega }}_n=\left\{(E_{n,t}^{in},G_{n,t}^{in})|s.t.(1),(2),(5)or(3),(4),(5)\right\}$, which is defined by constraints (1), (2), (5) or (3), (4), (5).

• Utility or payoff functions ${\mathit{U}}_{EP}^{\mathit{t}\mathit{o}}$ and ${\mathit{U}}_n$:

The payoff function ${\mathit{U}}_{EP}^{\mathit{t}\mathit{o}}$ of IEP denotes the profits gained by selecting the strategy $(E_t^{to},G_t^{to})$, and the utility function ${\mathit{U}}_n$ describes the revenue of selecting the strategy $(E_{n,t}^{in},G_{n,t}^{in})$ for each SEHO. It is obvious that the objectives of the IEP and each SEHO are to maximize their benefits (7) and (10), respectively, through adjusting their strategies.

Definition 1.

Let${\mathit{\varphi }}^{\ast }$denote the strategy vector of the energy supplier and${\mathit{\rho }}^{\ast }$ denote the strategy vector of all the smart energy hub operators. If the following conditions are satisfied, the strategy vector $({\mathit{\varphi }}^{\ast },{\mathit{\rho }}^{\ast })$ will be the Stackelberg equilibrium (SE) of the proposed $G_S$ game [10].

$${\mathit{U}}_{\mathit{e}\mathit{p}}^{\mathit{t}\mathit{o}}({\mathit{\varphi }}^{\ast },{\mathit{\rho }}^{\ast })\ge {\mathit{U}}_{\mathit{e}\mathit{p}}^{\mathit{t}\mathit{o}}(\mathit{\varphi },{\mathit{\rho }}^{\ast })\begin{array}{cc},& \forall \mathit{\varphi }\end{array}\in {\mathsf{\Omega }}_{EP}$$

(14)

$${\mathit{U}}_{\mathit{n}}({\mathit{\varphi }}^{\ast },{\mathit{\rho }}^{\ast })\ge {\mathit{U}}_{\mathit{n}}({\mathit{\varphi }}^{\ast },{\mathit{\rho }}_n,{\mathit{\rho }}_{-n}^{\ast })\begin{array}{cc},& \forall {\mathit{\rho }}_n\end{array}\in {\mathsf{\Omega }}_n\begin{array}{cc},& \forall n\in \mathit{N}\end{array}$$

(15)

where ${\mathit{\rho }}_{-n}^{\ast }$ denotes the equilibrium strategies of all energy hub operators except energy hub operator n. At the SE, no one will benefit by deviating from his strategies.

Theorem 1.

For the proposed Stackelberg game$G_S$, an unique Stackelberg equilibrium exists between the energy provider and smart energy hub operators if the following conditions are satisfied [10,15].

(1) The strategy set of each player is nonempty, convex, and compact.

(2) Each smart energy hub operator has an unique optimal best-response strategy once informed of the strategies of energy provider.

(3) The game leader energy provider will admit a unique optimal strategy profile, once given the best strategies for all the smart energy hub operators.

The proof progressions of this theorem are given in Appendix A.

3.4. Distributed DR Algorithm and Implementation

To protect the privacy of the integrated energy consumers and reduce the communication pressure [32], a distributed algorithm is developed to determine the strategy profiles of each SEHO and the IEP in the Stackelberg equilibrium (SE). Figure 3 illustrates the interactions between the IEP and the SEHOs when the IDR programs are implemented in the integrated energy system based on the real-time electricity and natural gas prices. Let k denote the iteration number. Let $p_{e,t}^k$ and $p_{g,t}^k$ denote the electricity price and natural gas price in iteration k at time slot t, respectively. The energy price update module embedded in the smart energy management system of the energy provider is responsible for updating the electricity and gas prices by the following formulas, respectively.

$$\{\begin{array}{l}{p}_{e,t}^{k+1}=[p_{e,t}^k+\gamma \left({\displaystyle \sum _{n\in N}{E}_{n,t}^{in*}}(p_{e,t}^k)-E_t^{to*}(p_{e,t}^k)\right){]}^{+}\\ p_{g,t}^{k+1}=[p_{g,t}^k+\gamma \left({\displaystyle \sum _{n\in N}{G}_{n,t}^{in*}}(p_{g,t}^k)-G_t^{to*}(p_{g,t}^k)\right){]}^{+}\end{array}$$

(16)

where ${[\cdot ]}^{+}$ is the projection onto the feasible space defined by constraints (1)–(12), $\gamma$ is the step size, and [[] has proved that there is an upper bound on the step size, a small enough step size can guarantee the convergence, in this paper $\gamma ={10}^{-4}$, which is selected by trial and error [33]. More information about the method to choose can be found in [9].

Actually, the energy price update module acts in the coordinator role between the energy provider and smart energy hub operators. As shown in Figure 3, in each iteration at time slot t, the energy price update module of the energy provider updates the prices $p_{e,t}^k$, $p_{g,t}^k$ according to the strategies of the energy provider and all the smart energy hub operators, and then broadcasts them to each smart energy hub operator and the energy supply update module. When the prices are received, each smart energy hub operator will update their energy consumption strategies $E_n^{in,*}(p_{e,t}^k)$ and $G_n^{in,*}(p_{g,t}^k)$, and send them back to the energy provider for updating the energy supply strategies $E^{to,*}(p_{e,t}^k)$ and $G^{to,*}(p_{g,t}^k)$.

The distributed algorithm is summarized in Algorithm 1. The first line is the initialization. $\delta$ denotes the iteration precision, $p_e^0$ and $p_g^0$ are the initial electricity and gas prices, respectively. The loop in lines 2–9 describe the interactions between the energy provider and the smart energy hub operators at each time slot t. In line 5, each smart energy hub receives the electricity price $p_{e,t}^k$ and gas price $p_{g,t}^k$ broadcasted by the energy provider. In line 6, each smart energy hub updates its optimal energy demands $E_n^{in,*}(p_{e,t}^k)$ and $G_n^{in,*}(p_{g,t}^k)$ according to (12), and sends them back to the energy provider while line 8 shows that the energy provider will update its optimal energy supply $E^{to,*}(p_{e,t}^k)$ and $G^{to,*}(p_{g,t}^k)$ by solving (8)–(9). Once the energy supply and energy demand strategies have been received, the energy price update module of the energy provider will update the electricity price $p_{e,t}^k$ and natural gas price $p_{g,t}^k$. In line 10, the iteration number k is updated and the stopping criterion for the algorithm is given in line 11. It is obvious that when the energy supply equals the energy demand, the energy prices will converge. That is to say, the algorithm guarantees the balance of energy supply and demand. Also, at each time slot, the real-time energy prices, the optimal energy supply and consumption strategies will be determined.

Algorithm 1: An iterative algorithm executed by the EP and SEHOs

1: Initialization: $k=0,\mathsf{\delta }={10}^{-5},p_e^0,p_g^0$. 2: While $t={\mathit{T}}_T$ 3: Repeat 4: For each SEHO $n$ 5: Receive the new electricity price $p_{e,t}^k$ and gas price $p_{g,t}^k$ broadcasted by energy provider. 6: Update the electricity consumption value $E_{n,t}^{in,k*}$ and gas consumption value $G_{n,t}^{in,k*}$ according to (12), and send them back to the energy provider. 7: End for 8: Update the amount of electricity supply $E_t^{to,k*}$ and the natural gas supply $G_t^{to,k*}$ amount by solving (8)–(9). 9: Update the electricity price $p_{e,t}^k$ and natural gas price $p_{g,t}^k$ according to (16). 10: k= k+1. 11: Until $\max \left(\left|p_{e,t}^{k+1}-p_{e,t}^k\right|,\left|p_{g,t}^{k+1}-p_{g,t}^k\right|\right)<\delta$. 12: End while.

4. Case Studies

4.1. Basic Parameters

In this section, a smart integrated energy system consisting of one integrated energy provider and four smart energy hubs are taken as an example to evaluate the performance of the proposed integrated demand response program. The first two smart energy hubs are assumed as type II, and the last two smart energy hubs are assumed as type I. Each smart energy hub is managed by one smart energy hub operator and is responsible for satisfying the power and heating demands of one residential community which accommodates about 500 persons. The simulation time cycle is divided into T = 24 time slots which represents 24 h of a day. Assume the hourly aggregate power loads $L_{n,e,t}$ and heating loads $L_{n,h,t}$ of each smart energy hub obey the normal distribution,$L_{n,e,t}~N\left({\mu }_{e,t},{\delta }_e^2\right),L_{n,h,t}~N\left({\mu }_{h,t},{\delta }_h^2\right)$[34]. The hourly aggregate power loads and heating loads of all the smart energy hubs are generated randomly from the normal distribution. The hourly mean parameters ${\mu }_{e,t}$ and ${\mu }_{h,t}$ are given in Figure 4. The standard deviation parameters for aggregate power loads and heating loads are 40 kW and 30 kW, respectively. The parameters of the two types of smart energy hubs are given in

Table 1. Parameters of smart energy hubs.

Smart EH Type	${\mathit{\eta }}_{\mathit{n},\mathit{T}}$	${\mathit{\eta }}_{\mathit{n},\mathit{G}\mathit{T}}^{\mathit{e}}$	${\mathit{\eta }}_{\mathit{n},\mathit{G}\mathit{T}}^{\mathit{h}}$	$\mathit{C}\mathit{O}{\mathit{P}}_{\mathit{n},\mathit{E}\mathit{H}\mathit{P}}$	${\mathit{\eta }}_{\mathit{n},\mathit{G}\mathit{B}}$
Type I	[0.95, 0.98]*	[0.3, 0.38]	[0.4, 0.48]	[2.5, 3.5]	—
Type II	[0.92, 0.96]	[0.28, 0.33]	[0.44, 0.5]	—	[0.85, 0.95]

¹ [a, b]* represents that the parameter obeys the uniform distribution of the range [a, b].

and all the parameters are chosen randomly from their corresponding intervals.

The coefficients ${\alpha }_{n,t}^E$ for the type I and type II smart energy hubs were chosen randomly from [4,5] and [2.5, 3.5], respectively. The coefficients ${\alpha }_{n,t}^G$ for the type I and type II smart energy hubs were chosen randomly from [3,4] and [4.5, 5.5], respectively. Two cases were considered, the energy shifting of aggregate power and heating loads of end users was not considered in case 1, but were considered in case 2. ${\beta }_n^E$ and ${\beta }_n^G$ for all the SEHs are 0.0025 and 0.002, respectively. $a_t^E=1\times {10}^{-4}$, $a_t^G=0.6\times {10}^{-4}$, $b_t^E=b_t^G=0$, $c_t^E=c_t^G=0$, $\delta =1\times {10}^{-4}$, $R_x=0.2$. The other parameters of the two types of energy hubs are given in

Table 2. Parameters of the devices in the smart energy hubs.

Type I	$\overline{E_n^T}$	$\overline{E_n^{EHP}}$	$\overline{G_n^{GT}}$
	1500 kW	500 kWh	4000 kWh
Type II	$\overline{E_n^T}$	$\overline{G_n^{in}}$	$\overline{G_n^{GB}}$
	2000 kW	5000 kW	1000 kW

.

4.2. Simulation Results and Discussions

4.2.1. Real-Time Energy Prices and Load Profiles

Figure 5 illustrates the power and gas prices before and after demand response. Figure 6 shows the aggregated electricity and natural gas loads provided by the energy provider before and after DR (Equation (5)), respectively. It can be seen that the energy prices vary with real time, proportional to the aggregated energy demands during that hour.

Figure 7 and Figure 8 show the purchased power and the purchased gas for each SEH, respectively. Also, the power and gas demand for each SEH are shifted from peak hours to off-peak hours. This is because, before the implementation of the demand response, the energy prices during peak hours are high; when the demand response is employed, all the four energy hub operators will change their energy demands, some peak loads are shifted to off-peak hours. As a result, the energy demands reduce during peak hours and increase during off-peak hours. The energy prices also vary with the total energy demands.

Besides, Figure 9 illustrates the hourly energy supply and demand of the whole integrated energy system before and after the demand response. It is clear that the supply and demand stays balanced at each time slot. This is because the pricing method in (16) can guarantee the supply and demand balance. The results demonstrate the effectiveness of this pricing method, and the integrated demand response also performs well in peak-load shifting.

4.2.2. Comparisons of the Peak-to-Average Ratio (PAR)

The energy provider can benefit from reducing the peak-to-average ratio (PAR) of the aggregated load demands. The PAR of electricity and natural gas provided by the energy provider can be expressed as:

$${\mathrm{PAR}}_E=\frac{\max E_t^{to}}{{\displaystyle \sum _t^T{E}_t^{to}}/T},{\mathrm{PAR}}_G=\frac{\max G_t^{to}}{{\displaystyle \sum _t^T{G}_t^{to}}/T}$$

(17)

Table 3. PAR of loads before and after DR.

Items	Before DR	After DR
PAR of Electricity	1.370	1.143
PAR of Natural gas	1.214	1.077

gives the PAR of loads before and after the demand response. If all of the energy hubs do not implement the demand response programs, the PAR for the electricity and natural gas loads of the energy provider is 1.370 and 1.214, respectively. If the demand response programs are applied to their end users, the PAR for the electricity loads of the energy provider is reduced to 1.143, about 16.57% less, and the PAR for the natural gas loads of the energy provider is reduced to 1.077, about 11.29% less. Thus, the profiles of electricity and natural gas loads become smooth, which demonstrates that the integrated demand response has an advantage in smoothing the load profiles.

4.2.3. The Convergence of the Demand Response Algorithm

Figure 10 shows the convergence processes of the electricity and gas prices without or with demand response of the end users. In each iteration, the IEP adjusts its energy supply strategies according to the energy consumption strategies of SEHOs, while the SEHOs adjust their energy consumption based on the real-time prices. According to (16), the interactions will stop when the supply and demand balance is satisfied. It can be seen from the results that the demand response algorithm converges very fast. The energy prices converge within 10 iterations.

Figure 11 shows the convergence processes of the payoffs of energy hubs and the cost of the energy provider. The payoffs and cost all converge within 10 iterations. Besides, the payoffs of all energy hubs will increase by considering the demand response and the cost of the energy provider will decrease by considering the demand response. Specifically, the payoffs of EH1, EH2, EH3 and EH4 before implementing the demand response are $74,307.52, $50,946.09, $67,747.64 and $55,291.27 respectively. When consider the demand response, the payoffs increase to $78,812.14, $55,352.93, $72,796.12 and $60,554.23, respectively. Additionally, the operation costs of the energy provider decrease from $48,150.61 to $46,005.76 by considering demand response. Thus, both the energy hub operators and energy providers can benefit from the integrated demand response. As discussed above, the pricing mechanism has a good convergence performance.

Figure 12 illustrates the deviations in energy prices (defined in line 11 of Algorithm 1) between two iterations at time slot 13, the deviation meets the iteration precision at the 10th iteration. When the iteration precision is satisfied, the energy prices are convergent, and the energy supply and demand is balanced. The game reaches equilibrium, neither the energy provider nor the energy operators will change their strategies. The deviations in energy prices for other time slots show the similar patterns.

5. Conclusions

In this paper, a real-time pricing scheme for energy management between one integrated energy provider and multiple energy hub operators was proposed. The interaction between the energy provider (leader) and energy consumers (follower) was formulated as a 1-leader and N-follower game. An interactive algorithm was proposed to derive the Stackelberg equilibrium, through which the best strategies of for energy provider and each energy hub operator are determined to maximize their own benefits. Moreover, the existence and uniqueness of the Stackelberg equilibrium have been proved.

Numerical results showed that the proposed demand response method based on the Stackelberg game can describe the interactions between the IEP and SEHOs and balance the energy supply and demand. Besides, this method can also improve the payoffs for players, as well as smooth the aggregated load profiles for all energy consumers. Furthermore, the pricing method has a good convergence performance and the error was no more than ${10}^{-5}$.

Renewable energy and energy storage devices were not considered in this paper, which could be considered as an extension of the current work. Also, more specific models of power or heating loads could be considered and discussed in future work.