Recurrent Neural Network-Based Adaptive Energy Management Control Strategy of Plug-In Hybrid Electric Vehicles Considering Battery Aging

Abstract

A hybrid electric vehicle (HEV) is a product that can greatly alleviate problems related to the energy crisis and environmental pollution. However, replacing such a battery will increase the cost of usage before the end of the life of a HEV. Thus, research on the multi-objective energy management control problem, which aims to not only minimize the gasoline consumption and consumed electricity but also prolong battery life, is necessary and challenging for HEV. This paper presents an adaptive equivalent consumption minimization strategy based on a recurrent neural network (RNN-A-ECMS) to solve the multi-objective optimal control problem for a plug-in HEV (PHEV). The two objectives of energy consumption and battery loss are balanced in the cost function by a weighting factor that changes in real time with the operating mode and current state of the vehicle. The near-global optimality of the energy management control is guaranteed by the equivalent factor (EF) in the designed A-ECMS. As the determined EF is dependent on the optimal co-state of the Pontryagin’s minimum principle (PMP), which results in the online ECMS being regarded as a realization of PMP-based global optimization during the whole driving cycle. The time-varying weight factor and the co-state of the PMP are map tables on the state of charge (SOC) of the battery and power demand, which are established offline by the particle swarm optimization (PSO) algorithm and real historical traffic data. In addition to the mappings of the weight factor and the major component of the EF linked to the optimal co-state of the PMP, the real-time performance of the energy management control is also guaranteed by the tuning component of the EF of A-ECMS resulting from the Proportional plus Integral (PI) control on the deviation between the battery SOC and the optimal trajectory of the SOC obtained by the Recurrent Neural Network (RNN). The RNN is trained offline by the SOC trajectory optimized by dynamic programming (DP) utilizing the historical traffic data. Finally, the effectiveness and the adaptability of the proposed RNN-A-ECMS are demonstrated on the test platform of plug-in hybrid electric vehicles based on GT-SUITE (a professional integrated simulation platform for engine/vehicle systems developed by Gamma Technologies of US company) compared with the existing strategy.

1. Introduction

Nowadays, the growing energy dilemma and environmental problem are initiating a revolution and innovation within the automobile industry. Hybrid electric vehicles (HEVs) have more of a degree of freedom for vehicle power distribution thanks to invertible energy storage devices and electric machines [1]. To ensure that all hybrid components work cooperatively, a lot of energy management strategies have been proposed. Generally speaking, an energy management control strategy can be categorized as a rule-based (RB) control strategy [2] and an optimization control strategy. The former is realized easily but the fine control performance is not guaranteed. The latter usually is categorized as instantaneous optimization, such as the equivalent consumption minimization strategy (ECMS) [3], model predictive control (MPC) [4], and global optimization including dynamic programming (DP) [5,6] and pontryagin’s minimum principle (PMP) [7,8,9]. DP can obtain a globally optimal solution in theory, but has a serve computational burden and cannot be used in real-time due to the requirement of knowing the global driving cycles in advance. PMP converts energy management to a minimizing Hamilton function, thus greatly reducing the computational burden and making it easier to implement. Despite this, it is still difficult to solve the numerical solution because the dynamic of the co-state is a function of the battery’s state of charge (SOC), which is nonlinear. In order to overcome the unknown driving information in advance, MPC is used to solve the energy management optimal control problem [10], which can predict driving information in the fixed prediction horizon. With the development of artificial intelligence, many intelligence algorithms have been applied to predict the driving information in MPC in order to obtain a closer to global optimum solution. For example, the neural network is used in [11] to predict demand power, which makes full preparation for the design of the energy management strategy. The radial basis function neural network (RBF-NN) is trained in [12] using engine working points that is optimized offline utilizing a distributed genetic algorithm. ECMS is applied to search the instantaneous minimum cost function and can be applied in real-time, which is evolved in PMP. Thus, ECMS can obtain more closely to a globally optimal solution by appropriately choosing the equivalent factor. [13] employed the shooting method of PMP to gain the initial co-state and then used the proportional integral (PI) controller to adjust the equivalent factor to guarantee that the SOC has a better trajectory in real time.

Most of the literature only regard minimizing fuel consumption as the optimization objective of energy management problem. In reality, the fading of a battery capcity and the shortening of its life due to the frequenct charge and discharge of a battery when the HEV is running, is inevitable. Furthermore, changing a battery before the HEV is scrapped will significantly increase the usage cost of a HEV. Many studies also have shown that battery life is responsible for the fuel economy of HEVs [14]. Thus it is necessary to consider battery life when designing an energy management control strategy. There are many factors that impact the battery life, such as a battery’s thermal management, driving conditions, environment temperature, regional climate, and so on [15]. For this, many ways have been proposed to prolong the battery life for HEVs. In [16], in order to extend the service life of the battery, ultra-capacitors are also equipped to protect the battery by optimizing the distribution of current and using ultra-capacitors to buffer the excessive charge and discharge flow of the battery. [17,18] took fuel consumption and battery capacity loss together into the cost function and solved this multi-objective optimization problem for HEV using PMP and DP, respectively. [19] derived a causal energy management strategy under consideration of battery life for HEV, which effectively reduces battery wear with a reasonable penalty on fuel economy by using ECMS. The distinguishing features of plug-in HEVs (PHEV) over the conventional HEVs are a large variation range of SOC and the repeated charge and discharge of the battery. Moreover, it has been shown that a large depth of discharge (DOD) and frequent use can accelerate the decay of battery life [20]. Consequently, research on the energy management control problem for PHEV considering the battery life is more attractive. [21] established an electrochemical mechanism model for the battery capacity attenuation of PHEV, and formulated a multi-objective optimal energy allocation problem that can be solved by shortest-path stochastic dynamic programming (SP-SDP) while achieving satisfactory vehicle fuel economy and extending battery life. [22] used a genetic algorithm to optimize the energy management strategy aimed at minimum fuel consumption and battery capacity degradation. [23] presented a model predictive control (MPC) strategy and analyzed the Pareto optimal front of the cost function comprised by the equivalent fuel consumption and battery capacity fade during the charge sustaining mode of the battery. [24] further provided the impact of the estimated SOC by the battery management system on the performance of MPC. [25] studied the nonlinear model predictive control for the energy management of a power-split hybrid electric vehicle (HEV) to improve battery aging while maintaining the fuel economy at a reasonable level. [26] employed the shooting method of PMP to obtain the optimal depth of discharge (DOD) and constructed a reference SOC with the optimal DOD, and then, a model predictive controller was used to optimize the conflict between the energy consumption cost and the equivalent battery life loss cost in a moving horizon. [27] used SDP and particle swarm optimization (PSO) to numerically solve the multi-objective optimal control problem under the consideration of the tradeoff between energy consumption and battery loss. The dynamic loop nest optimization of PSO and SDP was used to obtain offline an optimal solution according to the statistical characteristic of the real historical traffic data. The optimal solution was constructed as look-up mappings on different road segments and battery SOC so that in the online implementation of the management control strategy the power demand assignment can be obtained by these mappings without computational burden according to the current driving mode, system states, and road information.

It should be noted that it is necessary to keep a balance between fuel consumption and battery capacity loss in the design of the energy management control strategy for the economy of PHEV, while the designed management strategy should be integrated to global near optimization and the real-time performance. For this, both a globally sub-optimal and implementable energy management strategy, so-called recurrent neural network-based adaptive equivalent consumption minimization strategy (RNN-A-ECMS), is proposed in this paper for a power-split PHEV considering the battery life. Three efforts have been made. Firstly, RNN with long short term memory (LSTM) is trained utilizing the historical global optimal SOC trajectory that can be obtained by DP and real historical traffic data. Secondly, the maps of the weighting factor and main component of the ECMS’ equivalent factor (EF) depending on power demand and battery SOC are obtained by PMP and PSO utilizing the real historical traffic data. The PSO is employed to search the weight factor and co-state of PMP, ensuring the vehicle’s optimal economic performance, and the map of the PMP’s co-state is converted into the main component’s map of the ECMS’s EF. Thirdly, the maps of weight factor and the main component of the EF and the model of the well-trained offline RNN are inserted into the core structure of A-ECMS to carry out the energy management control strategy responsibilities for online implementation.

The remainder of this paper is organized as follows. The PHEV mathematical model is given in Section 2. The optimal control problem is formulated in Section 3. Then, RNN-A-ECMS is designed in Section 4. The simulation result and the comparison with SDP-PSO are given to demonstrate the effectiveness of this approach in Section 5. Finally, the conclusion is summarized in Section 6.

2. PHEV Model Description

The power-split PHEV with a planetary gear set (PGS), shown in Figure 1, is analyzed in this paper. The powertrain of the PHEV mainly consists of the engine, the battery, motor, and generator.

Considering the aerodynamic drag and rolling friction force, the longitudinal dynamic equation of the vehiclecan be written as [28]:

$$M\dot{v}=\frac{{\eta }_f{T}_{trac}-T_{br}}{R_{tire}}-Mg\left({\mu }_{r}cos\alpha +sin\alpha \right)-\frac{1}{2}\rho A{C}_d{v}^2,$$

(1)

where M is the PHEV mass. v denotes the velocity of the PHEV. ${\eta }_f$ is the transmission efficiency of the differential gear. $T_{trac}$ and $T_{br}$ are the traction torque and brake torque, respectively. g is the gravity acceleration. ${\mu }_r$ is the coefficient of rolling resistance. $\alpha$ is the grade of the road. $\rho$ is the air density. A is the frontal area of vehicle. $C_d$ is the drag coefficient.

The PGS containing the sun gear, carrier gear, and ring gear respectively connecting to the generator, engine, and motor is a core component of PHEV, which allows the PHEV to run not only the series mode in which the engine provide power to generator to charge the battery or to motor through an inverter to drive the vehicle but also the parallel mode in which the engine directly propels the vehicle together with the motor. Under the assumption of rigid connections in the powertrain and without friction loss, the PGS speed and torque relationships are described as:

$$T_r=\frac{R_r}{R_r+R_s}{T}_c,T_s=\frac{R_s}{R_r+R_s}{T}_c,$$

(2)

$$\left(R_r+R_s\right){\omega }_c=R_r{\omega }_r+R_s{\omega }_s,$$

(3)

where $T_r$, $T_c$, and $T_s$ are the torques of ring gear, carrier gear, and sun gear, and ${\omega }_r$, ${\omega }_c$, and ${\omega }_s$ are the speeds of ring gear, carrier gear, and sun gear, respectively. $R_s$ and $R_r$ are the teeth number of sun gears and ring gears, respectively. With the assumption that the connecting shafts are rigid, the speed relationship between planetary gear set and powertrain is described as follows:

$${\omega }_c={\omega }_e,{\omega }_r={\omega }_m,{\omega }_s={\omega }_g,$$

(4)

where ${\omega }_e$, ${\omega }_m$, and ${\omega }_g$ are the speeds of engine, motor, and generator, respectively. In addition, the motor speed can be computed by the following equation:

$${\omega }_m=\frac{g_f}{R_{tire}}v,$$

(5)

where $R_{tire}$ and $g_f$ are the tire radius of PHEV and the ratio of differential shaft.

Energy consumption chosen as one part of optimization objective contains fuel consumption ${\dot{m}}_f$ and electricity consumption ${\dot{m}}_{elec}$ which are defined as follows.

$${\dot{m}}_{fuel}=\mathrm{BSFC}({\omega }_e,T_e)·T_e·{\omega }_e·{10}^{-5}/36,$$

(6)

$${\dot{m}}_{elec}=s·P_{elec}/H_l,$$

(7)

where $T_e$ is the torque of engine. BSFC is brake specific fuel consumption. $H_l$ is the lower heating value of the fuel. s is EF. $P_{elec}$ is the total battery power, which can be expressed as follows:

$$P_{elec}=P_b+P_l=P_b+I_b^2·R_b,$$

(8)

where $P_l$ is the internal loss power of the battery, $I_b$ and $R_b$ are the current and equivalent internal resistance, respectively. $P_b$ is the electrical load at the terminals, which can be written as follows:

$$P_b={\eta }_m^{k_m}{T}_m{\omega }_m+{\eta }_g^{k_g}{T}_g{\omega }_g,$$

(9)

where ${\eta }_m$ and ${\eta }_g$ are the efficiency of the motor and generator, respectively. $T_m$ and $T_g$ are the motor torque and generator torque, respectively. $k_m/k_g=1$ when the motor/generator is in a discharging state, otherwise $k_m/k_g=-1$.

Moreover, it has been shown the battery performance is affected by both storage time and usage, often categorized as Calendar life and Cycle life. Calendar life is the ability of the battery to withstand degradation over time, which may be independent of how much or how hard the battery is used. While, cycle life includes deep cycle life and shallow cycles. Deep cycle life is the number of discharge-recharge cycles the battery can perform in the charge-depleting (CD) mode. For example, one complete deep discharge with starting at $90\%$ SOC, ending at $30\%$ SOC, and recharging back to $90\%$ SOC would complete one full cycle. Shallow cycles refer to SOC variations of only a few percent. These smaller variations occur throughout CD and charge-sustaining (CS) mode because the battery frequently takes in electric energy from the engine via a generator and from regenerative braking, and passes energy to the electric motor to power the vehicle. These frequent shallow cycles cause less degradation than deep cycles, but still affect battery life. Therefore, the management to the battery shallow cycle in operating modes should also be considered in order to minimize the battery life degradation in the discharge/charge cycles together with the minimization of energy consumption. There are many factors affecting battery life, such as temperature, Ah-throughput, and depth of discharge. With usage, battery performance in power and energy capacity can degrade. To get the depletion degree of battery capacity, the effective Ah-throughput ($A{h}_{eff}$) is defined as [14,20]:

$$A{h}_{eff}\left(t\right)={\int }_0^t\sigma (I_b,T_{batt},\mathrm{SOC})·\left|I_b\left(t\right)\right|dt,$$

(10)

where $T_{batt}$ is the battery temperature. $\sigma (·)$ is called as severity factor, which describes the aging effect of any cycle the battery undergoes with respect to the nominal cycle, which is described as follows.

$$\sigma (I_b,T_{batt},\mathrm{SOC})=\frac{\gamma (I_b,T_{batt},\mathrm{SOC})}{\mathsf{\Gamma }}=\frac{{\int }_0^{EOL}\left|I_b\left(t\right)\right|dt}{{\int }_0^{EOL}\left|I_{nom}\left(t\right)\right|dt},$$

(11)

where $\gamma (·)$ is the battery duration (Ah-throughput) corresponding to a given sequence of current, temperature, and SOC. $\mathsf{\Gamma }$ is the total Ah-throughput corresponding to the nominal cycle, called as the nominal battery life, which is expressed as:

$$\mathsf{\Gamma }={\int }_0^{EOL}\left|I_{nom}\left(t\right)\right|dt,$$

(12)

where $I_{nom}$ is the current profile under nominal conditions and $EOL$ denotes the battery end of life. The battery life is regarded as the end when $A{h}_{eff}=\mathsf{\Gamma }$. Then, it may be regarded that the capacity loss is as $Q_{loss}\%=A{h}_{eff}/\mathsf{\Gamma }$. Thus, prolonging the battery life is equivalent to decreasing the depletion degree of battery capacity, which is also equivalent to minimizing the effective Ah-throughput. It should be worth mentioning that the severity factor $\sigma (·)$ in this paper is obtained by the same approach as in [27], the fitting based on the experimental data.

Moreover, SOC dynamics in the energy management problem can be described as follows:

$$\dot{\mathrm{SOC}}=-\frac{I_b}{Q_b}=-\frac{P_b}{Q_b·U_{OC}},$$

(13)

where $U_{OC}$ is open circuit voltage, and $Q_b$ is the battery capacity.

3. Optimization Problem Formulation

The energy management control for PHEVs is actually an optimization problem, which in this paper is to distribute power among the engine, motor, and generator meeting the power demand of the driver while minimizing energy consumption and prolonging battery life. To this end, the whole objective function can be written as follows:

$$J={\int }_0^{t_f}\left\{(1-\theta \left(t\right))\frac{{\dot{m}}_f\left(t\right)+{\dot{m}}_{elec}\left(t\right)}{\mathsf{\Omega }}+\theta \left(t\right)\frac{\sigma \left(t\right)·\left|I_b\left(t\right)\right|}{\mathsf{\Lambda }}\right\}dt,$$

(14)

where $\theta \left(t\right)\in [0,1]$ is a weight factor balancing two contradict objectives. $\mathsf{\Omega }$ and $\mathsf{\Lambda }$ are introduced to make normalization, which are the optimal energy consumption with no consideration of battery loss, and the target effective Ah-throughput only the considering battery loss, respectively. The optimization is to calculate the control input u of the motor torque and the generator speed:

$$u^{\ast }=\left[T_m^{\ast },{\omega }_g^{\ast }\right]=arg\underset{u\in U}{min}J$$

(15)

subject to the dynamic constraint in Equation (13) and the physical conditions:

$$\left\{\begin{array}{c}{\mathrm{SOC}}_{min}\le \mathrm{SOC}\le {\mathrm{SOC}}_{max}\hfill \\ {\omega }_{e,min}\le {\omega }_e\le {\omega }_{e,max}\hfill \\ {\omega }_{m,min}\le {\omega }_m\le {\omega }_{m,max}\hfill \\ {\omega }_{g,min}\le {\omega }_g\le {\omega }_{g,max}\hfill \\ T_{e,min}\left({\omega }_e\right)\le T_e\le T_{e,max}\left({\omega }_e\right)\hfill \\ T_{m,min}\left({\omega }_m\right)\le T_m\le T_{m,max}\left({\omega }_m\right)\hfill \\ T_{g,min}\left({\omega }_g\right)\le T_g\le T_{g,max}\left({\omega }_g\right)\hfill \end{array}\right..$$

(16)

It should be noted from Equations (7) and (14) that the determinations of two factors $\theta ,s$ play an important role in satisfying the objective and seeking the optimal solution in the actual operation of PHEV.

The weight factor $\theta \left(t\right)$ should balance the energy consumption and the battery aging in real time according to the operating mode and current state of the vehicle, which yields a Pareto front. Consequently, the determination of $\theta \left(t\right)$ is also an optimization problem. For this, in this paper, $\theta \left(t\right)$ will be offline optimized by PSO and PMP utilizing historical traffic data, and then established as a map table about the battery SOC and power demand.

The EF $s\left(t\right)$ should be chosen to be linked to the optimal co-state of the PMP for guaranteeing more closely the global optimal solution of the energy management control problem. For this, in this paper, the major component of the $s\left(t\right)$ is determined by the optimal co-state of the PMP that is optimized offline by PSO, and the tuned component will then be obtained by the PI controller of the deviation between the actual SOC and the reference trajectory of he SOC from a well-trained RNN by historical traffic data.

4. RNN-Based Adaptive Energy Management Strategy

In order to ensure that the designed management strategy is integrated, for global near optimization and real-time performance, in this paper, ECMS is used as the core algorithm of energy management. Considering the uncertainty of the driving condition, a RNN-based adaptive ECMS (RNN-A-ECMS) energy management strategy is designed. The design process of RNN-A-ECMS can be divided into the offline design part and the online implementation part.

The offline design includes two parts. One is the offline training of the recurrent neural network (RNN), in which the base set of the reference SOC is first generated by the DP algorithm using historical speed profile data, and then the RNN is trained by the reference SOC from the base set, as well as road and vehicle speed information extracted from historical traffic information. The other is offline optimizing the weight factor $\theta \left(t\right)$ and the co-state of the PMP corresponding to the major component of EF $s\left(t\right)$ using PSO, and then to establish the maps of the weight factor and the major component of the EF of A-ECMS.

In the online part, the implemented energy management control strategy includes three parts: The core ECMS with the weight factor $\theta \left(t\right)$, mapping table considering the battery life, the adaptive mechanism of EF $s\left(t\right)$ combining the main component mapping table and a PI controller, and a well-trained RNN generating the SOC reference according to current traffic information and vehicle states. The sketch of the design process is shown in Figure 2.

It is worth mentioning that in the actual operation of PHEV the energy management control is not complicated and has no computing burden, but the near-global optimality can be guaranteed because of the existence of RNN and the adaptability of the equivalent factor and weighted factor.

The details of the offline design for RNN predicting the SOC reference and maps of the weight factor and the major component of EF are as follows.

4.1. Prediction Model of SOC Reference of RNN

PHEV is different from HEV with the distinguishing feature of a larger battery capacity and being recharged from the power grid. Thus, it is favorable for fuel consumption and battery health to plan ahead a reasonable reference SOC trajectory, which is obtained based on the historical data and current traffic information. More specifically, the average speed $\overline{v}$ of current driving information, distance D, and the SOC of the previous step are chosen as the input of RNN. The average speed ($\overline{v}$) is equal to the distance traveled every ten seconds of PHEV divided by the time. The SOC of the previous step is obtained by the DP optimization and historical traffic data. DP is the global optimization algorithm that can convert the continuous optimization control problem into finding an optimal decision problem for n segments under the known driving cycles. Therefore, by using the historical driving cycles of traffic data, the optimal SOC trajectory obtained from the optimal control sequence of DP optimization can be served as the required SOC of the previous step. At this point, the detail is as follows:

Optimization is regarded as the search for a control decision variable so as to minimize the cost function in Equation (17) while satisfying the constraint condition of Equation (16):

$$J=\sum _{k=0}^{n-1}L\left[x\left(k\right),u\left(k\right),k\right]=\sum _{k=0}^{n-1}{\dot{m}}_{fuel}\left(k\right)·∆t,$$

(17)

where n and L represent the duration of the driving cycle and instantaneous fuel consumption, respectively. $∆t$ is the increment of time step and chosen as 1s in order to alleviate the computational burden. According to the Behrman’s optimal principle, the optimal control problem described as Equation (17) can be decomposed into a series of single level decision problems. The specific steps are as follows:

$$Stepn-1:J^{\ast }\left[x(n-1)\right]=\underset{u(n-1)}{min}\left\{L\left[x(n-1),u(n-1),n-1\right]+G\left[x\left(n\right)\right]\right\},$$

(18)

$$Stepk(0\le k\le n-2):J^{\ast }\left[x\left(k\right)\right]=\underset{u\left(k\right)}{min}\left\{L\left[x\left(k\right),u\left(k\right),k\right]+J^{\ast }\left[x(k+1)\right]\right\},$$

(19)

where $J^{\ast }\left[x\left(k\right)\right]$ is the optimal cost of the step k. $x\left(k\right)$ is the state variable SOC. $G\left[x\left(n\right)\right]$ is the cost of the step n. The solution of DP algorithm can be divided into the backward and forward process. In the backward process, optimal control u of each step is solved reversely according to the Equations (18) and (19). In the forward process, the optimal control sequence solved by the backward process is substituted into the system state equation to calculate the optimal SOC trajectory, which is regarded as the base sets of training RNN prediction model.

As SOC, distance, and vehicle speed are a sequence about time, RNN is chosen as the predictive model. Although it might be difficult to learn long-term dependence due to the vanishing gradient problem resulting from the gradient propagation of the recurrent network over many layers, Long Short Term Memory (LSTM) can overcome the gradient disappearance in the basic RNN when it introduces a forgetting mechanism. Thus, LSTM can more accurately predict the SOC reference than the basic RNN.

The structure of RNN with LSTM is shown in Figure 3, where $x_i$ represents the input of LSTM including the average speed ${\overline{v}}_i$, distance $D_i$, and the SOC of the previous step. ${\mathrm{SOC}}_i$ is the output of LSTM representing the current SOC reference. The bottom of Figure 3 is the relationship among hidden layers which are named long and short term memory units.

The memory cell $c_t$ that retains data of the time step $(t-1)$ plays a important role in the LSTM model. Keeping the value or resetting the value of the cell $c_t$ is managed by several gates. Specially, forgetting, reading, and outputting the new cell value are decided by the forget gate $\left(f_t\right)$, input gate $\left(i_t\right)$, input modulation gate $\left(g_t\right)$, and output gate $\left(o_t\right)$, which are defined as follows:

$$f_t=\mathcal{F}\left(\left(\begin{array}{cc}{w}_{hf}& w_{xf}\end{array}\right)·\left(\begin{array}{c}h[t-1,:]\\ x[t,:]\end{array}\right)+b_f\right),$$

(20)

$$i_t=\mathcal{F}\left(\left(\begin{array}{cc}{w}_{hi}& w_{xi}\end{array}\right)·\left(\begin{array}{c}h[t-1,:]\\ x[t,:]\end{array}\right)+b_i\right),$$

(21)

$$g_t=tanh\left(\left(\begin{array}{cc}{w}_{hg}& w_{xg}\end{array}\right)·\left(\begin{array}{c}h[t-1,:]\\ x[t,:]\end{array}\right)+b_g\right),$$

(22)

$$o_t=\mathcal{F}\left(\left(\begin{array}{cc}{w}_{ho}& w_{xo}\end{array}\right)·\left(\begin{array}{c}h[t-1,:]\\ x[t,:]\end{array}\right)+b_o\right).$$

(23)

Moreover, the calculations on the cell update and output are defined as follows:

$$c_t=f_t\odot c_{t-1}+i_t\odot g_t,$$

(24)

$$h_t=o_t\odot tanh\left(c_t\right),$$

(25)

where ⊙ denotes the multiplying each element and the w matrices are the network important parameters. $h_t$ is the hidden state and employed to compute the current output and the next step hidden state. The LSTM can perfect to solve the vanishing gradients. The activation function $\mathcal{F}$ and tanh are the nonlinearity functions of logistic sigmoid and hyperbolic tangent, respectively.

4.2. Control Parameters Optimization Based on PSO-PMP

To improve adaptability to the changes in driving conditions, a kind of adaptive EF $s\left(t\right)$ is selected, which can be updated by a PI controller in real-time. The specific formulation is as follows:

$$s\left(t\right)=s_0+\delta s=s_0+\left[K_p({\mathrm{SOC}}_{ref}-\mathrm{SOC})+K_i{\int }_0^t({\mathrm{SOC}}_{ref}-\mathrm{SOC})dt\right],$$

(26)

where $s_0$ is the initial value of equivalent factor $s\left(t\right)$ (major component of EF) and $\delta s$ is the tuning component of EF. SOC, SOC${}_{ref}$ represent the actual SOC and the reference SOC trajectory from the RNN, respectively. $K_p$, $K_i$ are proportional and integral coefficients, respectively, which are determined using the estimation method for the upper and lower bounds of the EF presented in [29] and calibration with trials similar to [13].

In order to more closely obtain the global optimal solution of the energy management control problem, the $s_0$ is determined by the co-state $\lambda \left(t\right)$ of the PMP optimized offline by PSO. In the PMP, minimizing the objective function is converted to minimizing the Hamiltonian function:

$$H(\mathrm{SOC}\left(t\right),u\left(t\right),\lambda \left(t\right),\theta \left(t\right))=(1-\theta \left(t\right))\frac{{\dot{m}}_f\left(t\right)}{\mathsf{\Omega }}+\theta \left(t\right)\frac{\sigma \left(t\right)·\left|I_b\left(t\right)\right|}{\mathsf{\Lambda }}+\lambda \left(t\right)·\dot{\mathrm{SOC}},$$

(27)

where $\lambda \left(t\right)$ is the co-state, and its dynamics can be described as:

$$\dot{\lambda }\left(t\right)=-\frac{\partial H\left(SOC\right(t),u(t),\lambda (t),\theta (t\left)\right)}{\partial \mathrm{SOC}}.$$

(28)

The optimal control trajectory is given by:

$$u^{\ast }=\left[T_m^{\ast },{\omega }_g^{\ast }\right]=arg\underset{u\in U}{min}H(\mathrm{SOC}\left(t\right),u\left(t\right),\lambda \left(t\right),\theta \left(t\right)),$$

(29)

where the weighting factor $\theta \left(t\right)$ and the co-state $\lambda \left(t\right)$ are obtained offline by PSO.

The flowchart of PSO is shown in Figure 4.

In PSO, considering the optimization time and convergence efficiency, the number of swarm N and the maximum iteration $k_m$ is set as 5, 15, respectively. In order to facilitate online looking-up table, SOC is discretized as $\left[0.3,0.35,0.4,0.45,0.5,0.55,0.6,0.65,0.7,0.75,0.8,0.85,0.9\right]$. As the range of power demand of the studied PHEV is 0 kW to 50 kW and usually locates in 0 kW to 30 kW, power demand is discretized as $\left[0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,35,40,45,50\right]$. Thus, in the PSO algorithm, the swarm is defined as $X_1={\theta }_{swarm},X_2={\lambda }_{swarm}$, and the initialization swarm ${\theta }_{swarm}^0$ and ${\lambda }_{swarm}^0$ are denoted as two $\left[20\times 13\times N\right]$-dimensional tensors. The velocities of ${\theta }_{swarm}$, ${\lambda }_{swarm}$ are $V_1$ and $V_2$ which are also two $\left[20\times 13\times N\right]$-dimensional tensors. The update principles of the velocities $V_1,V_2$ and positions $X_1,X_2$ during the iterations are described as follows:

$$V_{1i}^{k+1}=w{V}_{1i}^k+c_1{r}_1\left(P_{1i}^k-{\theta }_{swarm}^k\right)+c_2{r}_2\left(G_{1i}^k-{\theta }_{swarm}^k\right),$$

(30)

$$V_{2i}^{k+1}=w{V}_{2i}^k+c_1{r}_1\left(P_{2i}^k-{\lambda }_{swarm}^k\right)+c_2{r}_2\left(G_{1i}^k-{\lambda }_{swarm}^k\right),$$

(31)

$$X_{1i}^{k+1}=X_{1i}^k+V_{1i}^{k+1},$$

(32)

$$X_{2i}^{k+1}=X_{2i}^k+V_{2i}^{k+1},$$

(33)

where $i=1,2,\dots ,N$ is the current number of swarm and $k=1,2,\dots ,k_m$ is the current iteration step. w, $c_1$, and $c_2$ are particle inertia and acceleration constants, respectively. $r_1,r_2\in \left[0,1\right]$ are uniformly distributed random values. $P_1/P_2$ is the individual extremum and $G_1/G_2$ is global extremum.

Figure 5 shows the optimization result of the fitness function, which can converge to a constant. The maps of $\theta$ and $\lambda$ according to the battery SOC and the power demand are shown as Figure 6.

From Figure 6, it can be seen that both weighting factor $\theta$ and co-state $\lambda$ have a complex functional relationship with SOC and power demand. However, some qualitative conclusions can be drawn, for example, when the power demand is higher than 30 kW and SOC is less than 0.5, the weight factor has larger value. Meanwhile, when the power demand is lower than 10 kW and SOC is larger than 0.7, the co-state has a smaller value. According to the objective function defined in Equations (14) and (7), the evaluation of battery aging is more important than that of energy consumption in optimization than if the $\theta$ had a larger value, and then the engine would be used more. Similarly, if the co-state had a smaller value, namely, the corresponding EF was smaller, it means that the evaluation of fuel consumption would be more important than that of electrical consumption in the optimization of energy consumption, which would lead to more use of the battery-powered motor.

5. Simulation Verification on GT-SUITE Test Platform

The effectiveness of the proposed energy management strategy in this paper is demonstrated on the GT-SUITE test platform. The detail of the PHEV system with the proposed energy management control strategy in the simulation is shown in Figure 7. Where the PHEV model is established in GT-SUITE environment so as to more realistically simulate the real PHEV powertrain. The proposed energy management control strategy is constructed in MATLAB/Simulink (MathWorks, Natick, MA, USA) which computes the required torque of motor $T_m^{\ast }$ and the required speed of generator ${\omega }_g^{\ast }$. Then, it can send the two control variable to PHEV in real-time. In Figure 7, the module of cost function consists of energy consumption and battery life characterized as effective Ah-throughput, which are balanced by the weight factor $\theta$.

In simulation, the parameters and specifications of model components are listed in

Table 1. PHEV model parameters.

Notation	Meaning	Parameters
M	Vehicle mass	1460 kg
A	Frontal area	3.8 m${}^2$
$C_d$	Air drag coefficient	0.33
$\rho$	Air density	1.293 kg/m${}^3$
$R_{t}ire$	Radius of the tire	0.2982 m
$\mu$	Coefficient of rolling resistance	0.015
$Q_b$	Battery maximum charge capacity	6.5 Ah
$P_m,max$	Motor max power	50 kw
$P_g,max$	Generator max power	30 kw
$J_e$	Inertia of the engine crankshaft	0.16 kg · m${}^2$
$J_m$	Inertia of the motor	0.035 kg · m${}^2$
$J_g$	Inertia of the generator	0.0265 kg · m${}^2$
$g_f$	Final differential gear ratio	40,113
$R_s$	Sun gear teeth number	30
$R_r$	Ring gear teeth number	78

.

Moreover, the historical traffic data used in training RNN and offline optimization of DP and PSO is from the real driving cycles provided by [30]. Then, the driving cycles, used in testing RNN and verifying the effectiveness of the online implementation of the designed energy management control strategy, are also from the historical traffic data provided by [30]. Additionally, the Urban Dynamometer Driving Schedule (UDDS) and New European Driving Cycle (NEDC) were chosen as the test driving cycles. However, it is worth pointing out that the actual traffic routes of the HEV [30] is 28 km from home to office and back from office. In research, it is assumed that the PHEV is charged once after a day’s commute, namely, SOC is the maximum of 0.9 before going to work in the morning and SOC is about 0.4 after 28 km, close to the minimum 0.3. Accordingly, the distances of both UDDS and NEDC are too short to fully verify the effectiveness and practicability of the proposed control strategy, because the SOC change is too small for such a short distance. For this regard, both UDDS and NEDC are repeated up to 28 km.

Firstly, because the precision of the RNN model has a significant impact on the adaptability of the energy management strategy, the constructed RNN is verified by a comparison between the basic RNN and LSTM. The third week of traffic data not used in training is chosen as the testing set. Figure 8 shows the comparison between the two in three evaluation indexes, mean absolute error (MAE), mean radial error (MRE), and root mean square error (RMSE), which are defined as follows:

$$\mathrm{MAE}=\frac{{\displaystyle \sum _{i=1}^I}|F_i-T_i|}{I},$$

(34)

$$\mathrm{MRE}=\frac{{\displaystyle \sum _{i=1}^I}|F_i-T_i|/T_i}{I},$$

(35)

$$\mathrm{RMSE}=\sqrt{\frac{{\displaystyle \sum _{i=1}^I}{|F_i-T_i|}^2}{I}},$$

(36)

where I is the total number of prediction point, $F_i$ is the prediction value of the battery SOC, and $T_i$ is the true value of the battery SOC.

It is obvious from Figure 8 that the LSTM has better precision than the basic RNN. That is because LSTM introduces the forgetting mechanism based on the basic RNN, which can capture long-term dependencies.

To verify the applicability of the proposed strategy on a real driving cycle on the third Monday week dating from [30], Urban Dynamometer Driving Schedule (UDDS) and New European Driving Cycle (NEDC) were chosen as the test driving cycles. Meanwhile, to demonstrate the advantage of the proposed energy management control strategy in performance improvement, the comparison results among the proposed strategy (RNN-A-ECMS), the SDP-PSO energy management strategy proposed in [27], and the conventional charge depleting-charge sustaining (CD-CS) strategy are given based on the same simulation environment. In the CD-CS mode strategy, the threshold SOC switching from charge depleting (CD) to charge sustaining (CS) mode was set as 0.35 instead of the lowest value 0.3 so as to avoid excessive discharging of the battery.

Figure 9, Figure 10 and Figure 11 show the simulation results and comparisons among the three strategies under the three driving cycles, respectively. Figure 9a–d are the operating results of the PHEV with the proposed RNN-A-ECMS strategy, where it can be seen that the actual vehicle speed profile could greatly track the reference speed profile, see Figure 9a, which guarantees the drivability due to low power demand (Figure 9b), the engine not working long (Figure 9c), and the torque and speed of engine, motor, and generator matching with the power demand of the driver and the PHEV working in pure electric mode most of the time (Figure 9d). Figure 9e–h are the comparisons on SOC trajectory, fuel consumption, effective Ah-throughput, and equivalent fuel consumption including electricity consumption. It indicates that the proposed energy management strategy had a better control performance: The actual SOC curve was closest to the reference SOC predicted by RNN (Figure 9e). Fuel consumption, effective Ah-throughput, and equivalent fuel consumption all were much lower than that of CD-CS although the effective Ah-throughput of the proposed RNN-A-ECMS was a little more than that of the SDP-PSO, the fuel consumption of RNN-A-ECMS was much lower than that of the SDP-PSO, so that the equivalent fuel consumption of the RNN-A-ECMS was the lowest.

Similar simulation results can be seen from Figure 10 and Figure 11. Although UDDS and NEDC are not in the database used by RNN, the driveability and control performance are also satisfied in these two conditions.

It shows that RNN with LSTM have a stronger generalization ability than the basic RNN. Moreover, the weight factor and the initial value of equivalent factor are converted into a 2-dimension table, which can obtain the different $\theta$ and $s_0$ according to the real-time traffic information to adapt the different driving conditions and get the optimal solution and better control performance.

To further demonstrate the advantage of the proposed energy management control strategy in the performance improvement, the comparison results among the RNN-A-ECMS, the SDP-PSO [27], and the CD-CS strategy are given under multiple driving cycles.

Firstly, the simulation result in another driving cycle on the second Monday week dating from [30] is shown in Figure 12.

Table 2. Simulation comparison results.

Day	CD-CS			SDP-PSO			RNN-A-ECMS
	$\mathbf{FC}$	${\mathbf{Q}}_{\mathbf{loss}}$	${\mathbf{SOC}}_{\mathit{fin}}$	$\mathbf{FC}$	${\mathbf{Q}}_{\mathbf{loss}}$	${\mathbf{SOC}}_{\mathit{fin}}$	$\mathbf{FC}$	${\mathbf{Q}}_{\mathbf{loss}}$	${\mathbf{SOC}}_{\mathit{fin}}$
Mon	3.175	$6.477\times {10}^{-4}$	0.35	2.614	$6.185\times {10}^{-4}$	0.48	2.496	$6.203\times {10}^{-4}$	0.32
Tue	3.798	$7.615\times {10}^{-4}$	0.35	3.311	$7.343\times {10}^{-4}$	0.52	3.203	$7.412\times {10}^{-4}$	0.36
Wed	3.054	$7.025\times {10}^{-4}$	0.35	2.745	$6.831\times {10}^{-4}$	0.43	2.599	$6.865\times {10}^{-4}$	0.39
Thu	3.531	$6.902\times {10}^{-4}$	0.35	3.029	$6.579\times {10}^{-4}$	0.41	2.925	$6.688\times {10}^{-4}$	0.35
Fri	2.967	$7.143\times {10}^{-4}$	0.35	2.450	$6.712\times {10}^{-4}$	0.39	2.314	$6.836\times {10}^{-4}$	0.33
ave	3.305	$7.032\times {10}^{-4}$	0.35	2.829	$6.730\times {10}^{-4}$	0.45	2.707	$6.800\times {10}^{-4}$	0.35

shows the simulation comparison results of the three strategies in the driving cycles of the workdays in the second week, which include the fuel consumption per hundred kilometers FC [L/100 km], battery $Q_{loss}$, and final SOC SOC${}_{fin}$.

From Figure 12, it can be seen that the final SOC of the RNN-A-ECMS was 0.42 which is lower than the final SOC of the SDP-PSO strategy, which was 0.49, and the final SOC of the CD-CS strategy was always 0.35. The lower average final SOC could reflect that RNN-A-ECMS was more dependent on battery for driving than the SDP-PSO. It may lead to that the battery $Q_{loss}$ of RNN-A-ECMS was slightly higher than that of the SDP-PSO. However, the fuel consumption of the RNN-A-ECMS is greatly lower than that of the SDP-PSO because the battery is frequently involved in driving in RNN-A-ECMS. It indicates that RNN-A-ECMS sacrifices small battery loss to greatly increase fuel consumption. Without the optimization management for either fuel consumption or the battery aging in the CD-CS strategy, no matter what driving conditions the electrical power is first used to propel the vehicle until the CD-CS switching threshold value of the SOC and then engine works while retaining the threshold level of SOC, as a result, both the battery $Q_{loss}$ and the fuel consumption are the highest.

According to the average of the second week in Table 2, it can be calculated that the fuel consumptions of the RNN-A-ECMS and the SDP-PSO were reduced by $18.1\%$ and $14.4\%$ compared with that of the CD-CS strategy, respectively. The battery losses of the two strategies were also reduced by $3.3\%$ and $4.3\%$, respectively. Meanwhile, the fuel consumption was reduced by $4.3\%$, however, battery loss only sacrificed $1.03\%$ between the two strategies. It means that the RNN-A-ECMS calculating different weight values for different SOC and power demand could be a better solution to the multi-objective optimization problem than the SDP-PSO.

6. Conclusions

This paper proposed a novel sub-optimal and real-time energy management control strategy RNN-A-ECMS to distribute power demand between the engine and electric machines while considering the fuel consumption and battery aging. Prolonging the battery life and decreasing the fuel consumption were contradictory. Thus, the energy management strategy including battery aging should be regarded as a multi-objective optimization problem. In order to gain the Pareto optimal set, PMP and PSO were used in this paper to solve the multi-objective optimal problem offline, and the time-varying weight factor and the major component the ECMS’s EF were obtained as two maps depending on power demand and SOC. In order to enhance adaptation to uncertain driving conditions, RNN with LSTM was trained offline using historically optimal SOC trajectory resulting from DP, and a PI controller was used to form the adaptive mechanism of the adaptive EF. In the implementation of the control strategy, the values of weighting factor and the major component of equivalent factor were generated online by looking up the two maps according to the current SOC of the battery and power without computational burden. Meanwhile, the equivalent factor was adjusted by the PI controller in order to make the actual SOC trajectory close to the optimal SOC trajectory, which could ensure that the real-time energy management strategy was closer to the optimal energy management strategy. The simulation verification and comparison with the existing strategy, which were implemented on GT-SUITE test platform, showed that the proposed energy management strategy in this paper possessed the effectiveness and adaptability to various driving cycles and had the advantage in compromising multi-objective of decreasing the fuel consumption and prolonging battery life.