Artificial Intelligence/Machine Learning in Energy Management Systems, Control, and Optimization of Hydrogen Fuel Cell Vehicles

Abstract

Environmental emissions, global warming, and energy-related concerns have accelerated the advancements in conventional vehicles that primarily use internal combustion engines. Among the existing technologies, hydrogen fuel cell electric vehicles and fuel cell hybrid electric vehicles may have minimal contributions to greenhouse gas emissions and thus are the prime choices for environmental concerns. However, energy management in fuel cell electric vehicles and fuel cell hybrid electric vehicles is a major challenge. Appropriate control strategies should be used for effective energy management in these vehicles. On the other hand, there has been significant progress in artificial intelligence, machine learning, and designing data-driven intelligent controllers. These techniques have found much attention within the community, and state-of-the-art energy management technologies have been developed based on them. This manuscript reviews the application of machine learning and intelligent controllers for prediction, control, energy management, and vehicle to everything (V2X) in hydrogen fuel cell vehicles. The effectiveness of data-driven control and optimization systems are investigated to evolve, classify, and compare, and future trends and directions for sustainability are discussed.

1. Introduction

The problems of air pollution, global temperature rise, and the volatility of traditional fossil fuels have impacted the globe over the past three decades. A major share of this is attributed to the logic that the world’s transportation sector still largely depends on fossil fuels to meet its energy requirements. As per the report by the International Energy Agency (IEA), the transportation sector contributes 30% of the global carbon dioxide emissions, of which almost 70% is from road-based vehicles [1,2]. In addition to the resulting emissions, excessive utilization of fossil fuels in the transportation industry is the major cause of the depletion of underground fossil fuel resources [3,4].

Research studies have shown that Internal Combustion Engine (ICE)-based automobiles are among the most significant contributors to air pollution [5]. This goes against the requirement of utilizing clean energy and achieving the objective of zero emissions, which is expected from the transportation industry in the modern world [6]. Today, Electric Vehicles (EVs) are recognized as a highly viable key to tackling the issue associated with environmental contamination and increased fossil fuel dependency [7]. The powertrain of an EV utilizes a mishmash of sources, such as ultracapacitors (UC), batteries, and fuel cells (FC) [8]. In addition to protecting the environment, EVs have reduced long-term costs associated with the operation of automobiles [9]. In terms of energy cost, EVs cost approximately 2 cents per mile, whereas studies show that gasoline-powered vehicles have a cost of around 12 cents per mile [10].

In general, an EV is more energy efficient as compared to an ICE-based vehicle [11]. According to the report published by the United States Department of Energy, ICE-based vehicles utilize only 30% of the total energy of the fuel to operate the vehicle, while this percentage is around 75% for EVs, indicating the degree to which EVs are more efficient than fossil fuel-based vehicles [11,12]. ICE-based vehicles waste 85% of the fuel energy as CO₂, which is the main culprit contributing to global warming [13,14].

Currently, EVs are more expensive than ICE-based vehicles with the same specifications. The battery and its associated energy management systems take up almost one-third of the vehicle cost. One may significantly reduce costs by using hybrid energy sources, more efficient energy storage systems, and improved Energy Management Strategies (EMS) [15]. Based on the energy sources utilized to run the vehicle, EVs are mainly categorized into Battery Electric Vehicles (BEVs), Fuel Cell Electric Vehicles (FCEVs), and Fuel Cell Hybrid types, utilizing an effective hybrid structure of battery/UC and FC-based energy source to run the vehicle [16].

This manuscript provides a comprehensive review of the recent advances in intelligent EMS for FCHEVs. We review how intelligent control systems can be effectively used to design EMS. The manuscript opens with a review of vehicle technology, followed by a brief discussion of the major energy sources and their characteristics. The discussion then progresses with the review of EMS and control strategies associated with FCEVs/FCHEVs.

2. Fuel Cell Vehicle (FCV)

2.1. Operating Principles of Fuel Cells

A vital worldwide issue, environmental pollution, may be addressed with the help of fuel cell (FC) technology since fuel cells can produce energy efficiently while not releasing any pollutants (the final product is water if hydrogen is applied as the fuel) [17,18,19]. The chemical energy of a fuel and an oxidant is directly converted into electrical energy by a fuel cell, an electrochemical device [20]. The fundamental physical composition of a single cell is made up of an electrolyte layer in contact with a porous anode and cathode on each side. Fuel cells typically produce electric current by electrochemical reactions at the electrodes in which gaseous fuels are regularly supplied to the anode (negative electrode), and oxygen from the atmosphere is continually supplied to the positive electrode (cathode) [21]. Figure 1 shows the basic concept of fuel cells [22].

The following electrochemical processes occur in a fuel cell with an acid electrolyte:

Anodic reaction: H₂ → 2H+ + 2e

(1)

Cathodic reaction: 1/2O² + 2H+ + 2e− → H₂O

(2)

The fuel cell’s total reaction results in the following production of water, heat, and electrical energy:

Overall reaction: H₂ + 1/2O² → H₂O + Water + Qheat

(3)

For ideal electric power generation, constant isothermal operation must be continual, eliminating the heat and water byproducts. Therefore, fuel cells’ efficient design and operation depend heavily on water and thermal management [23].

German scientist C. F. Schönbein developed the fuel cell’s basic operating concept in 1838. Based on this research, Welsh scientist Sir W.R. Grove exhibited the first fuel cell in 1839 [24]. Through the 20th century, further research was conducted. An English engineer named Francis Thomas Bacon exhibited the first completely functional fuel cell in 1959. His work was sufficiently outstanding to receive NASA’s approval and adoption. However, fuel cells are now used in stationary, portable, and transportation applications; the public and private sectors are gradually adopting them; they are becoming more reliant and long-lasting for operation; and they can operate using air and hydrogen produced through reformation as an oxidant and fuel, respectively.

The principle of fuel cells is to convert chemical energy into electrical energy by reverse electrolysis. The two reactants of Polymeric Electrolyte Membrane Fuel Cells (PEMFC) are hydrogen, stored in high-pressure tanks, and oxygen, extracted from the air.

Figure 2 shows fuel cell voltage and power density over current density [25]. In the case of a fuel cell, power density is highly dependent on the reaction area of the electrodes, meaning a highly porous surface area is favorable [26].

The polarization curve is mainly characterized by three types of losses, causing a decrease in voltage at different currents. These losses are activation losses, ohmic losses, and concentration losses. At low currents, activation losses arise from electrochemical reactions. Then, ion transport and electronic conduction induce ohmic losses. At high currents, the curve is characterized by concentration losses generated by mass transport. The thermodynamic cell potential U0 is the maximal voltage that can be reached in the whole operating range, as the ideal voltage is a theoretical value for an ideal process [27].

2.1.1. Fuel Cell Types

Today’s market offers a wide variety of fuel cells that can be classified based on the type of electrolyte. Their outputs of power, operating conditions, electrical efficiency, and common applications are different. Figure 3 displays the Classification of Fuel Cells (based on the type of electrolyte, power, and working temperature) [28].

PEMFCs are the most flexible and have the widest range of applications. The following are the most common technologies on the market [29]:

• Polymeric Electrolyte Membrane Fuel Cells (PEMFC);
• Direct Methanol Fuel Cells (DMFC);
• Alkaline Fuel Cells (AFC);
• Phosphoric Acid Fuel Cell (PAFC);
• Molten Carbonate Fuel Cell (MCFC);
• Solid Oxide Fuel Cell (SOFC).

Due to their high power density, quick startup time, great efficiency, low operating temperature, and simple and secure handling, PEMFCs are the most attractive choices for transport applications. Although pure hydrogen and oxygen provide the highest performance in AFCs, their short lives and intolerance to contaminants, particularly carbon oxides, limit their use in terrestrial applications. The most established fuel cell technology for use at intermediate temperatures is phosphoric acid fuel cells (PAFC). High-temperature fuel cells suitable for cogeneration and combined cycle systems include molten carbonate (MCFCs) and solid oxide fuel cells (SOFCs). In a size range of 250 kW to 20 MW, MCFCs have the highest methane-to-electricity conversion efficiency, but SOFCs are best used for base-load utility applications using coal-based gasses. The key distinctions between the market’s most popular fuel cell types are outlined in

Table 1. The most popular fuel cell types on the market.

	AFC	PEMFC	DMFC	PAFC	MCFC	SOFC
Operating temp. (°C)	<100	60–120	60–120	160–220	600–800	800–1000
Electrolyte	KOH	Nafion membrane	Nafion membrane	H3PO4	Li2CO3-K2CO3	YSZ
Charge carrier	OH−	H+	H+	H+	CO32−	O2−
Anode reaction	H2 + 2OH− → 2H2O + 2e−	H2 → 2H+ + 2e−	CH3OH + H2O → CO2 + 6H+ + 6e−	H2 → 2H+ + 2e−	H2 + CO32− → H2O + CO2 + 2e−	H2 + O2− → H2O + 2e−
Cathode reaction	½O2 + H2O + 2e− → 2OH−	½O2 + 2H+ + 2e− → H2O	3/2O2 + 6H+ + 6e− → 3H2O	½O2 + 2H+ + 2e− → H2O	½O2 + CO2 + 2e− → CO32−	½O2 + 2e− → O2−
Electrode materials	Anode: Ni Cathode: Ag	Anode: Pt, PtRu Cathode: Pt	Anode: Pt, PtRu Cathode: Pt	Anode: Pt, PtRu Cathode: Pt	Anode: Ni-5Cr Cathode: NiO(Li)	Anode: Ni-YSZ Cathode: LSM
Power	5–150 kW	5–250 kW	<5 kW	50 kW–11 MW	100 kW–2 MW	100–250 kW

[30].

2.1.2. Fuel Cells System Characteristics

Fuel cell systems have many benefits over conventional fossil fuel-powered electric generators [20]:

• Higher efficiencies;
• Low emission;
• Fast setup and modularity;
• Easier to maintain;
• Fuel adaptability.

Using sustainable fuels such as hydrogen and chemical energy directly into electrical energy, which improves overall efficiency and eliminates the system’s mechanical parts, is the fuel cell’s most significant advantage over conventional combustion engines [31].

Higher Efficiency

Since electric energy is directly (chemically) created from the fuel utilized, fuel cells have greater efficiency. The Carnot thermic cycle restrictions that plague all combustion-based electric generating systems are, therefore, applicable to the technology. For example, a fuel cell-powered vehicle is approximately twice as efficient as one driven by an internal combustion engine in light vehicles [32].

Reduced Emissions

Hydrogen-fueled fuel cell stacks only produce water, heat, and DC power. A hydrogen fuel cell stack is also emission-free, excluding high-temperature fuel cells’ manageable NOx emissions. Furthermore, the fuel cell’s ability to operate cleanly depends on how the fuel is produced (e.g., hydrogen). A complete fuel cell system produces greenhouse gases with a fuel reforming stage (e.g., CO and CO₂). This factor is pressuring academics and businesses to create clean water electrolysis-based hydrogen-generating technologies that are efficient and renewable to replace the current reformation-based ones. Systems that combine fuel cells with renewable hydrogen production are clean energy production and conversion systems that mirror what the energy sector aims for. It is important to note that certain heat engine systems seem less polluting than fuel cell systems when the emissions from the fossil fuel reformation process are considered [33]. This merely serves to highlight the relevance of the above conclusions on using renewable-based water electrolysis to produce hydrogen.

Modularity and Fast Startup

A single fuel cell may produce less than one volt of electrical potential. Thus, fuel cells are layered on top of one another and linked in a series to generate larger voltages. Repeating fuel cell units made up of an anode, a cathode, an electrolyte, and a bipolar separator plate makeup cell stacks. The intended power output and the performance of each cell determine how many cells should be in a stack. The sizes of the stacks range from a few hundred W to several hundred kW (up to several MW) [21]. Additionally, fuel cell systems often have excellent dynamic load-following properties; the fast electrochemical processes partly cause this within a fuel cell. Once more, the slower nature of the reformation process causes the load-following capabilities of the system to diminish dramatically when the fuel cell system has a fuel reformation step [34].

Low Maintenance

For the same type of fuel cells, it is generally simple to pinpoint and replace a damaged or malfunctioning cell enclosed inside a stack due to the great flexibility of generating systems. This trait results in decreased maintenance expenses [21].

Fuel Flexibility

Fuel cells have various uses, from micro fuel cells with power outputs of less than 1 W to multi-MW prime power production facilities. This is explained by their modular design and wide range of fuel cell varieties. As a result, the batteries used in consumer electronics and auxiliary vehicle power can be replaced with fuel cells. A fuel cell can replace heat engines used in transportation and power production due to their similar characteristics. The majority of renewable energy-producing methods may easily be integrated with fuel cells. Short warm-up periods are necessary for fuel cells that work in low-temperature ranges, which is crucial for portable and emergency power applications. Utilizing waste heat improves the system’s overall efficiency for fuel cells that operate in the medium- to high-temperature ranges while also providing an extra source of power output for Domestic Hot Water (DHW) and space heating in residential applications as well as CHP industrial-level applications. Methanol, methane, and hydrocarbons, such as natural gas and propane, are all acceptable fuels for reformation-based fuel cell systems. Through a process known as fuel reformation, these fuels are transformed into hydrogen. Alternative fuel cells that run on alcohol include direct methanol and direct alcohol fuel cells. Additionally, while water electrolysis produces the most hydrogen for fuel cells, a fuel cell system that uses natural gas reformation also has advantages over other methods [35]. However, even if fuel cells can offer significant benefits, the technologies are still in research. They are plagued by issues, making their usage less convenient than other technologies. Ref. [36] highlighted the following issues:

• The costs of stationary electric generation using fuel cells ((EUR /Wh)) are still too high and make them an appropriate replacement for technologies based on fossil fuels.
• There is still much to learn about the lifespan and rate of deterioration of many fuel cell technologies, particularly the high-temperature ones that are ideal for generating electricity.
• Hydrogen, one of the primary fuels for fuel cell technologies, is costly, and there is currently no system in place for its distribution and manufacture.
• The difficulty of containing a sufficient amount of hydrogen in small fuel containers and the hydrogen is a combustible and possibly explosive gas limit the usage of low-temperature fuel cells in the automotive industry (especially if compressed in small containers).

These factors prevent fuel cells from replacing many other less efficient technologies and have a more negative environmental effect.

2.2. Fuel Cell Vehicle

2.2.1. Electric Vehicle

EVs utilize electric energy storage systems, such as batteries, FCs, and UCs, as shown in Figure 4 [37]. EVs are still in their infancy and far from ICE vehicles in terms of carrying capacity, payload, boot space, etc. EVs exceed the ICE vehicles by a serious margin [38]. The main hurdles associated with using EVs include the relatively high price of the purchase, long charging duration, and short range [39,40,41,42]. EVs, despite their disadvantages, have several advantages as well. The main benefit of this type of vehicle is that it has minimal mechanical degradation due to a lack of mechanical and moving parts. However, the fact that these vehicles require high-torque traction motors reduces their overall efficiency. These motors are associated with an increased flow of current in armature windings, which causes heat loss [43,44]. To improve the driving speed and travel distance, gearbox mechanisms are introduced into EVs. However, this impacts efficiency because of mechanical energy loss associated with these mechanisms [13].

2.2.2. Fuel Cell Electric Vehicle (FCEV)

Similar to Battery Electric Vehicles (BEVs), FCEVs also utilize an electrical source to power the vehicle. Still, the difference lies in the fact that, unlike BEVs, which use a battery as the power source, FCEVs utilize an array of hydrogen fuel cells to power the vehicle [45,46]. An FC can be defined as a source of energy that utilizes a chemical reaction involving hydrogen as fuel to generate electricity. In an FC, hydrogen and oxygen are combined to produce water and energy. Since the byproducts of a fuel cell reaction emit no byproducts other than water, waste heat, and energy, FCEVs are generally categorized as zero-local emission vehicles [13]. Low pressure and temperature FCs, such as a Polymer Electrolyte Membrane Fuel Cell (PEMFC), are generally utilized for designing an FCEV since they show characteristics such as low operating temperature (60–80 °C), superior power density characteristics, and lower corrosion characteristics as compared to other fuel cell types [47,48]. In an FCEV, hydrogen is sometimes stored in a fuel tank onboard the vehicle or, in other cases, is extracted in real-time utilizing a hydrogen fuel processor [13,48,49]. The major merits of FCEVs include faster fueling, zero emissions, easy maintenance, better fuel efficiency, and a silent drive. However, their main disadvantages include challenges associated with safety, storage, and a general lack of hydrogen fuel stations and the associated cost and complexity of the technology [1]. FCEVs are ideally suitable for low-speed, constant-power vehicles and driving conditions [13].

The size and number of the FCs are utilized to determine the power generation capacity of the FC stack. Additionally, since the fuel cells generate heat and water along with electricity, a co-generation system could be employed to advance the overall system efficiency [50]. The powertrain configuration of an FCEV is shown in Figure 5. An FC system comprises an anode, an electrolyte layer, and a cathode. In an FC, hydrogen is allowed to pass through the anode, and oxygen is delivered through the cathode. In the anode, the hydrogen molecules are split into protons and electrons by using a catalyst. The protons are allowed to move through the electrolyte layer while the electrons are compelled to pass across an external circuit generating heat and electricity. At the site of the cathode, the electrons, protons, and the supplied oxygen combine to produce water. Since no moving parts are involved, FCs are quiet and highly reliable.

Of the different fuel cells, polymer electrolyte membrane fuel cells (PEMFC) are the most suitable for automotive applications. Mobile operation capability and lower working temperatures are requirements for vehicle fuel cell applications. In addition, they bring along advantages, including highly efficient operation of up to 55%, zero harmful local emissions, and low noise emissions due to no moving parts within the fuel cell [51]. Durability, cost, and heat management are internal challenges that have to be worked on, while energy management strategy has a big impact on characteristics such as durability and fuel consumption [47,51,52].

Various FCEV configurations rely on the required hybridization level, FC type utilized, and battery [48]. The system typically consists of a power source (primary), i.e., the FC system, and an ESS, such as a battery [53]. Similar to BEVs, FCEVs utilize an all-electric type of powertrain; nevertheless, the energy source is an FC stack [13].

The electricity produced in the FC stack of an FCEV usually follows any one of the two paths as per specific driving demands. It either courses to the EM and directly powers the FCEV or flows to charge the battery pack, which acts as energy storage until it is demanded. The function of the Power Distribution Unit (PDU) is to effectively distribute the power either to the battery pack or to the EM as demanded. Apart from this, a PDU also powers the associated auxiliary systems. The Peak Power Battery used in FCEVs, as continuously charged by the FC, is considerably smaller and thus lighter than the typical battery used in a BEV. A DC/DC converter is utilized to increase the voltage. The configuration also employs an inverter to achieve the DC to AC conversion to control the output of the electric drive [54]. The general configuration of an FCEV is shown in Figure 6.

2.2.3. Fuel Cell Hybrid Electric Vehicle (FCHEV)

Powertrain Configurations

This article focuses on options for combining fuel cells and batteries. Hence, two powertrain configurations for FCHEVs have been described that are mainly in use. These are series and parallel hybrid configurations. Before going into more detail, it is important to state why using a fuel cell as the only power source in a vehicle (see Figure 3) is not favorable.

If the only power source in a vehicle is a fuel cell, it must cover all loads derived from the road power demand and auxiliaries. This brings along a fluctuating power demand, leading to higher fuel cell degradation. In addition, the fuel cell’s overall efficiency is lowered due to operation at low power demands. Using a hybrid powertrain with a battery as a secondary power source, the fuel cell acts as a support power source and can run in highly efficient operation. This results in reduced overall energy consumption [51].

In both hybrid configurations, the fuel cell is connected to a unidirectional DC/DC converter that changes the output voltage of the FC to the desired voltage. Hence, the battery is connected to a bidirectional DC/DC converter since it can provide and recuperate energy (see Figure 4). Power electronics convert the DC to AC and operate the asynchronous induction motor [55].

The parallel hybrid configuration (see Figure 7) shows the fuel cell and battery adding up to provide power for the electric motor. The fuel cell usually has a higher power output, while the battery is smaller than the series configuration. A more intelligent energy management system is needed to realize the power distribution and ensure the fuel cell’s longevity while minimizing fuel consumption. Nevertheless, it can be used to decrease the size of the battery and further reduce the costs of the vehicle [56]. The parallel hybrid configuration is used in this paper as it is considered to be more efficient and future-oriented than the series hybrid. As the energy management strategy takes an important role in this hybrid powertrain, it will be discussed in more detail in the next chapter. Often, this configuration is used similarly to a range-extended vehicle with a comparably high battery capacity. The disadvantage is that a bigger battery brings more weight into the vehicle, increasing fuel consumption and total production costs. In addition to the less complex control strategy, running costs and fuel consumption can be lowered by designing the FCHEV as a Plug-In HEV [47].

Figure 8 shows a simplified powertrain structure. At first, information is forwarded from the driver to the controller using gas and brake pedals, which is then translated into a torque demand. Following, the controller establishes the power distribution between the fuel cell and battery based on the control method, which has information about the current operation point of the power sources through the EMS. The electric power is transferred to the EM, which translates it into mechanical power for the wheels.

Energy Management Strategy

Energy Management is the coordination of power sources and energy storage to minimize fuel consumption and emissions and the durability of components and systems. The main task is the determination of control values for the power sources. In the case of a hybrid powertrain, an EMS is required to control the power distribution between the fuel cell and the battery. Intelligent EMS can drastically reduce fuel consumption and thus prolong the lifespan of the power sources. More benefits are the extended range of the vehicle and fewer emissions [57].

The desired power distribution for an FCHEV is as described in previous sections. The fuel cell works efficiently under a stationary load while the battery absorbs the power peaks of the power demand. The power demand consists of road power, influenced by environmental conditions, driver behavior, internal demands, and the auxiliary system of the vehicle and fuel cell. If the EMS could adapt to various road and environmental conditions, such as road geometry, wind, and weather parameters, the vehicle performance would be improved. These influences can be further divided into slope and bend of the road, direction and speed of wind and temperature, solar radiation, and humidity in the ambient air. The environmental parameters also influence the thermal comfort of the vehicle, which is difficult to control. The air conditioning system ideally needs to be integrated since it plays a major role in the energy efficiency of auxiliaries [58].

3. Artificial Intelligence (AI) and Machine Learning (ML) Methods

Artificial intelligence is a constellation of many different technologies working together to enable machines to sense, comprehend, act, and learn with human-like levels of intelligence. Maybe that is why it seems as though everyone’s definition of artificial intelligence is different: AI is not just one thing. Technologies such as machine learning and natural language processing are all part of the AI landscape. Each one is evolving along its path and, when applied with data, analytics and automation, can help businesses achieve their goals, be it improving customer service or optimizing the supply chain [59].

3.1. Fuzzy Logic System (FLS)

The idea of FL aims to mimic human feelings and interpretation methods. Unlike the classic point-to-point type approach, FLC is a range-to-point or range-to-range control. The fuzzy controller output is obtained by fuzzifying the inputs and the defuzzification of outputs employing the associated membership functions [15]. A crisp input is first converted to fuzzy values using appropriate fuzzy membership functions. Then, the inference is made through a fuzzy inference system similar to a look-up table. The output of the fuzzy inference system is a fuzzy value. Finally, the output is de-fuzzified to obtain a crisp control action [60]. Fuzzy control systems have numerous purposes in the motorized industry, such as transmission shift control and antilock brake control [61,62].

3.2. Model Predictive Control (MPC)

MPC is a feedback control algorithm that uses the model plant to predict the future outputs of a process. It also uses the optimizer, which guarantees that the predicted plant output tracks the desired reference [63]. By solving an optimization problem, the MPC controller tries to minimize the error between the reference and the predicted output over a future horizon, possibly subject to constraints on the manipulated inputs and outputs [64,65,66].

3.3. Genetic Algorithm (GA)

The idea of the GA is founded on Darwin’s theory of evolution and is commonly utilized to find the optimum in non-convex optimization problems. The genetic algorithm can solve constrained and unconstrained optimization problems with natural selection based on biological evolution [67]. In this strategy, the population, which is a collection of candidate solutions, is made to evolve into a new set of the optimal population that relies on operators that are inspired by biological processes such as crossover, selection, and mutation [68]. During the evolution process, the fitness concept is employed to calculate the objective function; the fitter population remains, and there is more chance for it to produce more population [69]. The process of iteration is utilized to reach the optimal solution [70].

3.4. Machine Learning (ML)

Machine Learning (ML) is a type of Artificial Intelligence (AI) method that uses software applications to predict outcomes more accurately without being explicitly programmed [71]. The ML algorithms utilize previous data as input to predict new output data. There are four categories of ML methods based on what types of data need to predict [72].

Supervised learning: in this algorithm, the labeled data and defined variables correlate with allowing the software to predict the output data. The input and output data are specified in this method.

Unsupervised learning: this algorithm trains unlabeled data by scanning the data set to find any meaningful connection. The predetermined output comes from trained data to predict or recommend more data.

Semi-supervised learning: this algorithm is a mix of two previous methods. It has labeled training data, but the algorithm is free to explore through data on its own and develop knowledge of the data set [73].

Reinforcement Learning (RL): this algorithm tries to teach a machine to complete a multi-step process based on reward and punishment to complete a task. Because this algorithm utilizes for control and energy management systems in a vehicle in lots of papers, in the following section, the algorithm will define in more detail.

RL is inspired by living things, such as humans and animals, as a control system. The idea is to learn from experiences by setting rewards and punishments for each action taken [74]. In control engineering, the goal is to minimize a certain control cost, equal to maximizing the agent reward in reinforcement learning. As a result, an RL agent attempts to develop an optimal policy suitable for control theory as a heuristic method [75]. RL is one of the machine learning techniques where agents learn how to deal with the environment through the rewards or penalties associated with each action (see Figure 9). In more detail, an RL algorithm learns optimal behavior by interacting with the state through an action [76]. Once the action is complete, a reward is retrieved, which can be either positive or negative. A positive reward is an indicator of a correct action [76].

In RL, policy, value function, reward function, and the environment are the main elements of the model (see Figure 9). One of the essential parts of RL is the policy, which is defined as an agent’s behavior at a specific time. The policy is usually calculated by minimizing the value function over time. The optimal policy should be obtained through the RL process. The reward function is defined for each action the agent performs, and the agent tries to maximize its reward over time. The value function tries to predict the future reward based on the agent’s actions for a higher reward. There are two value functions: the state-value function and the action-value function. The state–value function is used for known environments, similar to Dynamic Programming (DP) methods.

On the other hand, the action–value function is used for the unknown environment, such as the Monte Carlo-based and Temporal Difference (TD) methods. The value function predicts the total future reward. In contrast, the reward function works only on the immediate reward, so all the actions will be considered depending on the state or action-value function.

As suggested in [76], dynamic programming (DP), Monte Carlo (MC), and temporal difference (TD) are three main variations of RL. DP requires an environment model, MC is entirely model-free, and TD combines DP and MC. Indeed, TD is model-free and can be easily used online with step-by-step calculation [77]. RL algorithms can also be classified into three types depending on how the optimal policy is calculated:

Value Iteration (VI) algorithms can be either model-based or model-free. VI Algorithms can find the optimal policy based on the optimal value function.

Policy Iteration (PI) algorithms evaluate policies to obtain value functions and use these value functions to improve policies. The PI algorithms also can be model-free and model-based.

Policy Search (PS) algorithms find an optimal policy with optimization techniques. PS algorithms can be either gradient-based or gradient-free.

4. AI and ML in EVs

4.1. Prediction of Fuel Cell Behaviour Using AI

The widespread adoption of fuel cells in mobile applications is limited due to their cost and durability. The Department of Energy United States of America has set the target of reducing the cost to 14 USD/kW_net and increasing the durability to 5000 h of active operation and substantial improvement in start/stop durability as well [78].

However, due to the inherent complexity of FC systems, their optimization is a multi-disciplinary challenge. It would require improvement on multiple fronts, such as material science, model-based system design, failure diagnosis, and system control. This section covers the literature on AI and ML algorithms to improve fuel cell systems’ failure prediction and fault diagnosis. The methods for predicting faults begin with system modeling. Two main system modeling approaches are widely used –model-based and data-based techniques. In addition, some hybrid strategies combine both model- and data-based methods.

The model-based methods, also known as residual-based diagnosis, differ from data-based in certain vital aspects. The fundamental approach relies on creating a system model that runs parallel to the physical system. It periodically records the residuals, i.e., the difference in the predicted and actual system behavior. This residual measurement is then used to identify and classify the fault. A system model represents the physical system, which must be characterized by different parameters that must be correctly identified to ensure high model accuracy. This type of analytical model, also called the white box model, often uses a series of algebraic and differential equations to exploit well-known physics, such as Nernst–Planck, Butler–Volmer, and Fick’s law, to capture the charge transport and mass transfer phenomena. Ref. [79] has presented a concise review of the model-based diagnosis of proton exchange membrane fuel cells. A summary combining different non-model-based fault prediction methods from [79] and [80] is shown in Figure 10.

On the flip side, non-model-based algorithms tend to circumvent the physical modeling approach by using the data obtained from extensive testing of the target system. Ref. [80] has presented a systematic classification of different data-based, also referred to as non-model-based methods. It highlights AI-based techniques and statistical procedures, such as PCA and PDA, which can be used for dimensionality reduction and purely signal-based methods. Over the years, AI and ML methods have increased considerably in the data-driven modeling of FC systems. Many novel concepts such as genetic algorithms, particle swarm optimization (PSO), artificial neural networks and deep learning, random forest method, and support vector machine have improved the state of data-driven models used for classifying and identifying different failure conditions [82]. Moreover, these advanced AI methods are often augmented by combining ML algorithms with genetic algorithms and statistical methods for feature reduction.

The methods enlisted above can be used to create FC system models that take are used to predict the behavior of a fuel cell system. The behavior can vary subject to various factors encountered while operating in a real environment, for example, temperature variation, humidity changes, and drastic changes in power demand. Using an accurate FC system model, one can accurately predict system properties, such as the state of health (SOH) or remaining useful life (RUL), or identify and classify faults in the FC system. In addition, it can sometimes predict the short-term power demand and rate power for optimal energy management.

4.1.1. State-of-Health (SOH) and Remaining Useful Life (RUL) Prediction Using AI

Over the years, many model-based methods have been used to estimate the SOH of an FC system. However, recently there has been an upward trend in the use of AI to model the incredibly complex FC degradation mechanism. Advanced AI techniques, such as Recurrent Neural Networks (RNNs), have successfully created accurate models. Raeesi et al. [83] compared the accuracy of Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs) and Long Short-term Memory (LSTM), and Bidirectional Long short-term Memory (BiLSTM). The results favored DNN in terms of model accuracy. Ma et al. [16] created an aging model using LSTM and Grid-LSTM and compared it to other data-driven models. The results showed higher accuracy and better real-time performance.

Similarly, Lin et al. [84] compared the LSTM networks with other popular AI methods such as Support Vector Regression (SVR), Gaussian Kernel–SVR (GK-SVR), and Artificial Neural Networks (ANNs) to predict the high-frequency resistance of a Fuel cell stack subject to different parameters. The HFR resistance is an important indicator for estimating degradation performance losses. The results showed that the LSTM networks generated a more accurate HFR prediction model. Moreover, using feature extraction techniques reduced the computational cost, making the model real-time feasible.

On the other hand, considerable research is being carried out to test the combination of Genetic Algorithms (GAs) with advanced AI methods for accurate modeling. For example, Chen et al. [85] compared augmented Multi-kernel Relevance Vector Regression (MRVR) with Whale Optimization algorithm (WOA), Particle Swarm Optimization (PSO), and Genetic algorithm with several conventional AI methods. The results showed that a considerable improvement was possible by using the augmented WOA + MRVR approach. Furthermore, Yue et al. [86] created an RUL prediction model that utilized particle filtering and fuzzy logic controller optimized using GA. At the same time, Chen et al. [87] combined GA with Extreme Learning Machines (GA + ELM) to predict voltage degradation over time. The resulting models have low computational cost and higher accuracy than conventional methods such as support vector machine (SVM) and Back Propagation Neural Networks (BPNNs).

Other studies have combined different AI concepts to improve accuracy and reduce computational costs. For example, Yang et al. [88] combined Multivariate Polynomial Regression (MPR) with ANN, and Huo et al. [89] combined Random Forest (RF) and Convolutional Neural Networks (CNN) to predict the performance changes in a fuel cell due to degradation. A summary of the reviewed works related to State-of-health (SOH) and Remaining Useful Life (RUL) prediction using AI in FCHEVs is listed in

Table 2. Summary of reviewed works related to State-of-health (SOH) and Remaining Useful Life (RUL) prediction using AI in FCHEVs.

Author	AI Method	Pros	Cons
Vichard et al. [90]	Echo State Neural Network (ESN)	A representative experimental test is conducted on a 1 kW fuel to simulate realistic operation as a Postal delivery vehicle (start–stop and temperature variations) Highlights the influence of ambient temperature and Energy throughput in Ah on the State-of-health (SOH) Online application possible	Lack of representative test cycles, such as NEDC and UDDC. The selected fuel is air-cooled and self-humidifying, which could have amplified the dependence between the ambient condition and SoH
Raeesi et al. [83]	Deep Neural Network (DNN) Recurrent Neural Network (RNN) Long Short-Term Memory Bi-directional LSTM (BLSTM)	Systematic comparison of different AI algorithms Experimental data used to train and compare the outputs	Inadequate information about the validation and generality of the fitted models
Ma et al. [16]	LSTM & G-LSTM Relevance Vector Machine (RVM) Non-linear Autoregressive, Elman	Rigorous testing data of 8 different FC systems Systematic improvements to a long-established RNN framework by adding LSTM nodes and Grid-LSTM Both short-term and long-term degradation is predicted is documented	The test profiles are simple steps or stationary inputs. A more dynamic loading could trigger different degradation. The use of standardized drive cycle power demand scaled appropriately could be more appropriate
Lin et al. [84]	LSTM with multi-layer perception (MLP) Linear support vector regression (L-SVR), Gaussian Kernel Support Vector Regression (GK-SVR), Artificial Neural Network (ANN)	Theoretical background on shortcomings of conventional CNNs and advantages of LSTM are well documented Experimental test data from a PROME P390 92 kW self-humidifying fuel cell system The model works better than other regression-based models By reducing the dimensionality and the computationally time	The lack of a standardized test cycle makes it difficult to compare the results with other literature The analysis understates the importance of inlet and outlet temperature and airflow rate, which contradicts the theoretical understanding
Wang et al. [91]	Navigation Sequence Driven LSTM (NSD-LSTM)	Fuel cell failure is predicted using the degradation trends Predicting RUL	Prediction is quite inaccurate at some points
Zuo et al. [92]	LSTM Gated Recurrent Unit (GRU)	Dynamic durability test data are used to test the prediction capability of the models	The lack of a standardized test cycle makes it difficult to compare the results with other literature
Yue et al. [86]	Particle filters, Fuzzy logic controller, and Genetic algorithm	Combining advanced concepts, such as particle filters for online estimation and using fuzzy logic controller optimized by Genetic Algorithm. Non-linear FC systems and battery models are considered Objective function includes fuel cell degradation, change in SoC, and H2 consumption The total cost of ownership approach to evaluate the final EMS results	Charing and discharging and SoC estimation model of the battery are straightforward Lack of experimental data and validation
Chen et al. [87]	GA + ELM Elman SVM, Adaptive Neuro-fuzzy Inference System (ANFIS)	The extreme learning method has a low computational cost. To increase the model’s accuracy, a GA algorithm is used to tune the parameters of the hidden layer. The method performs better than SVM and Elman network trained. Additionally, the prediction error is compared with the Adaptive neuro-fuzzy inference system (ANFIS)	Some other AI algorithms, such as LSTM, perform better with time-series data than SVM and Elman network. A comparison between LSTM and the proposed method would be more fruitful.
Yang et al. [88]	ANN-MPR & Rectified Linear Unit Artificial Neural Network (ANN)	Dead-ended anode (DEA) and anode recirculation are explained in detail in this paper Detailed representation of physical interaction in the GDL and electrodes is presented
Huo et al. [89]	CNN + RF Deep Neural Network (DNN)	Serious consideration of important design factors by preprocessing the data using the RF method The dataset is based on 64 high-quality research articles k-Fold cross-validation method due to the small size of the training data set	Lack of representative test cycles or loading cycles for independent comparison between different modeling approaches from the literature
Meraghni et al. [93]	Digital Twin Deep Transfer Learning (DTL) Stacked autoencoder Particle filter-based exponential empirical model	State-of-the-art DT prognostics methods and their industrial use is presented An evaluation study is carried out using real system measurements from the long-term PEMFC degradation experiment
Chen et al. [85]	Multi-kernel Relevance Vector Regression (MRVR) Whale Optimization Algorithm (WOA) Particle Swarm Optimization—MRVR (PSO-MRVR) GA-MRVR Back Propagation Neural Network (BPNN) K-Nearest Neighbors (kNNs) Support Vector Regression (SVR) Decision Tree (DT)	The proposed method is compared against several reference modeling methods, and the results show that a positive step up can be made using the MRVR approach Different combinations of MRVR with WOA, PSO, and GA are presented Both experimental and lab data are used to train multiple AI algorithms, and the results of the comparison are shown clearly

.

In summary, the degradation prediction of the fuel cell depends on many system-level parameters, material properties, control parameters, and operation strategies. To achieve an accurate estimation of system health, one must use an optimal AI approach or a combination thereof. LSTM shows good prediction capability and works well with time series data. GAs can be used to update the hyperparameters for online optimization and to improve the accuracy of data-driven models over time. Another important factor is using feature engineering to recognize essential features and reduce the model dimensions. This step can reduce the computational cost and make the algorithms real-time feasible, increasing their market useability drastically. Therefore, based on the data, a three-layer AI model can be applied to create a low-cost and accurate data-driven degradation model. The first layer is used to identify the critical model parameters using feature engineering, which is then used by the second layer comprised of popular AI algorithms, such as RNN, CNN, or DL to create a degradation model. Finally, the last layer adds another GA optimization step that can be used to optimize the hyperparameters during offline training or subsequently improve the model’s accuracy during its online application.

4.1.2. Fault Prediction and Classification Using AI

A fuel cell system comprises many interacting sub-systems and physical phenomena. For example, the water movement inside a PEM fuel cell occurs because of transport mechanisms such as Electro-osmotic Drag (EOD), which carry water from the anode to the cathode side. In contrast, Thermal-osmotic Drag (TOD) and Back Diffusion (BD) carry water in the opposite direction. In addition, Hydraulic Permeation (HP) can result in water transmission in either direction depending on the internal state of the membrane. Each of these transmission mechanisms depends on multiple system control and operating parameters such as electric load and current density, H₂ supply, coolant temperature and flow rate, air temperature and flow rate, and possibly other system properties [94]. As improper water management can result in flooding, drying, or fuel starvation of the membrane, it is crucial to predict such failures accurately.

Gu et al. [94] used LSTM networks to process the time-series data from onboard sensors to predict flooding faults in fuel cells. Ref. [94] shows the advantage of using memory-based networks compared to classic or “memory-less” networks, such as Support Vector Machine (SVM). Similarly, Zhou et al. [95] compared the SVM, BPNN, LSTM, and Wavelet Packet Decomposition (WPD) to predict flooding, drying, and starvation faults in a fuel cell system. In conclusion, based on simulated training–testing and experimental verification, the LSTM algorithm performed better than the rest in terms of accuracy and computational time in a subsequent paper.

In addition to LSTM, CNN has also shown considerable improvement in predicting flooding faults. Zuo et al. [96] created a CNN for flooding diagnosis. A batch normalization step was added to the pipeline to reduce the computational burden making the model feasible for online application. Zhou et al. [97] created a prediction model using Binary matrix Encoding and a convolutional network (BinE-CNN). The Binary encoder extracts essential features for the modeling, which is carried out by CNN using experimental data. Upon comparison, the BinE-CNN approach outperforms LSTM and SVM. Both these studies highlight the importance of feature engineering when creating online diagnosis models. Other methods, such as reservoir computing (Echo state networks), used to predict failures such as stoichiometry value fault, pressure drop, temperature drop, and failure on the cooling circuit are presented in [98] and XGBoost fault classifiers in [99]. The fault prediction and classification using AI in FCHEVs in listed in

Table 3. Fault prediction and classification using AI in FCHEVs.

Author	AI Methods	Pros	Cons
Zhou et al. [95]	SVM Metaclassifiers Back_Propagation Neural Networks (BPNN) LSTM neural Networks Wavelet Packet Decomposition (WPD)	Addresses multiple FC system faults such as flooding, starvation, and drying Use of PCC to reduce the data dimensions reduction The selected algorithm showed good precision and low computational time LSTM algorithm is suggested as the optimal	Only simulated data are used. However, Gaussian noise is added to data to simulate realistic environmental conditions The prediction accuracy and complexity are traded-off to decrease the computational cost
Gu et al. [94]	LSTM SVM	Information from the literature and understanding based on physical phenomena is used to decide the model input parameters, as shown in the figure Shows the advantage of memory-based algorithms, such as LSTM, over traditional “memoryless” algorithms, such as SVM The results were validated using experimental data Experimental test on a large 92 kW vehicle fuel cell system used for validation	Lack of validation with the untrained data set
Zuo et al. [96]	CNN with Batch Normalization Conventional ML Decision trees Gaussian Naïve Bayers Support Vector Machine (SVM) K-Nearest Neighbor	A real experimental FC fault dataset is adopted to evaluate the performance of the diagnostic method. The results indicated a 99% accuracy in predicting faults The proposed model has a low computational cost and online diagnosis functionality	Scaled-down prototype test rig (only 80 W) and lack of representative test cycles to create model results that can be objectively verified against similar models available in the relevant literature
Zhou et al. [99]	XGBoost CNN LSTM CNN-SVM CNN-LSTM	Novel fault classification algorithm using XGBoost classifier The data comes from a Fuel cell vehicle tested in the field over a period of many months	Although a good background on the algorithm is provided. The type of faults that can be detected and how their detection occurs is not well documented. Non-standardized test cycles are used, making it difficult to objectively compare the results with other literature. Labeling of faulty data into different levels is poorly explained
Morando et al. [98]	Reservoir Computing based on non-linear delayed feedback dynamics	Good description of RC computing for fault classification of FC system	Lack of comparison with established AI-based fault classifiers Lack of experimental data for training and validation
Zhou et al. [97]	Binary matrix encoding and convolutional neural networks (BinE-CNN) LSTM SVM WPN	Predicting the seven different fault mechanisms is diagnosed Better time-series performance compared to SVM and LSTM Real-time feasible Model is experimentally validated

.

In summary, it can be gleaned from the articles cited above that typically, memory-based algorithms outperform memory-less algorithms in accurately predicting the onset of faults. Additionally, the RNN method, which works well with time-series data, is commonly used, and CNN, which can be used for time-series classification. An important factor for online fault prediction is to use appropriate measures to ensure that only the most essential factors are considered for the online system modeling. As mentioned before, water management faults depend on various systems, material properties, and control parameters. Creating a model that considers all the parameters can be highly time-consuming and yield only a tiny improvement in accuracy.

4.2. Optimization

The optimization Based Control Approach is the most frequent type of EMS found in the literature. A cost function that represents durability and fuel economy quantifies the objective of this particular EMS [100,101]. Minimization of the fuel consumption associated cost of the architecture or their combination is identified as the main objective for the optimization task. Various approaches, such as Stochastic Dynamic Programming (SDP), Dynamic Programming (DP), and Equivalent Cost Minimization Strategy (ECMS), have been proposed to achieve the aim of optimal control [102]. To ensure the optimum integration of the power sources and the control systems, many inequality and equality conditions are also considered in the optimization-based strategies [69]. This strategy is mainly classified into global and real-time optimization or local optimization strategies [100,101].

4.2.1. Genetic Algorithm

Some of the recent works conducted in the domain of GA concerning FCHEV are reviewed in this section (summarized in

Table 4. Summary of reviewed works related to GA in FCHEVs.

Author	Research	Pros	Cons
Odeim et al. (2015) [103]	Proposed an experimental analysis of EMS incorporating PI, multi-objective, and proportional employing GA designed for FCHEV.	The result of the study obtained through simulation and experimentation were the same, which validates the authenticity of the study.	The study did not provide any relevant data on the improvement of battery life.
Odeim et al. (2016) [104]	Conducted investigation on both real-time and offline optimization of a power management method of an FC/battery/SC hybrid system (vehicular).	The study showed that the real-time-based strategy consumes slightly more amount of hydrogen fuel as compared to the offline optimum while considerably improving the durability of the system.	The study only utilized NurembergR36 and Manhattan driving cycles.
Zhang et al. (2017) [105]	Studied a genetic algorithm-based fuzzy EMS designated for FC-SC-based hybrid vehicle architectures.	The study showed that the proposed EMS provides less hydrogen fuel consumption (close to 9%) in comparison to other EMS based on fuzzy logic.	The study lacked experimental validation and was limited to simulations only.
Ahmadi et al. (2018) [106]	Designed a structure of FCHEV and suggested a new EMS (optimized) to advance the dynamic performance of the vehicle while maintaining requirements (vehicle) and extended battery life.	Enhancement of fuel economy, improvement of vehicle performance, sustaining capability of battery charging, and optimal distribution of energy are a few of the important consequences attained by the suggested optimized EMS.	The study did not consider reducing the size of the components (FC/ Battery/UC) associated.
Zhou et al. (2019) [107]	Suggested a constrained programming parameter (nonlinear) based model of optimization aiming at reducing consumption of fuel in FCHEVs.	The suggested strategy was able to reduce the total consumption of fuel associated with FCHEVs by 17.6% and 9.7%, correspondingly, under the UDDS and HWFET cycles, without negotiating the performance (dynamic) of the vehicle.	The study considered only HWFET and UDDS driving cycles.

).

In a study, Odeim et al. [104] proposed an experimental analysis of EMS incorporating PI, multi-objective, and proportional using GA designed for FCHEV. The NSGA-II was utilized as the simulation tool. The study used the Manhattan driving cycle to collect experimental data. The result of the study obtained through simulation and experimentation were the same. However, the major drawback of the study is that it did not provide any data on the improvement of battery life.

In another study, Odeim et al. [104] investigated real-time and offline optimization of an FC/battery/SC hybrid architecture (vehicular) power management method. The study initially compared two optimization-based algorithms (offline), namely DP and PMP. The optimum (offline-based) is then utilized as a standard for developing a real-time-based strategy, which is necessary since the optimum (offline) strategy is not capable of real-time management and is focused on reducing the hydrogen fuel consumption only could cause unnecessary battery overloading. The development and associated optimization of the real-time-based strategy utilizes a GA (multi-objective) while considering, other than hydrogen fuel consumption, other significant factors, such as reducing the burden on the battery and the slow dynamics associated with the FC system. The study showed that the real-time-based strategy consumes slightly more hydrogen fuel than the offline optimum while considerably improving the system’s durability.

Zhang et al. [105] studied a genetic algorithm-based fuzzy EMS designated for FC-SC-based hybrid vehicle architectures. The study was conducted using ADVISOR for HWFET (The Highway Fuel Economy Test), UDDS (Urban Dynamometer Driving Schedule), and NEDC (New European Driving Cycle) driving cycles. The study showed that the proposed EMS provides less hydrogen fuel consumption (close to 9%) compared to other EMS centered on fuzzy logic.

Ahmadi et al. [106] designed a structure of FCHEV and suggested a new EMS (optimized) to advance the dynamic performance of the vehicle while maintaining requirements (vehicle) and extended battery life. In the study, the optimization of the suggested EMS is achieved by employing a GA through a joined highway/city drive cycle with various initial conditions. The study defined a full function (multi-objective) to identify the vehicle’s targets, requirements, and performance constraints. The simulation results demonstrated that the suggested approach affects (progressively) the vehicle’s characteristics. A few of the important consequences attained by the suggested optimized EMS are enhancing fuel economy, improving vehicle performance, sustaining the capability of battery charging, and optimal energy distribution.

In a recent study, Zhou et al. [107] suggested a constrained programming parameter (nonlinear) based model of optimization aiming to reduce fuel consumption in FCHEVs. In this study, the parameters (principal) associated with the power tracking-based control approach are fixed as the variables (optimized), with the performance index (dynamic) of FCHEVs being restricted as the limiting condition. The study then utilized a GA in designing a control strategy to find an optimal solution to the task. The GA is then integrated with the vehicle’s model in ADVISOR to optimize parameters associated with the control strategy for two driving cycles, namely UDDS and HWFET. Conclusively, the strategies after and before the optimization process are simulated, associated performances are equated, and the control parameters (optimal) under various driving cycles are examined. The simulation outcomes showed that by employing the suggested control strategy, the total fuel consumption associated with FCHEVs could be decreased by 17.6% and 9.7%, correspondingly, under the UDDS mentioned above and HWFET cycles, without negotiating the performance (dynamic) of the vehicle.

Fang et al. [108] proposed an artificial intelligence-based multi-objective optimization method to optimize PEMFC components, kinetics, thermodynamics, and overall performance in Hybrid systems. Technical, economic, environmental and socio-political objectives have been considered in this study. The genetic algorithm and MOO are used to optimize components in FC.

4.2.2. Particle Swarm Optimization (PSO)

PSO is another heuristic population-based optimization strategy. The method starts with a population and is iterated to search for a suitable solution, focusing on a particular quality measure [103,109]. In this method, a search space is considered in which particles wander around under the guidance of the best-known positions. The swarm particles move when improved positions are identified. PSO can search very vast areas associated with candidate solutions. Although identified to be non-casual, this method does not demand a differentiable type of optimization problem, making it suitable for optimization problems possessing irregularity or noise [110]. A further improved model of PSO is Dynamic Particle Swarm Optimization (DPSO), which can avoid the issues of being trapped in local optima and associated stagnation without compromising the quick convergent abilities of PSO [111]. Some of the recent works conducted in the field of PSO concerning FCHEV are reviewed in this section (summarized in

Table 5. Summary of reviewed works related to PSO in FCHEVs.

Author	Research	Pros	Cons
Trovao et al. (2013) [112]	Suggested rule-based meta-heuristic energy management and optimization method.	The suggested approach was able to achieve effective and fast splitting of power between the battery and SC.	The study considered only ARTEMIS and ECE15 driving cycles.
Hegazy et al. (2013) [113]	Proposed a method for sizing of components associated with FC/SC, FC/Battery, and FC/Battery/SC hybrid architectures employing a control strategy founded on an efficiency map and PSO.	The study was able to demonstrate that the FC/Battery/SC-based topology provides improved performance as compared to the other two topologies.	The study considered only NEDC and FTP75 driving cycles.
Chen et al. (2018) [114]	Suggested an online EMS and gear-shifting method utilizing DPSO.	The proposed study was able to achieve reliable characteristics related to power splitting among different sources of energy. The suggested strategy also achieved a considerable decrease associated with hydrogen fuel consumption as compared to classic rule-based controls.	The study considered only FTP and ECE40 driving cycles. The focus of the study was mainly concentrated on gear shifting rather than on EMS.
Song et al. (2019) [115]	Suggested a multi-objective-based optimization design strategy depending on the PSO algorithm, aiming at optimizing the fuel economy, vehicle cost, and improved vehicle performance (dynamic).	The study was able to obtain the optimal scheme (hybrid) of the FCHV. The study can provide valuable insights toward the design of improved powertrains for FCHVs.	The study lacked experimental validation and was limited to simulations only.
Tifour et al. (2020) [116]	The study utilized a PSO for optimization and monitoring of the parameters (fuzzy) under various conditions, aiming at identifying the finest sets that can provide great improvement in the domain of fuel economy while considering the SOC (battery) maintenance associated.	The suggested approach showed an improved fuel economy when compared with the Power-Tracking-Controller-based Adviser under all circumstances and also the efficiency (overall) in most circumstances.	The study did not consider the sizing factor associated with power sources. The study lacked experimental validation and was limited to simulations only.

).

Trovao et al. [112] explained the rule-based meta-heuristic energy management and optimization method. Two cycles of driving were used in the study, namely, ARTEMIS and ECE15 cycles. The proposed study was able to achieve effective and fast splitting of power between the battery and SC. Nevertheless, the lack of experimental validation is the major disadvantage of this study.

In another study, Hegazy et al. [113] proposed a method for sizing components associated with FC/battery, FC/SC, and FC/battery/SC hybrid architectures employing a control strategy founded on an efficiency map and PSO. The study used NEDC and FTP75 driving cycles. The study demonstrated that the FC/battery/SC-based topology provides superior performance compared to the other two topologies in terms of fuel efficiency, volume, mass, and associated costing.

Chen et al. [114] proposed an online EMS and gear-shifting method utilizing DPSO. The study used two driving cycles, namely, FTP and ECE40. The proposed study achieved reliable characteristics related to power splitting among different sources of energy and a considerable decrease associated with hydrogen fuel consumption compared to classic rule-based controls. Nevertheless, the study’s focus was mainly on gear shifting compared to energy management and optimization.

In a recent study, Song et al. [115] suggested a multi-objective-based optimization design strategy depending on the PSO algorithm to optimize fuel economy, vehicle cost, and improved vehicle performance (dynamic). Integrated with a simulation environment, the suggested algorithm can search for the optimal solution in the whole reasonable collection of the degree of hybridization. This enabled the study to obtain the optimal scheme (hybrid) of the FCHV. The study provides a basis for the design of an improved powertrain for FCHVs.

In another recent study, Tifour et al. [116] suggested an EMS depending on first-order Sugeno fuzzy developed for FCHEV. The study utilized a PSO for optimization and monitoring of the parameters (fuzzy) under various conditions, aiming at identifying the finest sets that can provide great improvement in fuel economy while considering the associated SOC (battery) maintenance. The results of the study demonstrated an improved fuel economy when compared with the Power Tracking Controller-based Adviser under all circumstances and efficiency (overall) in most circumstances. The study also demonstrated that fine-tuning the suggested EMS under a single condition will not ensure the same performance (evaluated in terms of battery SOC) as when tested under different conditions. The study also identified that if the suggested EMS is fine-tuned under different circumstances, it can attain an improved fuel economy with smooth variations in battery SOC and a minimal change associated with SOC, which is critical to battery life extension.

4.3. Control

Mane et al. [117] suggested an MPC strategy for the two-loop control system in FC/UC-based HEV that is able to achieve constant DC-bus voltage and efficient power splitting between UC and FC. However, in this paper, hydrogen fuel consumption and the cost associated with hydrogen fuel consumption have not been considered.

4.4. Energy Management System (EMS)

Energy Management is an essential operating requirement of an FC vehicle. As previously explained, the FCEV powertrain can draw power from two or three energy sources—Battery, Ultracaps, and FC. The EMS decides the optimal source for the given duration and power demand. For a simpler understanding, one can assume that the battery fulfills all short-duration pulse power requirements for quick acceleration due to a low run-up period. However, longer-duration steady power is delivered by the fuel cell system, which is slowly driven up to meet the power to prevent damage. This description is too simple but can be used for initial understanding. In reality, the EMS has to make many crucial decisions based on vehicle data and driver input quite quickly. The EMS can be broadly classified into Online and Offline EMS, as explained in the following section.

4.4.1. Offline EMS

Fuzzy Logic Control

Chen et al. [118] proposed an EMS for a hybrid energy-based system comprising a PEMFC, a Lithium-Ion battery (LIB) pack, and DC/DC converters. The objective of the suggested EMS is to ensure a proper FC current to reduce hydrogen fuel consumption with restrictions on SOC and power. Additionally, the study ensures that by altering the SOC and the demand (load), the battery current (actual) can be kept up with the reference value. The effectiveness of the suggested EMS was validated by conducting a simulation.

Zhang et al. [119] proposed an EMS for achieving power split with an FL-based controller for the FCHEV powertrain. The study presents a power control (regenerative) and battery-based controller (local energy) to ensure that the battery SOC is modifiable without under-voltage and over-voltage. The simulation result demonstrated that the suggested method could retain SOC associated with the battery at expected levels. It can effectively absorb braking energy (regenerative) and minimize the load (dynamic) related to the FC to reduce/minimize fuel starvation.

In another study, Saib et al. [120] suggested an FL-based EMS applied on an FCHEV. The FL-based control approach has been designed to regulate the power flow in the system (hybrid) under two restrictions: the battery SOC and the FC response (low dynamic). Simulation studies have been performed in the MATLAB/Simulink setting to study the performance associated with the suggested EMS. The outcomes demonstrated reasonable improvement in the hybrid system’s performance: the battery supplies power to the system when peak power is demanded, which helps to create a smooth FC response while sustaining the battery SOC within a suitable window.

In a recent study, Essoufi et al. [121] proposed an EMS for an FCHEV (considering fuel cell and Lithium-Ion batteries as primary and secondary power sources, respectively). The suggested strategy depends on fuzzy logic. It aims to reduce fuel consumption (hydrogen) while improving the durability associated with the power sources by considering their associated constraints (dynamic) and the battery SOC. The simulation of FCHEV and the suggested EMS were established using a MATLAB/Simulink setting. The results of the simulation demonstrated the viability of the suggested model and showed that the suggested control strategy provides a good enhancement in reducing fuel consumption and in achieving a further improved and efficient energy distribution between the sources.

This approach is utilized to compute the real-time optimal power splitting possibilities among the available power sources. It is achieved by replacing the commonly utilized global cost function with the instantaneous cost function [13]. This strategy tackles the issues associated with the lack of real-time road data availability and the associated computation burden. A summary of reviewed works related to fuzzy rule-based strategy in FCHEV is listed in

Table 6. Summary of reviewed works related to fuzzy rule-based strategy in FCHEV.

Author	Research	Pros	Cons
Mohammedi et al. (2014) [122]	Proposed a fuzzy logic system dependent on passivity control.	The suggested approach improved the robustness of the system, reduced the consumption of hydrogen fuel, and reduced the overshoot associated with the system.	The study lacked experimental validation. No proof associated with a reduction in the consumption of hydrogen fuel was provided in the study.
Hemi et al. (2014) [123]	Suggested a fuzzy logic-based control on three configurations depending on the UDDS cycle of driving.	The study identified a practical configuration-based method to reduce the consumption of hydrogen fuel.	The study focused on hybrid configuration comparison only. The study lacked experimental validation.
Saib et al. (2017) [120]	The study suggested an FL-based EMS applied on an FCHEV.	The study demonstrated that the restrictions are respected effectively by the hybrid system.	The study lacked experimental validation.
Chen et al. (2018) [118]	The study suggested an EMS for an FC/ Battery hybrid energy system.	The study ensures that by altering the SOC and the demand (load), the battery current (actual) can be kept up with the reference value. The viability of the suggested EMS was validated through simulation results.	The study lacked experimental validation and was limited to simulations only.
Zhang et al. (2018) [119]	The study proposed an EMS, towards achieving power-split with an FL-based controller, for the FCHEV powertrain.	The study showed that the suggested method could retain SOC associated with the battery at levels expected, can effectively absorb braking energy (regenerative), and minimize the load (dynamic) associated with the FC to evade fuel starvation	The study lacked experimental validation and was limited to simulations only. The study only focused on the power-split characteristics of the EMS.
Essoufi et al. (2020) [121]	Suggested an FL-based EMS for an FCHEV (Considering fuel cell and Li-Ion based battery as a primary and secondary source of power correspondingly).	The simulation of FCHEV and the suggested EMS were established by employing a MATLAB/Simulink setting. The simulation outcomes demonstrated the viability of the suggested EMS.	The study lacked experimental validation.

.

4.4.2. On-Line EMS

Equivalent Consumption Minimization Strategy (ECMS)

The ECMS converts the electrical energy stored in the ESS into its equivalent consumption of hydrogen fuel [69]. Reduction in the total equivalent consumption and the sum of consumption of hydrogen fuel are the main objectives of ECMS. The value of the equivalent factor, which is crucial for ECMS, is usually affected by the SOC limits of the battery and the driving cycles [122]. Research shows that the lifetime associated with the power sources and fuel consumption efficiency can be enhanced by utilizing the possibility of equivalent factor adjustment [124]. Some of the recent works conducted in the field of ECMS concerning FCHEV are reviewed in this section (summarized in

Table 7. Summary of reviewed works related to ECMS in FCHEVs.

Author	Research	Pros	Cons
Hemi et al. (2015) [123]	Proposed an ECMS based on PMP, integrated with the approach of Markov chain.	The study was able to achieve the power demands of the hybrid system.	The study only considered the UDDS driving cycle. The study failed to show any difference in the consumption of hydrogen.
Feroldi et al. (2016) [53]	Suggested a hierarchical EMS based on ECMS and low-pass filter, aiming at improving the lifespan of energy sources, performance, and fuel efficiency for FCHEVs.	The study showed a reduction in the consumption of fuel when compared to a conventional oversized fuel cell. The suggested strategy was able to improve the global efficiency associated with the FC and propulsion system.	The suggested strategy Displayed an increased consumption of hydrogen fuel associated with a smaller-size SC.
Li et al. (2018) [125]	Suggested a SECMS for FCHEV powered by FC, battery, and SC.	The study demonstrated that the suggested SECMS has the minimal hydrogen fuel utilization and provides the highest FC durability.	The study considered only WVUCITY, LA92, and New York Bus driving cycle. The study lacked experimental validation and was limited to simulations only.
Liu et al. (2019) [126]	Proposed an EMS founded on ECMS aiming at reducing hydrogen fuel utilization and enhancing the battery life of an FCHEV.	The study demonstrated that when compared with a RULE-based strategy, the suggested approach minimizes hydrogen fuel consumption by 0.87%, thus improving hydrogen fuel economy and providing an extended battery life.	The study lacked experimental validation and was limited to simulations only.
Fu et al. (2019) [127]	Proposed a hierarchical EMS based on ECMS and low-pass filter, aiming at improving the lifespan of energy sources, performance, and fuel efficiency for FCHEVs.	The suggested EMS was modeled and tested by ADVISOR-Simulink and by utilizing an experiment bench. The effectiveness of the suggested EMS was validated both by experimentation and simulation.	The study did not consider the driver factor provided that the conditions of the road for the EMS suggested in this study are prior knowledge.

).

In a study, Hemi et al. [123,128] suggested an ECMS founded on PMP, integrated with the approach of the Markov chain. The study was conducted using the UDDS driving cycle. The study was able to meet the power demands of the hybrid system. However, the study failed to show any difference in hydrogen consumption.

In another study, Feroldi et al. [53] proposed an ECMS to be utilized by employing the sizing procedure. The method was compared with an optimal strategy of sizing that depends on deterministic dynamic programming. The study showed a reduction in the consumption of fuel when compared to a conventional oversized fuel cell. Additionally, it was identified that the proposed approach could progress the global efficiency associated with the fuel cell and propulsion system. However, the proposed strategy displayed an increased hydrogen fuel consumption associated with a smaller supercapacitor (SC) size.

In another study, Li et al. [125] proposed a sequential quadratic programming (SQP)-based ECMS (SECMS) for FCHEV powered by FC, a battery, and SC. Aiming to decrease fuel consumption and enhance the durability associated with power sources, FC is selected as the primary power source and acts as a steady current source, with the battery as the chief energy buffer and substitute for FC failure. SC is selected to provide peak demand for power. The study also designed a rule-based control approach and a hybrid ECMS-based operating mode control strategy (to compare with the suggested SECMS. The study also designed a test bench (experimental) to substantiate three designed approaches (comparative). The study demonstrated that, in comparison with the rule-based control approach and hybrid ECMS-based operating mode control strategy, the hydrogen fuel consumption of the suggested SECMS shows a reduction of 2.16% and 1.47%, respectively, and has the smoothest FC current, causing the lowest FC degradation.

Liu et al. [126] proposed an EMS founded on ECMS to reduce fuel utilization and enhance the battery life of an FCHEV. The study designed a variable equivalent factor of SOC consistency by considering the SC and battery SOC. The study obtained FC’s output power (optimal) by employing an ECMS. The study, based on Simulink and ADVISOR simulation, demonstrated that when compared with the rule-based strategy, the suggested approach minimizes hydrogen fuel consumption by 0.87%, thus improving hydrogen fuel economy and also providing an extended battery life.

In a recent study, Fu et al. [127] suggested a hierarchical EMS founded on ECMS and low-pass filters, aiming at improving the lifespan of energy sources, performance, and fuel efficiency for FCHEVs. In the higher-layer approach of the suggested EMS, SC is utilized to supply the demand for peak power and to recycle braking energy by employing the low-pass filter (adaptive) technique. Additionally, an ECMS is developed to achieve power allocation between the battery and fuel cell such that FC can work in high-efficiency ranges to reduce hydrogen consumption in the lower layer. The suggested EMS was modeled and tested by ADVISOR-Simulink and by utilizing an experiment bench. Through the synergistic decay of dual power source lifespan, can the optimal selection range of FCV power battery capacity manage energy consumption [129]. A power distribution unit in the modular fuel cell system can manage energy consumption in FCV [130].

Model Predictive Control (MPC)

MPC is a type of local optimization strategy generally utilized to tackle problems consisting of several constraints. It can accurately predict the changes that are to happen in the future through the analysis of the dynamic state, current values, MPC model, and by utilizing process variables [131]. In the MPC strategy, the current state of the architecture is analyzed, and an associated cost minimization strategy is calculated for a short time in the future. In other words, MPC allows for the optimization of current time slots while keeping track of future time slots. This is, in fact, one of the major advantages of MPC [132]. Figure 11 shows a basic block diagram of MPC-based EMS adopted by Kanchwala et al. [133]. Some of the recent works conducted in the domain of MPC concerning FCHEV are reviewed in this section (summarized in

Table 8. Summary of reviewed works related to MPC in FCHEVs.

Author	Research	Pros	Cons
Amin et al. (2012) [132]	Suggested an MPC EMS founded on DP.	The proposed strategy was experimentally validated, and it demonstrated the presence of a well-regulated DC-bus voltage. The proposed strategy was tested using dSPACE DS1104.	The study only focused on the voltage regulation of the DC-bus, and no focus was given to hydrogen fuel consumption.
Ahmed et al. (2013) [134]	Suggested an MPC-based tuning strategy, designed by comparing a statistically constrained type controller with back-off-based control.	The study was able to eliminate constraint-based violations virtually. The study successfully compared the simulations among the different statistically constrained types of controllers.	The study only focused on feasibility, and no focus was given to battery life or hydrogen consumption.
Mane et al. (2016) [117]	Suggested an MPC strategy for a two-loop control designed for an FC/UC-based HEV.	The suggested EMS achieved constant DC-bus voltage and efficient power splitting between UC and FC.	The study did not focus on hydrogen fuel consumption or its cost.
Tianyu et al. (2018) [135]	Suggested an MPC-based EMS employing Markov chain and NN techniques.	The proposed EMS improved fuel efficiency, increased the lifetime of FCs, and sustained the SOC of SC using NNs.	The study lacked experimental validation and was limited to simulations only.
Liu et al. (2018) [136]	Proposed a hierarchical- MPC strategy to optimize the performance and efficiency of a PEMFC-based HEV.	As per the suggested approach, 7.79% of equivalent fuel consumption is anticipated.	The study considered only the US06 driving cycle.
Furquim et al. (2020) [137]	Proposed an EMS for an FCHEV. The suggested EMS depends on nonlinear-MPC and utilizes a neural network (recurrent) for modeling a PEMFC.	The suggested nonlinear-MPC-based EMS provides improved fuel economy and minimizes FC degradation.	The study lacked experimental validation on a real FCHEV.

).

In Ref. [134], an MPC-based tunning strategy with back-off-based control was proposed, which could statistically compare all the system constraints. This method could eliminate constraint-based violations and compare all types of controllers statistically, but they did not consider battery life and hydrogen consumption. Amin et al. [132] suggested an MPC energy management strategy founded on Dynamic Programming (DP) that could demonstrate experimentally and have well-regulated DC-bus voltage. The focus of this study was on the voltage regulation of DC-bus, and hydrogen consumption was missed. In another piece of research, Liu et al. proposed a hierarchical MPC strategy to optimize the efficiency and performance of a PEMFC in the US06 driving cycle only. The study shows 7.79% of fuel equivalent consumption.

Amin et al. [132] suggested an MPC EMS founded on DP. The proposed strategy was tested using DS1104. The proposed strategy was experimentally validated and demonstrated the presence of a well-regulated DC-bus voltage obtained by the proposed strategy. However, the study suffered from the fact that no focus was given to the consumption of hydrogen fuel, and the focus was only on the voltage regulation of the DC-bus. Ahmed et al. [134] proposed an MPC-based tuning strategy designed by comparing a statistically constrained type controller with back-off-based control. The proposed study was able to eliminate constraint-based violations virtually. Additionally, the study successfully compared the simulations among the different statistically constrained types of controllers. However, the focus of the study was on feasibility rather than on battery life or consumption of hydrogen. Mane et al. [117] suggested an MPC approach for a two-loop control designed for an FC/UC-based HEV. The proposed EMS achieved constant DC-bus voltage and efficient power splitting between the ultracapacitor and fuel cell.

Tianyu et al. [120] proposed an MPC-based EMS employing Markov chain and NN techniques in another study. The proposed EMS improved fuel efficiency, increased the lifetime of FCs, and sustained the SOC of SC using NNs. However, the lack of experimental validation and worse performance of the Markov chain when compared to neural networks are the major drawbacks of the proposed EMS. In another study, Liu et al. [136] suggested a hierarchical MPC strategy to optimize the performance and efficiency of a PEMFC-based HEV. A control-oriented(linearized) FCHEV is first presented in the study. The study then develops the hierarchical -MPC strategy-based control, containing a lower and an upper-level MPC-based controller. The MPC controller (upper level) is developed toward the power splitting ratio optimization between the battery pack and the PEMFC. In contrast, the lower-level MPC-based controller is utilized to trace the PEMFC net output power (maximum) by optimizing the excessive oxygen ratio. The US06 cycle of driving is utilized in the performance analysis of the suggested method, and up to 7.79% fuel equivalent consumption is anticipated.

In a recent study, Furquim et al. [137] proposed an EMS for an FCHEV. The suggested EMS depends on nonlinear- MPC and utilizes a neural network (recurrent) for modeling a PEMFC. The suggested EMS was employed on a development board (low-cost), and the tests were conducted in real-time by utilizing a HIL test bench integrated with a real 3 kW fuel cell stack. The study results showed that the nonlinear- MPC-based suggested EMS can meet the vehicle’s energy requirement and ensure the FC’s operation in its highest efficient region.

Learning-Based EMS

Neural networks (NNs) and reinforcement learning (RL) are the two main parts of a learning-based energy management system for fuel cell vehicles [138]. The NN-based EMS must choose the input model considering different environmental data, driver behavior, and vehicle specification, and then use optimal power distribution as the output model to train NNs. In most papers for NN-based EMS for fuel cell vehicles, the NN algorithm is utilized for working condition classification or the prediction of a vehicle’s speed [139,140,141,142,143,144], as covered in previous sections. Teng et al. [57] reviewed all different NN methods for different strategies in fuel cell vehicles. Unlike the NN algorithms utilized for fuel cell vehicles, the RL methods EMS with small or big data inputs widely. The most common RL algorithms that are used in EMS are Q-learning, deep Q-network (DQN), deep deterministic policy gradient (DDPG), and twin delay deep deterministic policy gradient (TD3).

Q-Learning

Zhang et al. [145] proposed a dual reward functions Q-learning algorithm to reduce operation stress and guarantee safe and stable FCHEV operation. This method utilizes three-level efficiency optimization and has been tested in experimental conditions, and the results showed a reduction in hydrogen usage and fuel. Li et al. [146] suggested a speedy reinforcement learning algorithm based on Q-learning for energy management of fuel cell vehicles considering fuel cell lifetime. The operation time of the method showed that this method is adaptable for real-time EMS. Additionally, it is adaptable to three different driving cycle sources with different conditions. Sun et al. [147] proposed a data-driven reinforcement learning-based hierarchical energy management strategy for FCHEV based on a combination of Markov decision and Q-learning algorithms. The results show fuel consumption economy, optimal fuel cell efficiency, and low computational time, which have been tested in experimental conditions. The results are compared to DP as an optimal benchmark to show the method’s efficient performance.

In the Q-learning algorithm, connected state and action spaces lead to poor optimality and convergence in practical applications, so it is necessary to discrete state space and action space in algorithms. Under the actual working condition for the fuel cell vehicle, the state and action spaces are continuous, and the Q-learning algorithm could overestimate the Q value and have a dimension disaster problem in this situation [148], which will cause non-optimal results.

DQN

A multi-objective DQN algorithm was proposed by Li et al. [149] to reduce hydrogen consumption and increase the lifetime of the fuel cell. The suggested DQN algorithm is compared to the Q-learning algorithm, and the results show significant improvement in the convergence speed of the algorithm, durability and fuel consumption.

Tang et al. [150] utilized the DQN algorithm to prioritize experimental responses to energy management to minimize hydrogen consumption. The results were compared to the DP-based algorithm and showed significant improvement in hydrogen consumption in unfamiliar driving environments and untrained conditions. Zheng et al. [151] used the same strategy but tried to reduce hydrogen consumption and fuel cell durability based on a fuel cell degradation model. Additionally, the algorithm adaptability has been tested, and the results show that DQN-based EMS is more adaptable than other cited methods. In another paper, Zheng et al. [134] introduce a deep reinforcement learning algorithm based on DQN for fuel cell hybrid buses. The results compare to RL-based and DP-based EMSs that show 3.63% and 5.69% improvement in hydrogen consumption. The proposed method decreases the fuel cell degradation rate compared to the one without considering the fuel cell durability.

The proper definition of state and action spaces is important in reinforcement learning methods to reduce hydrogen consumption. Based on the review, the DQN performs better than the Q-learning algorithms. Still, there is a Q value overestimation problem in DQN because the action space cannot assume continuously, but compared with Q-learning, the results seem more convergence optimal.

DDPG

Zheng et al. [152] suggested a Deep Deterministic Policy Gradient (DDPG) algorithm for fuel cell hybrid vehicles that achieves continuous energy management control. The algorithm utilized the fuel cell system efficiency characteristic to improve the control effect. In the study, they tried to improve the computational efficiency of DDPG, and the results showed stable convergence and optimal and adaptive energy management strategy. Huo et al. [153] proposed DQL and DDPG algorithms to minimize fuel consumption and prolong the fuel cell stack lifespan in fuel cell hybrid vehicles. In this research, the fuel economy and power fluctuation combined to create a multi-objective reward function for the DDPG algorithm to be tested for four different driving cycle sources. The DDPG algorithm shows adaptability for use in multi-cycles compared to DRL. Zhou et al. [154] designed a DDPG algorithm to regulate SOC, assisting power consumption in different driving cycle sources. The same authors in another research [155] utilized the DDPG algorithm for power distribution based on vehicle speed, acceleration, and SOC to improve fuel economy. The results showed increased durability of fuel cells and reduced hydrogen consumption. Still, the DDPG algorithm suffers from Q value overestimation and unstable training.

Twin Delayed DDPG (TD3)

In [156], Deng et al. applied the Twin Delayed DDPG (TD3) algorithm for EMS in fuel cell hybrid vehicles. Based on the TD3 characteristics, the method is more stable and economical in stochastic environmental conditions. The problem with this algorithm is the long learning time which is not suitable for real-time energy management systems in vehicles. Zhou et al. [157] proposed a TD3 algorithm for intelligent transport systems consisting of different vehicle topologies to provide more valuable environmental signals for agents. This method tries to optimize hydrogen consumption with an RL-based strategy. A summary of the learning-based used in FCHEVs is listed in

Table 9. Summary of reviewed works related in FCHEVs.

Author	Research	Pros	Cons
Zheng et al. (2022) [158]	Proposed a Deep Reinforcement Learning (DRL) energy management strategy for fuel cell hybrid buses to improve fuel economy	The study compared the DRL result with DP and RL algorithms which showed significant improvement. The degradation rate of fuel cells decreased by using the DRL algorithm. The DRL algorithm is adaptable to a new driving cycle.	The study lacked experimental validation and was limited to simulations only.
Li et al. (2022) [146]	Suggested a speedy reinforcement learning-based energy management strategy for fuel cell vehicles considering fuel cell lifetime	The algorithm was able to extend the fuel cell system’s lifetime The study successfully trained the algorithm for three driving cycles and validated it on another driving cycle. The convergence speed of the algorithm is increased, and it has the potential to work in real-time mode.	The study did not compare its results with the optimized-based method as a baseline. The study lacked experimental validation and was limited to simulations only.
Zhang et al. (2021) [145]	Suggested a learning-based EMS based on dual reward functions Q-learning algorithm, which can guarantee the safe and stable operation of FCHEV.	Suggested a learning-based EMS optimization with a three-level efficiency. The method can improve energy efficiency and slow the FC’s aging by reducing its operation stress. The proposed method has been tested on experimental 1.2 kW FCHEV.	The study has not checked the method’s adaptability for different driving cycles.
Sun et al. (2018) [147]	Suggested a data-driven reinforcement learning-based hierarchical energy management strategy for FCHEV	The proposed EMS could achieve low computation cost, optimal fuel cell efficiency and energy consumption economy. For the simulation, the authors utilized experimental data. The method compared to DP as a baseline shows how much this method can be near to the optimized-based method.
Zheng et al. (2022) [151] Tang et al. (2022) [150]	Proposed a DQN energy management system considering the priority of experimental reply	The suggested method shows a significant impact on hydrogen consumption It compared to the DP-based method as a benchmark The method is adaptable to unknown environmental conditions and untrained situation	The study utilized real data, but there is not any experimental validation for the results
Zheng et al. (2021) [152]	Proposed an EMS for an FCHEV. Based on the DDPG algorithm for continuous control strategy	The suggested method improves computational efficiency, and the results showed stable convergence, optimal and adaptive energy management strategy	The algorithm suffers from an overestimation of the Q value for the algorithm and creates unstable training sometimes The study lacked experimental validation on a real FCHEV.
Huo et al.(2022) [153]	Suggested DDPG algorithms to minimize fuel consumption and prolong the fuel cell stack lifespan in fuel cell hybrid vehicles	It is utilized for different driving cycle sources, and it shows the adaptability f the DDPG as a multi-cycle algorithm	The computational efficiency is not convincing, and the training is unstable.
Zhou et al. (2022) [154,155]	Proposed DDPG algorithm to minimize hydrogen consumption in FCHVs	This study focuses on SOC and power distribution, considering vehicle speed and acceleration. It can improve fuel economy significantly	There is not any experimental validation for the results The unstable training of the Q value is another problem.
Deng et al. (2022) [156]	Suggested a TD3 algorithm for energy management control in transportation systems with different vehicles	The results show improvement in fuel economy.	The environmental data are just stochastic, and they are not real data. The long learning time, which is not working as a real-time controller
Zhou et al. (2019) [157]	Suggested a TD3-based energy management algorithm for FCHVs	The method optimizes fuel consumption in the transportation system	The data are stochastic. The learning time or the algorithm is long.

.

4.4.3. V2X EMS

This section reflects on another crucial aspect of fuel cell electric vehicles. With a sizeable onboard battery and a fuel cell system, the cars can do much more than just supply power to the drivetrain and propel the vehicle. Some studies have focused on integrating fuel cell systems into “zero-emission buildings” [159,160]. The development of smart grids using distributed energy resources (DERs), including photovoltaic (PV) systems, wind turbine, and PEM fuel cells for intermittent energy supply to avoid blackouts, is an essential aspect of future energy research. The PEMFC from a fuel cell electric vehicle (FCEV) can be used to support such smart grids. However, the presence of multiple energy sources requires intelligent energy management. Hafsi et al. [161] carried out a comparative study based on classical PI CPI and state machine strategy (SM), fuzzy logic controller (FLC), and Artificial Neural Network (ANN) to optimize the energy demand distribution amongst multiple energy sources, such as PV, wind turbines, batteries, and a fuel cell. Similarly, Hassan et al. [162] created a microgrid simulation to integrate and synchronize Solid Oxide Fuel cells (SOFC), electrolyzers, and Ultracapacitors.

Additionally, the FC systems are equipped with state-of-the-art sensors and high-tech computers, which allow their integration into the Internet of Things (IoT) or Internet of Vehicles (IoV) [163]. Furthermore, using the information from the IoV, such as traffic flow and traffic signals, one can augment the energy management strategy of a fuel cell vehicle to address parameters beyond the internal state parameters such as power demand and battery SOC and include traffic flow information. This can further reduce the chances of a full stop due to traffic lights and traffic congestion, improving consumption and reducing degradation due to high start-stop frequency. Yan Mei et al. [162] created a flow diagram (Figure 12) depicting a dual-layer Energy management strategy that utilizes the IoV and traffic information to improve the utilization of the Fuel cell system. A combination of a launch control using Deep Reinforcement Learning (DRL) and an Energy management system based on Model Predictive Control using a Bi-directional Long Short-Term Memory (BiLSTM) network for power prediction is proposed. This study showed significant improvement in the H₂ consumption and degradation due to start–stop and idle time.

5. Remarks and Future Scope for Research

This article provides an in-depth summary of a fuel cell system and the challenges associated with a fuel cell electric vehicle powertrain. A significant portion of the research papers reviewed in this article deal with the inherent complexity of a dual-energy source powertrain. A complex algorithm that optimizes multiple objective functions must be designed to improve the utilization of the total onboard energy by improving the power distribution, reducing consumption of H₂, and preventing degradation of the fuel cell. AI and ML algorithms have been applied to this domain with considerable success.

The main advantage of AI and ML algorithms is their model flexibility. These algorithms allow the creation of accurate system models under challenging conditions, such as limited data or time series data or developing models that require very low computational power for online applications. Not only this, but adding multiple layers to the algorithm can allow the system to make modifications to the hyperparameters online to improve its accuracy while the system is running, taking new input data in real time and improving the accuracy.

Today there are several FC vehicles available in the market, such as the Toyota MIRAI, but the price is still higher than that of a typical family car; further cost reduction is required for widespread adoption of the FCV. An improved PtCo/C catalyst is applied in the MIRAI, but the amount of platinum in the catalyst is still quite significant. Physical modeling and optimization using AI can be key to improving the situation. The challenge of improving the system is a Multiphysics multidisciplinary problem that requires improvement in energy management, thermal management, optimal system sizing, vehicle design, and transmission design. Only by improving on all these fronts can one bring down the cost of FC vehicles significantly while reducing the operating cost and improving longevity. The flexibility of AI models make them ideal for modeling and solving such complex problems. Future work on modeling such interdisciplinary system interactions using AI must be emphasized to reduce cost and improve the market share of Fuel cell EVs. The future work on AI could be:

• Improve AI accuracy while the system is running, taking new input data in real time and improving the performance.
• With using AI methods should reduce costs and improve the share of FCVs

Even with the above improvements and cost reductions, the MIRAI price is still higher than that of a typical family car; further cost reduction is required for the widespread adoption of the FCV. An improved PtCo/C catalyst is applied in the MIRAI, but the amount of Pt in the catalyst is still quite significant. Higher specific activity (SA) and lower electrochemical surface area (ECSA) or higher Pt utilization in the catalyst with roughly equal ECSA and SA compared to existing catalysts are being explored, for example, using Pt nano-frame and core–shell catalysts, respectively. Modeling and simulation are expected to improve the associated issues. At the same time, non-noble metal catalysts, oxide catalysts, and carbon alloys are being actively researched. The Pt loading in the anode catalyst layer is also expected to be reduced after Pt reduction is realized at the cathode. For the per fluorinated membrane and the GDL, the chemistry and processes for manufacturing are nearly fixed, and cost reductions will come with scale. FC durability has been and will be mitigated by operational optimization. For example, the Pt dissolution that causes the ECSA reduction is minimized through the current operation with slow sweep rates and upper and lower voltage limits. Technology for efficient exhaust heat management is still required. The PEMFC is usually operated at around 60 to 80 °C, whereas the conventional ICE vehicle is operated at around 110 °C. This smaller temperature gap between the FC and the ambient air requires a bigger radiator size when the FCV is driven at maximum power. The high-temperature operation would increase the heat rejection rate from the radiator, so high-temperature operation is a candidate among many solutions to keep the radiator size low. Technologies to reduce exhaust heat have not yet received much attention. In terms of infrastructure, it is imperative to establish and operate hydrogen-fueling stations through the initial market introduction period of the FCV. To help with this aspect, not only have governments made plans to subsidize the establishment and operation of these fueling stations, but also Toyota, Nissan, and Honda have announced partial financial support for the operation of hydrogen stations in Japan.