The Neural Network Classifier Works Efficiently on Searching in DQN Using the Autonomous Internet of Things Hybridized by the Metaheuristic Techniques to Reduce the EVs’ Service Scheduling Time

Abstract

Since the rules and regulations strongly emphasize environmental preservation and greenhouse gas GHG reduction, researchers have progressively noticed a shift in the transportation means toward electromobility. Several challenges must be resolved to deploy EVs, beginning with improving network accessibility and bidirectional interoperability, reducing the uncertainty related to the availability of suitable charging stations on the trip path and reducing the total service time. Therefore, suggesting DQN supported by AIoT to pair EVs’ requests and station invitations to reduce idle queueing time is crucial for long travel distances. The author has written a proposed methodology in MATLAB to address significant parameters such as the battery charge level, trip distance, nearby charging stations, and average service time. The effectiveness of the proposed methodology is derived from hybridizing the meta-heuristic techniques in searching DQN learning steps to obtain a solution quickly and improve the servicing time by 34%, after solving various EV charging scheduling difficulties and congestion control and enabling EV drivers to policy extended trips. The work results obtained from more than 2145 training hypothetical examples for EVs’ requests were compared with the Bayesian Normalized Neural Network (BASNNC) algorithm, which hybridize the Beetle Antennae Search and Neural Network Classifier, and with other methods such as Grey Wolf Optimization (GWO) and Sine-cosine and Whale optimization, revealing that the mean overall comparison efficiencies in error reduction were 72.75%, 58.7%, and 18.2% respectively.

1. Introduction

There are many differentiators between planning and scheduling policy paradigms, but these two paradigms are complementary to each other. A pure policy assignment can be defined in terms of its applicability in the relevant disciplines such as “The difficulty of Policy is in identifying a set of actions that will transform the existing circumstance into one in which the target description is correct.” [1,2]. While Marsay, D. J. focuses on moving the policy world from specified beginning circumstances to a specified target state is the crux of the AI Policy issue through sequence of actions.” [3]. Another viewpoint on the policy of planning task is to evaluate it as a design or synthesis activity. This perspective differs considerably from the usual approaches. The policy can be defined as an inventing, styling, or forming something, such as the pieces’ ranking of a thing through execution of actions, according to the dictionary [4]. There are a few inferences that may be made about the nature of a policy assignment after examining the numerous definitions aforementioned. When policy is a job, the primary focus is developing a series of actions that will enable the problems necessary to achieve.

The many constraints that restrict the range of potential solutions frequently impose restrictions on the generated actions.

The policy is not a synthesis of activities’ sequence through possible known actions merely because the time element which defines the interdependence between these actions is uncharted.

One of the critical ways that various manufacturers promote and differentiate their EVs is by providing a vast network of charging stations throughout the advertising region or reducing the charging time schedule. Rapid technological advancements over the past several decades have led to both huge beneficial improvements in daily living but rising in pollution levels. Extensive and radical sustainable policies have been done to simplify electromobility globally to avoid polluting transportation emissions through secure actions’ (e.g., EVs, electric vehicles) to create clean cities: as cited in Jan. 2018 about the Energy Future and Mobility innovation discussed by the Global Economic Summit. This desired transformation pushes the researchers to enhance the deep learning mechanisms to reduce the error in obtaining the best solution (i.e., reward) when assigning the object with low consumption cost to its destination, so candidate electricity and vehicle EVs to grid station ${\delta }_i$, [5,6] who suggested a reinforcement learning-based/free energy management system. The Markov decision process has been used to represent the state space (i.e., the number of EVs), transition probability, action space (i.e., assignment condition), and reward function of the energy management system as a scheduling issue. Clean cities require some physical infrastructure layers (e.g., electric cars, charging stations, transformers, electric lines, etc.,) and cyberinfrastructure layers (e.g., IoT devices, sensor nodes, meters, monitoring devices, etc.,). These layers make up the majority of the electric vehicle system EVs [7,8] when using successive relations sketched as in Figure 1 based on tackling through one of branches appeared in Figure 2.

The authors accentuate that supporting the infrastructure layers of charging in congestion-ridden urban areas relies on understanding the long-term change in mobility to save them from pollution risk. To close the gap between the demands for EV charging (i.e., requests) [9] and charging station availability (i.e., invitations), which is named bidirectional connectivity network (biCN), the smart reinforcement scheduling policy learning (RL) is needed, which is one of the three keys of machine learning paradigms, whether supervised learning and unsupervised learning for tackling the network elements [10]. The RL can be written by many programming language, the authors tend to formulate the desired (biCN) by MATLAB and discuss their relations which consists of: the operator (O; assigning) learns and trains to accomplish a goal by interacting with the environment through selective policy (π; Function steps) that manage the back propagation for the operator’s series of actions ($a_t$; prefered assigning index. An alleged policy is a function that when applied to a function by deep neural networks, maps every action of an operator that takes to the anticipated result or reward. The Actions Space is limited or continuous, with a combination of all appropriate actions in a certain group that allows different actions and often describe the actions space via the number of moves or the sequence of states (i.e., the policy that guide trajectory $\tau$; Electric vehicle) available to the operator and are represented by real-valued vectors. An operator (O) uses a rule known as a policy to determine what actions to be selected, which may be deterministic, in which case the alluded symbol $\mathsf{\mu }$ is typically used, or stochastic, usually denoted by π. Therefore, Figure 1 discusses the state, actions, parameters architecture distribution relationships based on the following relation expressed by $a_t=\mu \left(s_t\right)\to O_t, a_t~{\pi }_{\theta }\left(a_t|O_t\right)$(hint, denotes “sampling” from the stochastic process)$\mathrm{and} {\pi }_{\theta }\left(a_t|s_t\right)$,$\mathrm{where}$ the policy of fully observed is based on some actions using the states to make the observation like these actions. A state ${\mathrm{s}}_{\mathrm{t}}$ is an exhaustive account of how things are in the world. As a result, every state had a place in the ecosystem under study. While the observation ${\mathrm{O}}_{\mathrm{t}}$ just partially describes this state and may leave out important details. The operator could be able to see a state completely or in part, though. If only a portion, the operator creates an internal state (or state estimation). In order to observe states mechanism, the deep RL approach always uses a genuine array, vector, or polynomial tensor to observe it. The states are the significant features (parameterized) that affect the objective achievement in minimum time and errors, which are called parameterized policies and depend on a set of parameters θ as aforementioned. State transitions are dependent exclusively on recent events and describe what changes occur in the environment timely $\left(s_t\right)$ and the state at time $\left(s_{t+1}\right)$, where if it determinstic $f\left(s_t,a_t\right)$ or stochastic $P(.|s_t,a_t)$ and create the valid actions by the operator according to its policy. The rewards function R which pull the solution in specific paths (i.e., Trajectories) is critically important in searching the route by giving back a reward$r_t=R\left(s_t,a_t,s_{t+1}\right)$, which can be simplified to $r_t=R\left(s_t\right)| R\left(s_t,a_t\right)$ i.e., the value for one movement in the propagation for the state is representative usually as a real +ve|−ve number = the cumulative ${\mathrm{r}}_{\mathrm{t}}$ for a whole trajectory τ at the end) and can be expressed as follow:

$$R\left(\tau \right)={{\displaystyle \sum }}_{t=0}^{T-1}{r}_t={{\displaystyle \sum }}_{t=0}^{T-1}R\left(s_t,a_t,s_{t+1}\right)$$

(1)

The selection policy frequently use discount benefits γ (0,1) received in the candidate trajectory as expressed as $R\left(\tau \right)={{\displaystyle \sum }}_{t=0}^{\infty }{\gamma }^t·r_t={\gamma }^0·r_0+{\gamma }^1·r_1+{\gamma }^2·r_2+\cdots$.

Nevertheless, the operator’s ultimate objective is to choose an ideal policy of action that optimizes the due reward (overall trajectories) and define the probability distributions by assuming stochastic policy and environmental changes that can be expressed as:

$$P\left(\tau |\pi \right)={\rho }_0\left(s_0\right){\displaystyle \prod }_{t=0}^{T-1}P\left(s_{t+1}|s_t,a_t\right)·\pi \left(a_t|s_t\right)$$

(2)

(${\rho }_0$initial state probability) and $P\left(s_{t+1}|s_t,a_t\right) \mathrm{is}$state transition probability of environment [11,12] $\pi \left(a_t|s_t\right)$ is action probability of agent. The expected $\left(r_t\right)$ can be denoted by J (π) that represent the charging stations’ actions${\delta }_i^a$ that can be expressed by

$$J\left(\pi \right)={{\displaystyle \int }}_{\tau }^{n}P\left(\tau |\pi \right)·R\left(\tau \right)={\mathrm{E}}_{\tau ~\pi }\left[R\left(\tau \right)\right]$$

(3)

and from there in the illustrative case study in Section 3, the primary optimization issue in RL may be articulated as alternative ${\pi }^{*}={argmax|}_{\pi }J\left(\pi \right)$, where ${\pi }^{*}$ being the optimal scheduling policy that emphasizes the importance of function value (i.e., selection criteria index discussed in Equation (8)) for a the pair state-action ($Q\left(s_t,a_t\right)$), which represent the EVs’ requests and the stations’ invitations. This issue has several origin of uncertainty because interconnections across different operators, such as EVs, ${\delta }_i$, the grid of electric power, connectivity networks (biCN), and the electricity supplier. The applications of deep RL in this field, focuses on the operational and demand response control between the pair Q above and their energy consumption management [13] with regard to the electric power grid [14]. Therefore, the researchers classified the policies to four functions to reach the destination through minimum route and consuming less time. The first function indexing value (+ve) is called Action-value that provides the anticipated result if starting from the provided state ($s_t^{\tau }$), and performing any action ($a_t$) forever according to policy (π) and delivers the return ($r_t$) if starting from the given state ($s_t^{\tau }$) and repeated along policy π directly, while the function without action is named negative policy value. The first function can be expressed by the $Q^{\pi }\left(s_t,a_t\right)={\mathrm{E}}_{\tau ~\pi }\left[ R\left(\tau \right) | s_0=s_t,a_0=a_t\right]$ and the second is expressed by $V^{\pi }\left(s\right)={\mathrm{E}}_{\tau ~\pi }\left[ R\left(\tau \right) | s_0=s \right]$. The second, if the function followed optimal policy whether on value or on action value can be expressed by $V^{*}\left(s\right)$ and $Q^{*}\left(s,a\right)$ where both act according to optimal policy ${\pi }^{*}$ at Equations (4) and (5), and the bellman style as in Equations (4a) and (5a).

$$V^{*}\left(s\right)=\underset{\pi }{\max }{ \mathbf{E}}_{\tau ~\pi }\left[ R\left(\tau \right) | s_0=s \right]$$

(4)

$$V^{*}\left(s\right)=\underset{a}{\max }{\mathbf{E}}_{s^{\prime }~P}\left[ r\left(s,a\right)+\gamma ·V^{*}\left(s^{\prime }\right) \right]$$

(4a)

$$Q^{*}\left(s,a\right)=\underset{\pi }{\max }{ \mathbf{E}}_{\tau ~\pi }\left[ R\left(\tau \right) | s_0=s, a_0=a \right]$$

(5)

$$Q^{*}\left(s,a\right)={\mathbf{E}}_{s^{\prime }~P}\left[r\left(s,a\right)+\gamma ·\underset{a^{\prime }}{\max }{Q}^{*}\left(s^{\prime },a^{\prime }\right)\right]$$

(5a)

Its fundamental tenet is that “the reward which is expected to gain from being there in a specific position, plus the value of wherever will arrive next,” sums up the worth of the beginning state. The authors note that the connection between $V^{\pi or*}$ and $Q^{\pi or*}$ is as follows, $V^{\pi }\left(s\right)={\mathbf{E}}_{a~\pi }\left[Q^{\pi }\left(s,a\right)\right]$ and the $V^{*}\left(s\right)=\underset{a}{\max }{Q}^{*}\left(s,a\right)$ and special self-consistency equations called Bellman equations are obeyed by the discussed functions according to relative action $a_t$. Therefore, the proportionate benefit of such action is expressed by:

$$A^{\pi }\left(s,a\right)=Q^{\pi }\left(s,a\right)-V^{\pi }\left(s\right)$$

(6)

The taxonomy of reinforcement learning illustrates several algorithms that show the position of the proposed methodology as in Figure 2.

Figure 2 shows that when the operator has access to an environment’s model through forecasting state transitions via specific functions and gains rewards [15], it aims to learn an optimal policy as described by the model-based RL algorithms without actually acting out the policy aims to assigning some of the movement states that request some of the actions Q ($s_t^{\tau }, a_i^{\delta }$) [16]. According to Zhang H. et al. the AlphaZero is interested in chess pieces’ movement issues, which are discussed in the Google DeepMind project (i.e., the predecessor played go to enhance their expectation which considered a scheduling issue) [17]. The lack of scheduling in this issue demands instantaneous request behavior of the mayday case and consumes long time to generate solution. Therefore, the scheduling time is critical indicator [18]. With more EVs on the road, it will be harder to locate connections. Although services such as ChargePoint or ChargeHub (e.g., cloud) give EVs drivers actual-time information guide about available ${\delta }_i$ stations, the capability to reserve connectors for a later time has not yet been implemented [19]. Therefore, using the autonomous internet of things (AIoT) enhances this drawback. The stations were selected according to many models, which helps in reducing the route such as Zero-only action is unlike prior iterations of AlphaGo actions [20,21], whether Go or shogi rules were not initially known to the neural network training. The AI used reinforcement learning to practice playing against itself (i.e., attempting to understand oneself) until it could foresee its own movements and how they would impact the result of the game, in contrast AlphaGo involves slow training and consumes a long time to reach the same level [22]. In the free models in the left side of Figure 2, the operators often do not have access to a ground-truth representation of the environment’s model, which is the fundamental disadvantage and drawback of these algorithms, then need experience through behaving erratically in public environment. Therefore, the operators need learning to catch optimization for selected route or use called Q-learning. The functions of the free-model are enhanced directly by optimizing the parameters ($\mathsf{\theta }$) by gradient ascent toward $J({\pi }_{\theta })$, while Q-learning focuses on the value of the functions and their policies ${\pi }_{\theta }(a|s)$ that are managed by the neural network rules as discussed in Equations (6) and (7).

$$V^{\pi }\left(s\right)={\mathbf{E}}_{a~\pi , s^{\prime }~P}\left[ r\left(s,a\right)+\gamma ·V^{\pi }\left(s^{\prime }\right) \right]$$

(6a)

$$Q^{\pi }\left(s,a\right)={\mathbf{E}}_{s^{\prime }~P}\left[r\left(s,a\right)+\gamma ·{\mathbf{E}}_{a^{\prime }~\pi }\left[Q^{\pi }\left(s^{\prime },a^{\prime }\right)\right]\right]$$

(7)

Moreover, described implementations of the deep RL technique to deal with a variety of new issues, including management rate of aggregated data, convert the data to arguments, flexible dynamical access for the network, bidirectional wireless catching in actual time as illustrated in the right side of Figure 3, network security, etc., and regarding communication and networking. The Lei, L. et al. discuss also the managing of the DRL environment that consists of three layers (e.g., the perception, the network, and the application) by the autonomous IoT (AIoT) systems, which are the next generation of IoT systems [23]. The main advantage of these models (i.e., model-Free RL) [24] are principled techniques for policy optimization that lead to goals directly, which are more trustworthy and stable in deducing the results. Therefore, the authors integrated the Q-Learning and use the Policy-Optimization branch in this taxonomy for finding their proposed methodology. The proposed methodology relies on the Policy Gradient (e.g., Stochastic or Deterministic evaluated by Monte-Carlo or Actor-Critic have multi agents) [25] and calculates $Q_{\theta }\left(s,a\right)$that focuses on increasing the performance of function indexing used by minimizing error when tackled as Bellman equation $Q^{*}\left(s,a\right)$ (gap among destination and the gaind output position) and performs gradient ascent to directly maximize performance when following Equation (8) in Section 3. If the policy optimization is proximal through maximizing a surrogate, objective function may be called PPO (proximal policy optimization). This integration aims to reduce the failure in mechanism behavior of DQN (deep Q-learning network) when obtaining the results to reach $Q^{*}$ as trying to enhance C51 method. The hybridization occurs when DDPG is implemented. The main objective is obtaining the shortest route between the selected beginning position and nearest destination (i.e., Electric charging station), which helps save time in service. Therefore, the method suggested by the Lee et al. and Ahmed M. Abed supports the paper objective and reduces the total trip time while accounting for the dynamic nature of traffic situations and unforeseen future charging requirements [26,27]. It returns the optimum route ($r_t$) and charging stations, which is considered as a challenging issue. The operator’s ultimate objective is to create a policy that yields the highest reward over an extended period of time [28]. For the policy EV to charge properly, it is suggested to use RL to learn about generated government energy usage, which decreases energy consumption costs as well as encourages Evs’ marketing [29,30]. Therefore, many researchers discuss ways to increasing the power grid utilization to reach a steady state of load and to trying to prevent electric potential fluctuations reductions by using RL approaches [31,32,33]. The famous model for selecting the shortest route between source position and destination is the Markov model, which has been mimicked by the proposed methodology, that schedule the assignment of states (EVs) to the preferred charging stations (${\delta }_{\pi }$). Therefore, Wang, K. et al. and Wan, Y. et al. suggest to tackling the intended scheduling such as (MDP) because of its random behavior especially for the traffic parameter [12,34], while EVs arrival time in an uncertain case is based on the recommendations for using DRL [30] to solve the shortcomings of the conventional model-based methods branch that require a smart model to be efficient, where delivering an EV charging approach is scalable and adaptable [35]. A model-free real-time scheduling as illustrated in Figure 2 of electric car charging relies on DRL [36], which is considered an uncertain transition probability and is developed. The performance of proposed policy relies on forecasting method, predicting the readiness of ${\delta }_i$ by the EV using an LSTM neural network to create an intelligent scheduling [37], which present promising results. Therefore, the authors verify those results [38], after mimicking their function formulation recommendations to adjust the operators and mimic the methodology.

2. The Policy Formulation and Research Contribution

The majority of the current research relies on inaccurate traditional econometric or time series approaches that have uncertain behavior, which motivate the authors to study the feasibility of EVs stations and their usability management via tuple parameters (state, action, transaction function, reward). The review guides to use RL with motivated searching mechanism that preferred to be a heuristic to expand the solution space [39,40]. Therefore, adopted policy, a rewarding argument is the gained value from the transaction function ${{\displaystyle \sum }}_{t=1}^{n}J{\left(\pi \right)}_{\theta }$ through sequences of decisions (i.e., Markov principles) enhanced by a metaheuristic technique. The possibility of obliteration and inflation makes the value of the future less valuable than a reward now, $r_{t\to 0}\ge r_{t\to n}$. The movement of state (EV) relies on the available locations $J{\left(\pi \right)}_t$ in the backlogs-list (e.g., the layout map) having two cluster directions, unidirectional or bidirectional. The gained reward $r_t$ tends to support policy ${\pi }_A$ or policy ${\pi }_B,$ when $J\left(\pi \right)$s was not on the proposed routes. The charter of the work is illustrated in

Table 1. The work charter design sheet.

Title	Scheduling EVs Charging Stations on Their Trip Route Using AIoT
Goals	Reduce idle time actions (e.g., setup time, inspection time, response time, etc.) Reduce the service time by divide the charging span into fuzzy zones. Select the preferred station which closer to the main flight route.
Problem Scope: The scope of the work will focus on scheduling the different selection of requests for EVs stations to achieve the goals above through modelling fuzzy network trained by reinforcement learning bidirectional DQN and manage it by the AIoT. The Q, V loss function is used to generate the initial searching values formulated as follow: ${\lambda }_t^Q=(Q(s_t$,$a_t$);$\theta -{\lambda }_t^Q{)}^2$ and ${\lambda }_t^V=(V(s_t$,$a_t$);$\theta -{\lambda }_t^V{)}^2$ The OEE is a verification indicator revealing the effectiveness of the proposed scheduling by three terms: Availability = Station Run Time/Planned Service Time. Performance = (Ideal Charging Cycle Time × Total Satisfied EVs Count)/Station Run Time. Quality = Good EVs servicing Count/Total serviced Count.
Government/Customer Impact: Deployment of EVs to help clean the environment Smart Scheduling Q Network Map Transportation without hindrances.
Business Impact: The expected savings time via comparing the results by Qing Wu, (2019) [38] and Lee et al. (2020) [29] to reduce trip time and increase the stations utilization by measuring the OEE [47].
Current State (Baseline)						Objective and Stretch Target
$OEE=\left[\left(Satisfied EV\times Ideal Charging Cycle Time\right)/Planned service Time\right]$ $OEE=Availability\times Performance\times Quality$
Metrics		“As-is”		“Should-be”		Tolerance	Set Mfg. condition
Standardization		63%		97%		No control	No procedures
Idle-Time		23%		6%		Control	Fixed procedures
Construct cause and effect						Construct FMEA
Policy function: Y = f (battery charge level, route length, trip distance, the role on queue list L)
Major contribution of this work smart fuzzy scheduling for DQN managed by AIoT
Zap Map on google engine search			Pareto Chart		Proposed Solutions

. To improve the searching mechanism of various computations is among the significant parameters to find the best hybridization to meet optimization processes based on fifth different metaheuristic algorithms [41]. The first is the Scatter search, made famous by the Tabu search, and is hybridized with the immune algorithm. The second is the differential evolution algorithm (DE), which is called evolutionary computation because its behavior relies on iteratively trying to improve the candidate’s solution and solve the complex parameter optimization problem to optimize profitability regardless of time. Therefore, the researchers hybridize the Grey Wolf Optimizer (GWO) with the immune algorithm to minimize the total service time, [42] as a third trial. On the other hand, the fourth hybridization is done between the Sine–cosine and Whale Optimization Algorithm, which creates initial random agents and requires them to fluctuate outwards or toward the best possible solution based on sine and cosine functions [43,44,45,46]. Finally, the process parameters were optimized using reinforcement learning and a Bayesian normalized NN utilizing the beetle antennae search (BASNNC) algorithm, which outperformed the previously discussed processes [38]. Therefore, the proposed methodology selects the branch of “need-based,” as illustrated in Figure 2 above, which matched with understanding the behavior of the tackling parameters through servicing (i.e., EVs charging operation) mainly if multiple constraints are considered. Therefore, this work tries to enhance this mechanism to find a preferred solution in the minimum time and reduce the service time under some of the abovementioned considerations. The proposed design consists of two recruitment networks; a deep Q network represent the relation between the state and actions Q ($s_t^{\tau }, a_t^i$) for the best action-value function indexing discriminative features for the idle time. The suggested system does not need to know in advance variables such as arrival and departure times or power usage, in contrast to optimization-based techniques that can be managed by the autonomous clouding technique, because the neural network can estimate the right choice based on the present parameters (${\theta }_i$). The paper discusses the reinforcement learning mechanism for optimization algorithms in the review section and constructs the formulation for the problem that targets reducing the total service time for EVs in the suitable station taking into account the minimum route distance to pair them. Section 2 provides the paper charter that expounds the evaluation indicator, which highlights the contribution as presented in Section 3. Section 4 discusses how to analyze the DQN via fuzzy scheduling iterations to pick good distributions of EVs around the stations on the road by the proposed metaheuristic methodology. Finally, the results are discussed in Section 5. In the end, the conclusions are presented and the drawbacks are discussed, as well as future work directions are envisaged. The conclusion also presents the comparative results with outputs mimicked.

3. Contribution of This Study

The contribution is based on the inquiry about the advantages of using hybrid approaches to be used to maximize the research functions in uncertain networks through elective policy systems. This policy aims to determine minimum route length between the EVs and candidate stations and achieve the related objective which is minimum service time.

• A new routing metaheuristic approach is suggested in order to save expenses, time, and eliminate hardship of travel.
• Analyzing how suggested stations affect the effectiveness of routing heuristics.

${\lambda }_t^Q$ and ${\lambda }_t^V$ are specified as the target values of Q-functions and value functions, respectively, to derive the loss functions. When using value-based approaches, the regression loss may be used to assess how successfully NN mimics Q functions or value functions.

The Environment Layout Map Description

The proposed policy seeks for selecting the shortest route length between main successive EVs requests and their expected service time. There are five sequential actions discussed through Markov to identify the shortest route in lowest time, when modeling the RL (travelling state $\left(s_t\right)$ to a desired place, looking up pricing information, serviceable actions $\left(a_t\right)$, setting up if necessary, and making other quick decisions). The effectiveness and execution of any subsequent actions policy have four tactical choices affecting the result [48]. The analyzed environment (i.e., layout design and its chosen locations) sketched as a rectangular-like illustration is presented in Figure 3.

Shape of Charger Station	Rectangular
Number of stations	$2\le J_{\pi }\le 2100$
Main starting position	Bottom, Right, Up
Street span	As safety instructed
Policy of Service	${{\displaystyle \sum }}_{t=0}^n{\displaystyle {\displaystyle \sum }_{a=1}^m}{\pi }_t^a$
Proposed system	Dynamic and stochastic

In assignment policy, priority facility movement and routing policy are considered the basis for the suggested methodology. The layout design is stressed as an essential component impacting the process performance and routing distance in the proposed partitioning of the map according to the studied characteristics, where the policy consuming time to complete assigning n (locations) to ${\delta }_i$ indicated equals the number of desired selected locations. The rear and front walls of the borders are close by and parallel to the two end streets. Each place on the layout has a definition of each symbol, which is also represented in

Table 2. Symbols describe the illustrated map discussed in Figure 3.

$x,y$	Coordinates of available serviceable stations.	$∆A$	The gap between two streets.
${\delta }_{i,j}$	The number of stations on the map that were tallied (1, 2, 3, …, i, …, m)	$∆F$	The separation between the major thorough fare and the first candidate station.
$N$	The number of EVs places on each street, regardless of whether ${\pi }_A$ or ${\pi }_B$ is used.	$∆B$	The distance between the last $i, j$ on the map’s vertical street and the rear street.
$F{G}_i$	The fuzzy group for selected ${{\displaystyle \sum }}_{i=1}^{N=3}{\delta }_i$	${∆}_l$	The separation between two nearby stations $J_i$ and $J_{i+1}$
${\lambda }_i^d$	The route length under cost consideration	$L$	The list of suggested stations

. Numerous potential stations are suggested on the right or left of the roadway in the proposed layout plan, which was created using Google Maps and is framed in a fixed two-dimensional rectangular interior [19], as shown in Figure 4. Through “Zap Map,” a website displays charging stations accessible in several European nations and a wealth of beneficial information to assist the EVs drivers charge the vehicle quickly and securely via creating most preferred scheduling map according to all requests done in the same time road, to locate the finest station for your needs, such as minimum charging time, waiting time, and closer to the main flight route.

The charging stations ${\delta }_i\left(\pi \right)$ is ascending where $i\in \left\{1,2,\dots ,m\right\}$ is $\left(m-1\right)N+1$, the second station may be available as ${\delta }_2$ is $\left(J-1\right)N+3$, …,$\left(J-1\right)N+i,$ and so on. The distance between the two stations ${\delta }_i$ and ${\delta }_{i+1}$ must be the shortest and calculated by Equation (8).

$${\lambda }_i=min{\displaystyle \sum }_{j=1}^n[∆l(2N-\delta i_x-\delta i_y)+2∆B+\left|{\delta }_x-{\delta }_y\right|∆A+2∆D, ∆l\left(\delta i_x+\delta i_y-2\right)+2∆F+\left|{\delta }_x-{\delta }_y\right|∆A$$

(8)

where $\delta i_x=\left[\raisebox{1ex}{$x$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\right],\dots i_x=x-\left[\raisebox{1ex}{$x$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\right]N,\dots \delta j_y=\left[\raisebox{1ex}{$y$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\right],\dots i_y=y-\left[\raisebox{1ex}{$y$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\right]N$.

At the start of each period $\left(t\right)$ when requested, the drivers must determine the number of serviceable stations $\left(\delta \right)$ and if they are serviceable.

The originality of the suggested method in creating recruitment datasets needed to train the algorithm via simulator because of dearth of datasets [27,32,49]. The sequential stages of proposed methodology characteristics are illustrated in Figure 5.

4. The Methodology Description

The forward propagation of aggregated data inputs and the reverse propagation of errors are two bidirectional processes that make up the classic gradient-based BP neural network learning process. Training is repeated, and deviations and network weight changes are continually computed in the direction of the relative error function gradient. It takes time to get closer to the objective.

Traditional gradient-based BP neural networks do, however, have certain intrinsic drawbacks, such as delayed convergence and a propensity toward local optimums. Researchers often experiment with different activation functions, change the network layout, and enhance weight-definite approaches. The proposed methodology works in case of many requested stations at the same time. The aim is selecting the closest five EVs to specific station according to distance on the main road trip and their requested service. The analysis of the procedures requested to reduce the total idle time of EVs and increase the utilization of the charging stations, which is calculated by the OEE indicator. Since there are an infinite number of possible scenarios, it is impossible to keep the ideal answer for each one of them in a cloud database. This issue served as the impetus for us to create the AIoT-DQN algorithm, which employs a function to identify the optimal course of action in each situation and is controlled by the autonomous Internet of Things AIoT response [39,50], which suggests the weight-and-structure-determination (WASD) algorithms, i.e., technique of employing linearly independent or orthogonal polynomials as activation functions, and several other ways. The relative gradient descent technique used by the heuristic random search algorithm gives it a powerful global search capability integrating the APSO method with a neural network to improve network parameters [40,51]. The proposed model is based on hybridizing with three model-free rules to gain an advantage to tailor some of the heuristic steps empowering the searching mechanism, such as the DQN, and DDPG, and policy gradient as illustrated in Figure 2. The proposed AIoT-DQN methodology was written in MATLAB to tackle the drawbacks in native DQN through two sequential phases [52]. The first phase is interested in building the network for mimicking the environment illustrated in Figure 5 when selecting the EVs and serviceable time at available stations via specific policies to get the minimum route between the car’s position and destination [53]. The second phase is an interest in cost analysis. The different zones of any charging request procedures shown in left side of Figure 5 represented by ($Ji$) have a sequence of actions ($a_i$). Zones (a, b, and d) are the BVA actions (i.e., idle time -service time) that are represented by (select many available stations, inspection procedures before charging, average setup time procedures, arrival distance, and waiting time). At the same time, zone (c) is the VA activity (i.e., service time) which must be scheduled in a minimum period. These zones were deduced from previous studies. Because of its large solution space, the authors resort to smart scheduling solution managed by the AIoT. If there are $N$ of EVs requests and $m$ stations ${\delta }_i$, the total number of potentially viable solution is equal to ${\left(N!\right)}^m$. After analyzing all of the feasible alternatives, an optimal solution for a certain performance metric may be identified among these possibilities. For instance, if there are five EVs making a request at the same time, the Q ($s_t^5, a_t^5$) = $2.48\times {10}^8$ alternatives, which consumes long computational time.

The proposed smart scheduling helps the EV drivers to plan their trip by selecting suitable stations on their roads and desired charging levels. These stations are not conditioned for full charging but charging to a particular level which reduces the queue time, and the remaining charge can be compensated on the road from the scheduled stations. The drivers can accept this plan or cancel to select another station for another charging policy (i.e., dataset’s list). The proposed methodology is identified through many parameters and decision variables as shown in

Table 3. The AIoT-DQN equations parameters and Decision Variables.

Parameters
$C^{\pi A}$	The cost of distance of arrive the EVs to ${\delta }_i$ as suggested via ${\pi }_A$.
$C^{\pi B}$	The cost of distance of arrive the EVs to ${\delta }_i$ as suggested via ${\pi }_B$.
$C^{d\pi B}$	The cost of disposing stations ${\delta }_j$ due to distance or to EVs driver canceled and switch to another ${\pi }_B$.
$h^{\pi A}$	Processing fees for usable station to be available list of EVs in (${\pi }_A$) at period $\left(t\right).$
$h^{\pi B}$	Processing fees for usable station to be available list of EVs in (${\pi }_B$) at period $\left(t\right).$
$I_0^S$	Number of stations ${\delta }_i$ candidate at time ($t$) according of (${\pi }_A$).
$I_0^R$	Number of stations ${\delta }_i$ candidate at time ($t^{\prime }$) according of (${\pi }_B$).
$r_t^{AB}$	Uncertain serviceable number of ${\delta }_j$ in (${\pi }_B$) and/or hybrid at (${\pi }_A$) in period $t+∆t$
${\overline{r}}_t^{AB}$	The average number of $d_i+d_j$ sites that were upgraded to be serviceable during period t is estimated.
${\overline{\widehat{r}}}_t^{AB}$	Maximum variation from the mean for the number of ${\delta }_i$ upgraded to serviceable in period $t$,
${\tau }_t^d$	The maximum number of uncertain requests and (re)requests for the ${{\displaystyle \sum }}_{i=1}^n{\delta }_i$ along the route that can diverge from the mean concurrently until the end of period $\left(t\right)$.
$d_t$	Uncertain ${\delta }_i$’ service time requested for the list $L$.
$b$	EVs backlogging cost to serviceable ${\delta }_j$ station at (${\pi }_B$) after finishing (${\pi }_A$).
$\gamma$	The discount parameters for control reward gained
Decision Variables
$S_t^{\pi A}$	The number of serviceable ${\delta }_i$ that are distributed in period $t$ of (${\pi }_A$).
$S_t^{\pi B}$	The number of serviceable ${\delta }_j$ at (${\pi }_B$) that is updated of (${\pi }_A$) in period $\left(t’\right)$.
$X_t^D$	The number of ${\delta }_i$that form list L and eliminated during period $t’$ owing to cancellation or going against the cost analysis step and being delayed until route actions were complete
$I_t^S$	The number of backlogging EVs $i+j$ at the end of period $\left(t\right)$ for (${\pi }_{A \&B}$).

.

4.1. Phase I: The Smart Fuzzy Scheduling Formulation, Fecrease EVs Service Time

Many authors met to integrate the heuristic steps to enhance their founding solution with the minimum errors. A back propagation neural network based on particle swarm optimization was suggested after they investigated how to carefully choose input parameters to obtain desired outcomes [54]. The conjugate gradient approach [55], the least squares method, and better methods based on numerical optimization exist in addition to those mentioned above. Therefore, the heuristic step is an important approach that must be used in enhancing the solution. The smart fuzzy scheduling for DQN managed by autonomous Internet of Things (AIoT) depends on applying the proposed fuzzy metaheuristic steps expressed in

Table 4. The ($dEVs{S}_t$) pseudo steps to construct the DQ Network.

Step 1:	Construct the serviceable network that represents the EVs requesting the charger stations.
Step 2:	Sort the (${\pi }_A$) generated requested list $L$ for available stations ${\delta }_i$ in ascending order from nearest to farthest.
Step 3:	The route begins from the request point till reaching all station ${\delta }_i$ generated by the policy ${\pi }_{A|B}$.
Step 4:	All serviceable ${\delta }_i^d$ and ${\delta }_j^d$ generated by (${\pi }_A$) and (${\pi }_B$) list for stations on road and on opposite side respectively are subject to constraints as in Equation (7) that lead to select shortest route $\lambda =d_i+d_j$, and go to step 8.
Step 5:	${\lambda }_{im} and {\lambda }_{jn}$are the Euclidean distance between the current ${\delta }_n$ and ${\delta }_m$, where ${\delta }_m$ and ${\delta }_n$ are the first and last station according to the list L of (${\pi }_B$).
Step 6:	Choose the serviceable stations ${\delta }_j$ along the suggested route to achieve the goals of being closest and serving in the least amount of time and saving money in (${\pi }_B$), and take into account the last ($i)$ as the starting point to search others, keeping the previous calculated distance route as ${\displaystyle \sum }{\lambda }_i$.
Step 7:	If the ${{\displaystyle \sum }}_{cost=0}^d{\pi }_A \le {{\displaystyle \sum }}_{cost=0}^d{\pi }_B$, select ${\pi }_A$ all ${\delta }_i$ branch and neglect ${\pi }_B$for all ${\delta }_j$ branch and save it in the backlogging list to check the objective attainability from ${\pi }_A$. Otherwise, hybrid of EVs up to first six object on the route.
Step 8:	If min [(${\lambda }_{im}$> ${\lambda }_{jn}$+ $∆F$)] < y, arrange the stations in list L in descending order, and follow the function indexing Equation (8) to classify the $r_t^{\tau }$.
Step 9:	If all subsequent items in L are serviced, then repeat steps 3 and 4.
Step 10:	Return to the main street before proceeding to the feeding place.

called the decrease EVs service time ($dEVs{S}_t$), and arranges the output index [$\alpha$] and relative index [${\alpha }_r$] in descending order as expressed in Equation (8), then constructs the Gantt chart in its first construction. This schedule presents more effective service-span path than some of the other published rules in the same context. Equation (9) discusses all possible solutions for assigning different EVs in a pair aspect in sequence. The first step computes the result of this rule and determines its priority (priority index). The second step arranges these indexes in descending order for groups of stations that aim to reduce waiting time. If two EVs follow in the arrangement loaded by one station, then test the best starting EV that saves time. This testing is the third step. After that, it reschedules EVs for every station alone in its waiting time by sliding EVs to finish two processes simultaneously, still rescheduling till it stops reducing the total idle time. Optimality is achieved when rescheduling the EVs by the same route with the next assumption. The acronym used in the proposed equation, ($dEVs{S}_t$), this rule is compared with Lee et al. and other published rules; this model is effective in most examples. The formula relies on six parameters as:

ES finalEVs	Earliest start request time of final EVs estimated for certain station.
ETfinalEVs	Executing request time of final EVs.
ES firstlEVs	Earliest start request time to first EVs estimated assigned to a station.
ES predecessorEVs	Earliest start request time of predecessor of first EVs estimated time.
ETpredecessorEVs	Executing request time for EVs selected.
ETfirstlEVs	Executing request time for first EVs estimated chosen time.

$${(dEVs{S}_t)}_{\alpha }=\frac{{\mathit{ES}}_{\mathit{finalEVs}}+{\mathit{ET}}_{\mathit{finalEVs}}-{\mathit{ES}}_{\mathit{firstlEVs}}}{{\mathit{ES}}_{\mathit{predecessorEVs}}+{\mathit{ET}}_{\mathit{predecessorEVs}}+{\mathit{ET}}_{\mathit{firstlEVs}}}$$

(9)

The methodology has pseudo-code composed of ten sequential steps as follow:

4.2. Phase II: Cost Analysis Formulation

According to actions distribution shown in Figure 6, the priority for requests to be sequenced to ${\delta }_i$ is its servicing time, where the shortest one for a request is finishing the request service earlier. One drawback is that the EVs request that picked the longest service time will be serviced last in scheduling, even though it has a high priority. Therefore, find the $C_{max}\left(\pi \right)$ expressed in Equation (10) i.e., the make span index of the processing actions to choose the serviced EVs.

$$C_{max}\left(\pi \right)=\underset{0\le t_1\le t_2\dots \le t_{m-1}\le N}{\max }\left({{\displaystyle \sum }}_{j=0}^{t1}{p}_{\pi }\left(a_j\right)1+{{\displaystyle \sum }}_{j=t_1}^{t2}{p}_{\pi }\left(a_j\right)2+\dots +{{\displaystyle \sum }}_{j=t_{m-1}}^N{p}_{\pi }\left(a_j\right)m\right)$$

(10)

The route length under cost consideration of specific service span generated by policy function can be expressed by Equation (10), where the cost is according to branch $({\pi }_A)$ but not to the proposed trip route, plus to all ${\delta }_{i,j}$ that are serviceable, or if one of them are disposed due to time consideration.

$$\begin{gathered} {\lambda }_i^d=\min {{\displaystyle \sum }}_{t=1}^T\left[h^{\pi A}\right(I_0^S+{{\displaystyle \sum }}_{i=1}^n(r_i-∆B(C_i^{\pi A}-C_i^{\pi B}\left)\right))+I_t^S+C_i^{\pi A}{\lambda }_i{S}_t^{\pi A}+C_i^{\pi B}{\lambda }_i{S}_t^{\pi B}+C^{d\pi B}{X}_t^D]+h^{\pi A}{𝛽}_t{𝜏}_t^r+ \\ T{h}^{\pi B}{I}_0^R+{{\displaystyle \sum }}_{i=1}^t(T-t+1){\stackrel{-}{r}}_t^{AB} \end{gathered}$$

(11)

$$S_t^{\pi A}=S_0^{\pi A}+{{\displaystyle \sum }}_{k=1}^{t-1}(d_k+{\overline{r}}_t^{AB})S_{tk}^{\pi A}, \dots \forall 1<t<T$$

(12)

$$S_t^{\pi B}=S_0^{\pi B}+{{\displaystyle \sum }}_{k=1}^{t-1}(d_k+{\overline{r}}_t^{AB})S_{tk}^{\pi B}, \dots \forall 1<t<T$$

(13)

$$X_t^D=X_0^D+{{\displaystyle \sum }}_{k=1}^{t-1}(d_k+{\overline{r}}_t^{AB})X_{tk}^D, \dots \forall 1<t<T$$

(14)

$$I_t^S=I_{t0}^S+{{\displaystyle \sum }}_{k=1}^{t-1}(d_k+{\overline{r}}_t^{AB})I_{tk}^{\pi b}, \dots \forall 1<t<T$$

(15)

Subject to:

$$I_t^S\le h^{\pi A}\left[I_0^S+{{\displaystyle \sum }}_{i=1}^t(S_i^{\pi A}+S_i^{\pi B}-{\overline{d}}_i)+q_t{\tau }_t^d+{{\displaystyle \sum }}_{i=1}^t{\vartheta }_{it} \right]\dots 1\le t\le T, d\in {\lambda }_i^d$$

(15a)

$$I_t^S\le b\left[I_0^S+{{\displaystyle \sum }}_{i=1}^t(S_i^{\pi A}+S_i^{\pi B}-{\overline{d}}_i)+q_t{\tau }_t^d+{{\displaystyle \sum }}_{i=1}^t{\vartheta }_{it} \right]\dots 1\le t\le T, d\in {\lambda }_i^d$$

(15b)

$$S{I}_i=-{{\displaystyle \sum }}_{\delta =1}^m\raisebox{1ex}{$\left[\delta -\left(2J-1\right)\right]t_{ij}$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$$

(16)

where $S_t^{\pi A}, S_t^{\pi B}, X_t^D>0$ and $S{I}_i\ge S{I}_{i+1}\ge S{I}_{i+2}\dots S{I}_N$.

For each potential realization of $\delta$, Constraints (12) and (16) maintain track till the end of period $t$ or return to station at the end of period $t$. The block of ${{\displaystyle \sum }}_{i=1}^t(S_i^{\pi A}+S_i^{\pi B}-{\overline{d}}_i)={\tau }_t^d/I_0^{R|S}$ represents the capacity of the requested service at the end of the schedule time $t$, which follows Exponential and Weibull behavior throughout the day from 5 a.m.: 11 a.m. and from 5 p.m.: 11 p.m. respectively. Constraint (13) generates a priority index SI as Palmer’s algorithm, i.e., the job ordering. The policy mechanism shown in Equations (11)–(16) should be reformulated to be a traceable model according to index $\alpha$ value, and the suggested cost analysis should be used to regulate the uncertainty behavior of the right-hand sides of the variables $S_t^{\pi A}, S_t^{\pi B}, X_t^D$, $I_t^S,$as discussed in Equation (17).

$${\alpha }_0+{{\displaystyle \sum }}_{t=1}^T({\alpha }_t{a}_t+{\gamma }_t{\widehat{a}}_t)\le 0 \forall -{\gamma }_t\le a_t\le {\gamma }_t, and 1\le t<T$$

(17)

Some researchers reported that ANN, but due to the trajectory, tended to become entrapped in a local optimum, therefore is not used for training alone. This is the main reason that some researchers show that hybrid approaches perform better than the traditional ANN [28,56]. This paper observed the Zap map for a long time and generated 120 random examples for requests from three to six EVs simultaneously at a specific station as selected according to Equations (9)–(16). The proposed methodology explains an illustrative example of serving six actions for six different EVs by tackling the DQN network to serve the EVs in minimum time. The proposed methodology focuses on significant parameters illustrated in Figure 7 and Figure 8 and requested actions according to their policy $\pi$ that guides the driver to the shortest path to the station. The AIoT tackles this hybridization to enhance the efficiency of (biCN) or the bi-directional relationship between the EVs and available stations. While BASNNC search algorithm was used to optimize the process parameters only via reinforcement learning and were superior to the previous mechanisms aforementioned [38]. Therefore, the illustrative methodology examples are compared by them to the study’s efficiency via the OEE indicator.

The time of obtaining results for examples of some EVs that need some of the stations’ actions to be served ranges between 60 and 488 s according to its complexity. The analysis of the results as discussed in the next section of the paper show that the significant factors affect the response of reducing total service time as illustrated in Figure 7 and Figure 8 above, where the actions such as charging point dev., inspection time, and the Battery size must be controlled to manage the average service span-time illustrates in Figure 9. The recharge service RcS problem is modelled in this work and mimicked as Markov movement steps with uncertain transition probability. A deep Q ($s_t^n, a_t^{\tau }$) network with function ranking as discussed in Equation (8) approximation has been utilized to find the optimum EVCS selection policy.

The fuzzy group $F{G}_i=\left\{{\alpha }_{a1\to a3}=J{\left(EVs\right)}_1=6.25, {\alpha }_{a4\to a2}=3.8, {\alpha }_{a4\to a6}=2\right\}$ set is arranged in a descending order, this means the $a_{1~}J{\left(EVs\right)}_1$ precedes $a_{4~}J{\left(EVs\right)}_4\approx J_4$, and $a_4$ precedes $a_{6~}J{\left(EVs\right)}_6\approx J_6,a_{2~}J{\left(EVs\right)}_2\approx J_2.$ On the other hand, the remaining actions have been tested in $F{G}_j$ which are arranged in ascending order, where ${\alpha }_{a3\to a6}=0.3142$, ${\alpha }_{a6\to a3}=0.44$. Therefore, $a_{6~}J{\left(EVs\right)}_6\approx J_6$ precedes $a_{3~}J{\left(EVs\right)}_3\approx J_3$ and so on as shown in

Table 5. {${\mathrm{FG}}_{\mathrm{i}}$} set; the fuzzy index for the groups of jobs {${\mathrm{FG}}_{\mathrm{i}}$} set of actions.

The Fuzzy Index for the Groups of Jobs {Ui} Set
$${\mathbf{J}}_{\mathbf{act}}\left(\mathit{E}\mathit{V}\mathit{s}\right)$$	EV(1)	EV(2)	EV(3)	EV(4)	EV(5)	EV(6)	$${\mathbf{J}}_{\mathbf{act}}\left(\mathit{E}\mathit{V}\mathit{s}\right)$$	EV(1)	EV(2)	EV(3)	EV(4)	EV(5)	EV(6)
a1 then a2	10.5	0.4	13	2.2	0.5	0.8	a4 then a1	−ve	2	−ve	0.112	0.31	−ve
a1 then a3	6.25	2.3	5	−ve	0.54	−ve	a4 then a2	3.8	1.6	0.34	0.78	0.19	0.2857
a1 then a4	2.25	0.1	15	0.47	0.231	0.9	a4 then a3	2.1	5.4	−ve	−ve	0.61	−ve
a1 then a5	5.75	0.8	9	0.88	−ve	0.2	a4 then a5	1.9	2.48	−ve	0.56	−ve	−ve
a1 then a6	6	−ve	30	−ve	0.35	−ve	a4 then a6	2	0.6	1.34	−ve	0.42	−ve
a2 then a1	−ve	1.25	−ve	−ve	0.57	−ve	a5 then a1	−ve	0.083	0.112	−ve	0.82	0.143
a2 then a3	−ve	3.375	−ve	−ve	0.9	−ve	a5 then a2	0.42	−ve	1.67	0.92	0.65	0.762
a2 then a4	−ve	0.625	0.47	−ve	0.567	0.364	a5 then a3	0.21	1.5	0.656	−ve	1.3	−ve
a2 then a5	−ve	1.5	0.07	−ve	0.174	−ve	a5 then a4	−ve	−ve	1.67	−ve	0.82	0.857
a2 then a6	−ve	0.375	1.47	−ve	−ve	−ve	a5 then a6	−ve	−ve	3.34	−ve	1	−ve
a3 then a1	−ve	−ve	0.2	1.34	−ve	0.647	a6 then a1	−ve	3.34	−ve	2.34	0.03	0.934
a3 then a2	1	−ve	2.6	4.7	−ve	1.411	a6 then a2	1.12	2.67	−ve	7.34	−ve	1.802
a3 then a4	−ve	−ve	3	1.45	−ve	1.529	a6 then a3	0.44	9	−ve	1	0.31	0.734
a3 then a5	0.274	−ve	1.8	2.3	−ve	0.705	a6 then a4	−ve	1.67	−ve	2.5	0.03	1.933
a3 then a6	0.312	−ve	6	0.11	0.051	0.353	a6 then a5	0.36	4	−ve	3.67	−ve	1.267

, which expounds 180 direct potential relations and 6! indirect relations.

{1.0/1}	{0.9/4, 1.0/5}	{1.0/2, 0.9/3}
{1.0/2, 0.2/4}	{1.0/5}	{0.7/2, 1.0/3}
{0.5/4, 1.0/5}	{1.0/7}	{1.0/6}
{1.0/4}	{1.0/9}	{1.0/3, 0.9/4}
{1.0/5, 0.9/6}	{1.0/2, 0.8/3}	{1.0/4}
Service after added ${\mathsf{\lambda }}_1^{\mathrm{d}}$	Service after added ${\mathsf{\lambda }}_2^{\mathrm{d}}$	Service after added ${\mathsf{\lambda }}_3^{\mathrm{d}}$
{1.0/1}	{0.9/5, 1.0/6}	{0.9/7, 1.0/8, 0.9/9}
{1.0/3, 0.2/5}	{0.9/10, 1.0/11}	{0.7/12, 0.9/13, 1.0/14}
{0.5/7, 1.0/8, 0.2/9, 0.2/10}	{0.9/15, 1.0/16}	{0.9/21, 1.0/22}
{0.5/11, 1.0/12, 0.2/13, 0.2/14}	{0.9/22, 1.0/23}	{0.9/25, 1.0/26, 0.9/27}
{0.5/6, 1.0/17, 0.9/18, 0.2/19, 0.2/20}	{0.9/24, 1.0/25, 0.8/26}	{0.2/29, 1.0/30, 0.9/31}

Therefore, requests more than N of EVs are a complex problem and needs to be programed on a suitable mobile application. The fuzzy intervals for expected service time according to requested actions (e.g., inspection time) by the EVs according to the bidirectional relationship between the stations and EVs on the road for the illustrative case study are discussed in Table 5 with aggregated data, which constructs the deep Q network illustrated in Figure 11, and other actions also have its intervals.

5. Results Analysis

In this work, we use data aggregation to train the reinforcement learning of the suggested approach using counting of EV charging data from the city of Dundee’s open data site, which presents statistics expound various EV requests [53,57]. Each charging request provides details about the chargers’ monikers, start and end times, energy consumption, power output, and physical locations, as well as the shortest distance, reaching time, and idle time required to accomplish the charging operation and the various six actions. Every studied station has three distinct types of chargers: chargers (7 kW, slow), (22 kW, fast), and (>43 kW, rapid) [49,57]. In the present study, a four-months observed requests dataset aggregating across (127 days) is used to produce the descriptive statistics. The excluded outliers, or 0.83%, are the charging times at rapid type that differ by $\pm 3\sigma$ from the median (the total number of quick charger charging requests used in this investigation is 4645). According to

Table 6. The descriptive analysis according to training data.

Seven Charger Point Type in Station	# of Studied Points of Charging in Selected Stations	# of Charging Requests via Bidirectional Cloud Management	# of Requests per Day per Charging Point over Horizon Period t	Service Span Time (h).
				$\mathit{\mu }$	$\mathit{\sigma }$
Slow (7 KW)	36 (69.2%)	2145	0.446875	194.1	319.7
Fast (22 KW)	9 (17.3%)	716	0.745833	426.4	195.38
Rapid (43 KW or up)	7 (13.5 %)	4645	1.523148	24.08	7.62
Total	52	7506		644.58

, the rapid chargers’ standard deviation is 24.08 min, whereas their typical charging time is 516.5 min. There are five main factors that affect the charging sequence acceleration of EVs such as battery size (the larger the battery capacity, the longer it takes to charge), battery status (empty vs. full, or when it is half full), high vehicle charging rate, high charging rate point, and weather (charging time tends to be longer at lower temperatures, especially when using a fast charger). Moreover, EVs are less efficient at lower temperatures. So, too much travel distance cannot be added according to the charging time. The descriptive statistics of EVs charging requests used in the training study region are described as shown in the Table 6 for a long four months. The preferred scheduling obtained from ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) after 59 iterations in 1-EVs size to 9116 iterations in 6-EVs size to advice the EV drivers and stations ${\delta }_i$by the shared beneficial interests paired together by managing of the AIoT to the deep Q network illustrated in Figure 10 according to the utilization% illustrated in the Figure 11. In case of assigning more than 150 EVs in the same time for specific station which generate non available time. Moreover, if the problem size increases more than 559, EVs will not get any scheduling solution by (BASNNC) algorithm, while the ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) generates solution in time over 5.7 min, while BASNNC fails over 1374 EVs. The absolute average for ideal EVs service time is (644.58/52) = 10.73 h/EVs. The scheduling of the different actions for requested charging orders according to Bayesian regularized BASNNC search algorithm illustrated in Figure 12 to calculate the average time of charging six EVs per hours i.e., 55 h/Six EVs (9.16 h/EV). While the proposed scheduling according to AIoT if integrated in managing the DQN reduces the idle time of the stations and the waiting time of the EVs to gain their requests actions as illustrated in Figure 13 to average 41.25 h/Six EVs (6.875 h/EV). Both are affected by the behavior of the index ${\tau }_t^d/I_0^{R|S}$ throughout the day every 10 min. The average of ${\tau }_t^d=35$ and $I_0^{R|S}=12$ at period 5 a.m.: 11 p.m., and the ${\tau }_t^d=45$ and $I_0^{R|S}=8$ at the period 5 p.m.: 11 p.m. for four months. The tracing of the EVs’ requests behavior reveals that the following exponential distribution in the morning changes to Weibull in the afternoon along the studying interval, which presents the lowest error in servicing time expectation as illustrated in Figure 14 and Figure 15.

The part of results of the 2145 generated hypothetical examples dealing with the EVs requests or stations invitations through (biCN) managed by the AIoT for four months from all EVs to execute six potential actions for specific stations checks the condition of the shortest route between EVs’ requests and the stations on the main three roads stored in dataset at the downtown are listed in the

Table 7. Illustrative solutions for the first group FG for four EVs or less size in [hour].

No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	BASNNC	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	BASNNC	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	BASNNC	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	BASNNC	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	BASNNC
1	12.48	13	7	9.48	12.725	13	7.91	10.78	19	7.217	10.462	25	6.364	8.784
2	8.217	10.627	8	7.217	10.737	14	8.219	11.799	20	9.219	12.929	26	6.969	10.679
3	8.15	12.395	9	9.15	12.67	15	8.364	7.654	21	12.219	12.089	27	11.05	10.92
4	12.55	12.03	10	9.55	12.96	16	7.969	10.099	22	12.15	11.63	28	8.364	8.234
5	10.695	10.175	11	10.695	10.94	17	11.05	10.63	23	9.55	12.602	29	6.969	10.389
6	9.48	12.89	12	7.91	11.33	18	10.91	11.62	24	7.695	10.285	30	7.05	11.76

.

MATLAB is used to predict the total service time to proposed methodology and BASNNC. The aggregated data are classified into four groups according to the problem size, the first is only three EVs up to six EVs and executed all main actions at assigned stations. The average ideal time approximates 9.167 h, while the proposed discuss average approximates 8.562 h, while BASNNC presents average 10.42 h OEE indicates that ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) methodology has 72.4% for generated 2145 (i.e., 43 × 50) hypothetical examples and 59% for other algorithm. The worst results of the hypothetical examples shown in Table 7 according to the proposed methodology ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) have been chosen and tackled by using grey wolf optimization (GWO) and extracting other four groups ($F{G}_{31}: F{G}_{34}$) for resolving their scheduling and obtaining the solutions shown in

Table 8. Another verification for proposed methodology with GWO scheduling in [hour].

No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	GWO
4	12.55	14.1	15	8.364	9.65	21	12.219	13.089	27	11.05	11.92	31	11.62	11.98	33	11.05	13.1
5	10.695	10.32	17	11.05	11.6	22	12.15	12.63	28	8.364	8.173	32	12.78	11.72	34	8.31	9.14

. The worst results of the hypothetical examples are shown in Table 8 and extract other three examples having large scale (more than eight EVs) according to the proposed algorithm ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) have been chosen again and tackled by using the third metaheuristic optimization named (Sine-cosine & Whale; SCW) for resolving their scheduling to check its efficiency [58] and obtain the solutions shown in

Table 9. Another verification for proposed methodology with Sine-cosine and Whale optimization scheduling in [hour].

No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW	No.	$\left(\mathit{d}\mathit{E}\mathit{V}\mathit{s}{\mathit{S}}_{\mathit{t}}\right)$	SCW
5	10.695	11.52	28	8.364	8.37	32	12.78	13.01	35	13.04	13.3	36	12.7	12.7	37	11.05	-----
38	7.62	9.1	39	7.92	11.6	40	7.6	12.7	41	8.04	11.78	42	7.54	10.65	43	9.67	-----

. The average service time of proposed ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) if compared by the GWO is superior by only 4%.

Table 7, Table 8 and Table 9 indicate the test of the average service time of Q (s, a) groups, consisting of 50 hypothetical examples that have the same number of EVs and requested actions with different arrangements, and wait for the solution to be extracted after 60 s or less after running the methodology by the MATLAB code. The authors noticed the failure of SCW in obtaining solutions in time and for a long time, reaching 26 min for all these examples that have more than seventeen EVs’ requests simultaneously. While the solutions of GWO and SCW are close to the size of the problems, they are less than six EVs/point charger/station. The proposed methodology is superior to SCW, GWO, and over six EVs up to eight. The average service time of proposed ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) if compared by the GWO is superior by only 15%.

6. Conclusions

Many authors tried to integrate the heuristic steps to enhance their founding solution with the minimum errors [39,54,55]. This work tries to modify the DQN searching mechanism for tackling the uncertain EVs requests and the electric charging stations’ invitations to find the preferred assignment in the minimum time reliving it by guiding it to the closest electrical charging station. This bidirectional connectivity is managed by the AIoT, which tries to achieve two objectives. The first is to reduce its scheduling of service time and must be on the trip’s road through analyzing the route distance length. Therefore, the proposed methodology consists of two sequential phases, the first aims to constructing a fuzzy metaheuristic scheduling steps enhancing the searching in the DQN for the best action-value function (Equation (8)). Figure 9 discusses the network managing by autonomous Internet of Things (AIoT) to manage the requests and invitations for the EVs during their trips for decreasing EVs service time ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) [29], and gained important feature, which are not required in advance variables such as arrival and departure times or power usage, in contrast to optimization-based techniques that can be manage by the autonomous clouding technique, because the neural network can estimate the right choice based on the present parameters $\left({\theta }_i\right)$. The second phase carried out the cost analysis for assigning specific $N$ of EVs to preferred m of station to begin extracting the Gantt chart for distributing EVs at the same time to receive request actions, especially if taken advantage of scheduling EVs’ servicing distribution exponential or Weibull for the charging actions rate during service as illustrated in Figure 14 and Figure 15. The proposed method is verified by computing the OEE for proposed and comparative presented by Qing Wu, [38]. Because of its large solution space ${\left(N!\right)}^m$, the authors resort to smart scheduling solution programming. The proposed AIoT-DQN methodology was written in MATLAB to tackle the drawbacks in native DQN [52,59]. Therefore, the metaheuristic was a preferred mechanism to tackling the problem. The aggregated data are classified into four groups according to the problem size, the first is only three EVs up to six EVs and executed all main actions at assigned station. The average ideal time approximates 9.167 h and the proposed method has an average of approximately 6.875 h, while BASNNC presents an average of 10.47 h. The OEE indicator indicates that equal 79.06% for ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) methodology and 58.7% for other algorithms discussed above.

Table 10. The comparative average of thirty examples for four different problem sizes.

	Three EVs	Four EVs	Five EVs	Six EVs	Eight EVs	Up to
Extracting solution time [s]	6.6	8.1	21.3	34.7	58.6	mins
Average distance among the perfect three stations and EVs’ request position [km]	7.8	8.1	8.4	10.6	16.1	21
Ideal servicing time [h]	7.121	7.535	7.918	8.075	10.2	Over day
Average Span time of ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) [h]	6.921	7.143	7.474	6.876	7.9	infeasible
Average Span time of BASNNC [h]	9.006	9.221	10.743	11.761	16.1	-------
Average Span time of GWO [h]	9.215	9.7	10.74	11.361	18.7	infeasible
Average Span time of SCW [h]	9.153	9.825	10.241	11.162	14.6	-------
OEE ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$)	78.1%	75.2%	72.1%	76.5%	54.1%	-------
OEE of BASNNC	30.1%	29.1%	43.7%	71.0%	52.4%	-------
OEE of GWO	33.1%	35.8%	43.7%	65.2%	51.2%	-------
OEE of SCW	32.2%	37.5%	37.0%	62.3%	51.1%	-------

shows some comparative indicators to check the scope of superiority of ($\mathit{dEVs}{\mathit{S}}_{\mathit{t}}$) over BASNNC and BASNNC algorithm in finding scheduling distribution for different EVs that need some service actions at the same time.