Multi-Step Parking Demand Prediction Model Based on Multi-Graph Convolutional Transformer

Abstract

The increase in motorized vehicles in cities and the inefficient use of parking spaces have exacerbated parking difficulties in cities. To effectively improve the utilization rate of parking spaces, it is necessary to accurately predict future parking demand. This paper proposes a deep learning model based on multi-graph convolutional Transformer, which captures geographic spatial features through a Multi-Graph Convolutional Network (MGCN) module and mines temporal feature patterns using a Transformer module to accurately predict future multi-step parking demand. The model was validated using historical parking transaction volume data from all on-street parking lots in Nanshan District, Shenzhen, from September 2018 to March 2019, and its superiority was verified through comparative experiments with benchmark models. The results show that the MGCN–Transformer model has a MAE, RMSE, and R² error index of 0.26, 0.42, and 95.93%, respectively, in the multi-step prediction task of parking demand, demonstrating its superior predictive accuracy compared to other benchmark models.

1. Introduction

In recent years, with the rapid development of modern society, the number of motor vehicles in cities has shown an increasing trend year by year. According to statistics from the Chinese Ministry of Public Security, the total number of motor vehicles in the country reached 411.7 million in 2022, with 319 million being automobiles [1]. As the number of motor vehicles grows significantly, urban parking spaces cannot effectively meet the demand for parking. According to related studies, on average, each driver wastes 17 h per year searching for available parking spaces, and this time will further increase with the growth in vehicle ownership [2]. The trend of changes in parking spaces and motor vehicles from 2016 to 2022 is shown in Figure 1 [1,3]. Therefore, the problem of “parking difficulty” in cities is gradually evolving into a major obstacle to urban development. Despite government efforts to address the “parking difficulty” problem by expanding parking spaces to alleviate the issue of excessive parking demand, the current utilization of existing parking spaces still faces certain problems: parking demand is unevenly distributed in space and mainly concentrated in some areas, while the utilization rate of parking spaces in other locations is insufficient.

Parking demand prediction is a method based on historical parking demand data within a specific area to forecast parking demand for future time intervals. By predicting future parking demand within the area, it can effectively improve the utilization and management efficiency of parking spaces, greatly alleviating the uneven distribution of parking demand. Numerous models have been proposed in past research for parking demand prediction tasks. In the early stages, mathematical models such as Autoregressive Integrated Moving Average (ARIMA) models, Markov chain (MC) models, Kalman filter models, etc., were used to analyze parking demand time series for prediction tasks [4]. However, these models failed to integrate the spatial distribution patterns of parking demand and did not achieve a good learning accuracy in predicting parking demand. Subsequently, to improve the predictive performance of parking demand, researchers employed machine learning models by applying machine learning and deep learning models to parking demand prediction tasks. Some studies also utilized hybrid methods for parking demand prediction models. However, these studies only considered the static topological relationships between parking lots and ignored the temporal dynamics of the correlation between parking lots. Furthermore, parking often occupies spaces for extended periods, so the task of predicting parking demand needs to forecast demand for multiple time intervals in the future. The models proposed in the previous research often do not perform well in multi-step prediction tasks.

In this paper, we aim to propose an innovative deep learning algorithm called Multi-Graph Convolutional Network–Transformer (MGCN–Transformer) to consider the time-dependency and complex spatial correlations of parking demand in long-term prediction tasks. The key contributions of this paper are as follows:

(1)

This study is the first to use MGCN–Transformer for predicting parking demand over multiple time intervals. By exploring complex spatiotemporal dependency features, it aims to reveal the spatiotemporal distribution characteristics of parking demand under the influence of multiple features;

(2)

This study aims to investigate the impact of multiple features on the spatiotemporal distribution of parking demand. Through feature ablation experiments, it analyzes the influence of each feature on the prediction accuracy of the spatiotemporal distribution characteristics of parking demand;

(3)

This study trained with real data from on-street parking lots in Nanshan District, Shenzhen, and validated the performance of the MGCN–Transformer model in multi-step parking demand prediction through model comparison and analysis. The results indicate that the MGCN–Transformer model demonstrates robustness and adaptability in predicting parking demand for different combinations of input and output steps.

The remaining sections of this paper are organized as follows. Section 2 reviews previous research on parking demand prediction. Section 3 outlines the relevant definitions of parking demand prediction. Section 4 introduces the structure and mathematical formulas of the MGCN–Transformer model. Section 5 presents the model performance evaluation based on real data from Nanshan District, Shenzhen. Section 6 summarizes the content of this study.

2. Related Work

Parking demand prediction, as one of the foundations of intelligent parking management systems, has been the focus of extensive research by numerous scholars. These studies primarily aim to predict future parking demand by proposing parking demand prediction models, which can be categorized into three main types: statistical models, machine learning models, and deep learning models.

In the early stages of research, scholars used statistical models for parking demand forecasting, including ARIMA, MC, and Kalman filter models. These models analyzed the time series data to predict future parking needs. Boyles et al. [4] used a continuous-time MC model to forecast parking demand and space utilization; Zheng et al. [5] applied MC models to simulate parking demand, introducing a method for determining forecast intervals based on parking trends. Tang et al. [6] compared ARIMA, Kalman filter, and Back Propagation (BP) models, finding that the ARIMA and BP models had better accuracy. However, these models, due to their simple structure, struggled with complex, multi-step parking demand predictions.

As the field of traffic prediction has advanced, simple mathematical models are no longer sufficient for our prediction needs. Consequently, machine learning models, with their superior learning capabilities, have been applied to short-term parking demand forecasting tasks. Zheng et al. [7] used a Support Vector Machine (SVM) approach to predict short-term parking occupancy rates. Slavova et al. [8] utilized decision tree models to forecast future parking demand based on the time features and historical occupancy data of individual truck parking lots. Zan et al. [9] employed Gradient Boosting Decision Trees (GBDTs) to predict time-varying parking demand in urban areas, using license plate recognition data, parking lot entry and exit vehicle data, and taxi GPS data. Ye et al. [10] proposed a short-term prediction model for available parking spaces based on GBDT and a Wavelet Neural Network (WNN) and obtained the initial weight of the optimal threshold through Particle Swarm Optimization (PSO). Parmar et al. [11] applied Artificial Neural Networks (ANNs) to analyze the relationship between driver characteristics and parking time to predict parking demand. Paudel et al. [12] developed a linear regression model and two ANN frameworks to predict parking demand by dividing campus parking lots into Parking Demand Zones (PDZ). The best feature selection varies with different scenarios, and since the optimal features for machine learning prediction tasks are often chosen manually, relying solely on manually selected features may not meet the high demands of complex scenarios. This can result in less accurate parking demand predictions when using machine learning methods for complex issues.

To address the high demands of parking demand prediction in complex scenarios, deep learning models are widely used in traffic forecasting, especially for short-term parking demand prediction. Long Short-Term Memory (LSTM) neural networks are effective in handling the non-linearity and randomness of complex time series and can remember internal correlations within time series, achieving good results in urban dynamic traffic flow prediction. Shao et al. [13] proposed a new framework based on recurrent networks and used the LSTM model for parking demand prediction by taking parking occupancy rates and vehicle departure probabilities as performance indicators. Anagnostopoulos et al. [14] utilized the LSTM network for parking demand prediction within a Multi-Agent System (MAS) framework, achieving higher daily parking demand prediction accuracy. Considering potential noise in the raw data, some studies use wavelet algorithms to process the data. W et al. [15] applied a heuristic threshold algorithm to denoise wavelet-decomposed traffic flow data and used the LSTM model for short-term traffic flow prediction. Ji et al. [16] leveraged the excellent learning performance of WNNs in short-term prediction, applying WNN methods to parking guidance systems to develop a short-term available parking space prediction method. Wang et al. [17] combined a WNN for decomposing and modeling the original time series with PSO for initial weight and optimal threshold calculations, designing an Improved Wavelet Neural Network (IWNN) model to predict available parking spaces. Li et al. [18] proposed a hybrid prediction model based on improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) and gated recurrent unit (GRU) models for parking demand prediction tasks, using GRU neural networks for training and prediction after decomposing the parking time series.

In addition to temporal features, spatial correlations between parking lots also affect parking demand to some extent. To account for these spatial correlations, Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs), both capable of extracting spatial correlations, have been applied to parking demand prediction tasks. Yang et al. [19] used a hybrid model combining GCN and LSTM. This model employs Graph Convolutional Neural Networks (GCNNs) to extract spatial relationships in large-scale networks and recurrent neural networks (RNNs) with LSTM to capture temporal features, thus addressing parking occupancy prediction from both spatiotemporal perspectives. Ma et al. [20] proposed a Spatiotemporal Graph Attention Long Short-Term Memory (STGA-LSTM) neural network framework for short-term bike-sharing demand prediction using multi-source data. Zhao et al. [21] employed MGCN and LSTM networks to analyze the spatiotemporal correlations of parking demand and validated the model’s accuracy using historical transaction data.

With the further development of deep learning research, multi-step sequence prediction has become increasingly popular. Ye et al. [22] proposed a prediction model based on attention mechanisms and Temporal Convolutional Networks (TCNs) for the multi-step prediction of ride-hailing demand within a region, validating the model’s accuracy through comparative experiments. Liu et al. [23] introduced a Multi-Sequence Spatiotemporal Feature Fusion Network (MSSTFFN) based on trend decomposition, using Graph Convolutional Networks and Transformer networks to predict peak hour passenger flow at urban rail transit stations after seasonal trend decomposition. In the realm of multi-step parking demand prediction, Mei et al. [24] proposed a multi-step prediction algorithm using Fourier-Transform Least Squares Support Vector Machines, addressing the complexity of parking space occupancy time series. Chen [25] employed Wavelet Transform (WT) for multi-scale decomposition and reconstruction, integrating BP neural networks with multi-step prediction strategies for forecasting various components. Liu et al. [26] constructed a deep learning network combining CNN, Conv-LSTM, and LSTM modules for the multi-step prediction of shared parking inflow and outflow in different regions. Ji et al. [27] developed a multi-step prediction method combining WT, an ANN, and available parking spaces (APSs), comparing the prediction accuracy across different strategies.

In summary, deep learning models, compared to statistical and machine learning models, offer more complex network structures and superior feature extraction capabilities, making them better suited for fitting real data in parking demand prediction tasks. The time-related aspects of parking demand prediction often use time-recurrent neural networks, such as RNNs and LSTMs. However, these models tend to accumulate prediction errors with increasing forecast horizons, limiting their effectiveness in multi-step predictions. Thus, RNNs and LSTMs may not be ideal for long-term parking demand forecasting. Regarding spatial feature correlations, the existing research frequently uses CNNs to capture local spatial information. While effective for localized features, this approach does not fully represent the overall spatial correlations and does not adequately consider factors such as land use, road infrastructure, and travel demand attributes. Therefore, to improve the prediction accuracy, a model that incorporates overall regional characteristics for long-term parking demand forecasting is needed.

3. Problem Definition

The intelligent parking system has generated a massive amount of historical parking transaction data. Based on this, research will be conducted on the temporal correlation, spatial correlation, and Point of Interest (POI) feature correlation of parking demand within the region, while also considering the impact of external variables such as weather and holidays on parking demand. The parameters involved in the research are shown in

Table 1. Notations used in this study.

Notations	Descriptions
$adj\_dis$	Connection matrix between parking lots
$i$	Parking lot No. $i$
$j$	Parking lot No. $j$
$d_{i,j}$	Distance between parking lot $i$ and $j$
$\sigma$	Standard deviation of distances between all parking lots
$adj\_poi$	POI relationship matrix between parking lots
$E()$	Expectation
$R_{area}$	The degree of correlation between different parking lots
$X_i$	POI feature count of parking lot $i$
$X_j$	POI feature count of parking lot $j$
$Pearson()$	Pearson coefficient
$A(A\in {ℝ}^{N\times N})$	Parking lot space connection matrix
$X(X\in {ℝ}^{N\times F^0})$	Time series matrix of parking demand
$N$	Number of regional parking lots
$F^0$	Historical parking demand of each parking lot
$H(H\in {ℝ}^{N\times F^0})$	Spatial feature relationship matrix between parking lots
$f()$	Activation function
$\stackrel{\wedge }{D}$	Degree matrix of spatial connectivity in parking lots
$W$	Weight matrix
$dat{a}_{total}$	Overall spatial characteristic relationship
$dat{a}_{dis}$	Spatial distance characteristic relationship
$dat{a}_{POI}$	POI feature relationship
$\alpha$	Learnable parameters for feature weighted fusion
$\odot$	Dot product operation
$Wea(Wea\in {ℝ}^{N\times F^0})$	Weather characteristic information
$P(P\in {ℝ}^{N\times F^0})$	Workday label
$Inpu{t}_{demand}$	Collection of parking demands
$Inpu{t}_{\mathrm{exogenous}}$	Set of exogenous variables for parking demand
$k$	Enter time step size
$n$	Predict time step size
$P{E}_{(POS,2m)}$	Position encoding matrix for even positions
$P{E}_{(POS,2m+1)}$	Position encoding matrix for odd positions
$pos$	Required encoding data location
$m$	Vector index dimension
$d_{model}$	Total number of vector dimensions
$Attention()$	Attention mechanism calculation function
$\mathrm{softmax}()$	SoftMax activation function
$Q$	Query vector matrices
$K$	Key vector matrices
$V$	Value vector matrices
$d_k$	The feature dimension used for normalization processing
$y_i$	Actual parking demand of parking lot $i$
${\widehat{y}}_i$	Predicted parking demand for parking lot $i$
$\overline{y}$	Average parking demand in the parking lot

.

3.1. Parking Demand Features

The research analyzed historical parking transaction volumes from September 2018 to March 2019 across all roadside parking facilities within Nanshan District, Shenzhen, Guangdong Province, China. The data encompass a range of critical details such as the location of each parking spot, unique parking identifiers (IDs), timestamps for vehicle entry and exit, and comprehensive payment information. Partial raw data are shown in

Table 2. Partial historical parking transaction volume data in Nanshan District.

Region	Area	Parking Lot	Berth Number	Delivery Time	Storage Time	Fee Situation
Nanshan District	Nantou District	Changxing Road	206001	2 September 2018 09:30:00	2 September 2018 09:12:02	Free
Nanshan District	Nanshan Central District	Hyde Second Road	204171	3 December 2018 20:13:07	3 December 2018 20:00:05	Free
Nanshan District	Houhai area	Dengliang Road	205130	25 January 2019 20:00:47	25 January 2019 18:56:47	CNY 25

.

It was found that there are approximately 43 roadside parking facilities dispersed throughout the area, and their geographical locations are vividly depicted in Figure 2. Acknowledging the broad temporal extent of the data, the study employed a meticulous 15 min time interval for the calculation of parking demand frequencies. This statistical approach allowed for the creation of time series data that precisely capture the dynamic patterns of parking demand, as exemplified in

Table 3. Partial parking demand time series data.

Parking Lot Number	No. 1	No. 2	No. 3	…	No. 42	No. 43
2 September 2018 7:30–07:45	0	0	0	…	0	0
2 September 2018 07:45–08:00	18	43	1	…	1	1
2 September 2018 08:00–08:15	20	49	3	…	1	0
2 September 2018 08:15–08:30	19	39	6	…	1	1
…	…	…	…	…	…	…

.

3.2. Parking Lot Distance Features

To explore the spatial correlation between parking lots and its impact on the parking demand distribution, this study will utilize the GCN model to consider the spatial distance features of parking lots. The spatial correlation in the GCN network structure is represented by an adjacency matrix, and as there are no obvious connection relationships $adj\_dis$ between parking lots, the study will use a Gaussian kernel function to define the adjacency matrix. The specific calculation formula is as follows:

$$adj\_di{s}_{i,j}=\left\{\begin{array}{cc}\exp (-{\displaystyle \frac{d_{i,j}^2}{{\sigma }^2}})& \begin{array}{ccc}if& d_{i,j}>0& \end{array}\\ 0& otherwise\end{array}\right.$$

(1)

where $d_{i,j}$ represents the distance between parking lots $i$ and $j$, and $\sigma$ represents the standard deviation of all distances.

3.3. POI Information Features

Parking demand, as a product of travelers’ travel behavior, is also influenced by factors such as land use, road transportation facilities, and the nature of workplace units. Therefore, it is necessary to consider the impact of various types of POI information features on the parking demand distribution.

As the central urban area of Shenzhen, Nanshan District presents diversified POI features, covering multiple fields such as commercial services, residential areas, transportation facilities, educational institutions, leisure, and entertainment. This makes research on the prediction of parking demand based on the historical transaction volume data of parking lots in the region highly valuable in practice and in theoretical significance.

This research employs the Gaode Maps platform to gather data on the number of POI features within a 1 km vicinity of each parking facility. The Pearson correlation coefficient is used to calculate the correlation of POI information between each parking lot, and a Gaussian kernel function is used to construct the POI adjacent matrix $adj\_poi$ The formula for calculating the Pearson correlation coefficient is as follows:

$$Pearson(X,Y)={\displaystyle \frac{E(XY)-E(X)E(Y)}{\sqrt{E(X^2)-E^2(X)}\sqrt{E(Y^2)-E^2(Y)}}}$$

(2)

$$R_{area}=Peason(X_i,X_j)$$

(3)

where $E()$ represents the mathematical expectation; $R_{area}$ represents the degree of correlation between different areas; and $X_i$ and $X_j$, respectively, represent the number of POI features for parking lots $i$ and $j$.

3.4. Other Features

In addition to spatiotemporal features, exogenous variables such as weather features and holiday features can affect travelers’ travel behavior, thereby leading to changes in the spatiotemporal distribution of parking demand. Therefore, we chose to compare the parking demand data of Parking Lot 1 on two days of heavy rain and sunny weather (both days are non-working days).

In terms of weather features, severe weather conditions such as heavy rain will significantly reduce travelers’ willingness to travel, thereby decreasing parking demand; high temperatures can lead to an increase in the demand for private car travel, thus increasing parking demand. As shown in Figure 3, the parking demand curve clearly indicates that compared to rainy weather, parking demand is significantly higher on sunny days. Therefore, this study aims to incorporate weather feature information into the predictive model to achieve high-precision parking demand prediction tasks.

In terms of holiday features, due to the influence of factors such as regional land use and the nature of workplace units, the distribution and total volume of parking demand in the same area varies. As shown in Figure 4, the parking demand in the same area exhibits significant differences between weekdays and non-weekdays. Compared to weekdays, parking demand on non-weekdays occurs later in the day and has a larger overall volume. This is because residents do not need to go to work on non-weekdays, and their travel behavior mostly consists of non-work commutes, resulting in later appearances and a larger total volume of parking demand. Therefore, incorporating holiday features into this study will better achieve accuracy in parking demand prediction tasks.

4. Methodology

4.1. Overall Architecture

This study proposes the MGCN–Transformer multi-step parking demand prediction model to predict the distribution of parking demand in the region over a longer period. As shown in Figure 5, this model consists of four main modules. Firstly, there is the data processing module, which includes parking lot feature data and exogenous variable data. The parking lot feature data include the adjacency matrix of parking lots, the POI feature adjacency matrix, and the parking demand time series. The exogenous variables include holiday features and weather features. Secondly, there is the MGCN module, which comprises two GCN layers and a feature fusion layer. The two GCN layers extract information from the adjacency matrix of the parking lots and the POI feature adjacency matrix and combine it with the parking demand time series. Following this, the two outputs from the GCN layers are dynamically weighted and fused by the feature fusion layer. Next is the Transformer module, where a zero sequence is constructed. Through positional encoding, the exogenous variables and the upper-layer output are combined with the zero sequence and input into the encoder and decoder layers. The output of the Transformer module is then obtained from the decoder layer. Finally, the encoder layer output is linearly transformed by the fully connected layer to produce the prediction results.

Overall, by effectively utilizing the collaboration between the various modules, the proposed model in this study demonstrates strong capabilities in extracting the spatiotemporal feature correlations of parking demand, thereby significantly enhancing the accuracy of multi-step parking demand predictions. The algorithm process is detailed in Algorithm 1.

Algorithm 1. Training algorithm of MGCN–Transformer

Input: Time series matrix of parking demand $X(X\in {ℝ}^{N\times F^0})$; Connection matrix between parking lots $adj\_dis$; POI relationship matrix between parking lots $adj\_poi$; Weather characteristic information $Wea(Wea\in {ℝ}^{N\times F^0})$; Workday label $P(P\in {ℝ}^{N\times F^0})$;Enter time step size $k$;Predict time step size $n$

Output: Learned MGCN-Transformer model

1: for all available time intervals t

$\left(\mathrm{a}\right) Inpu{t}_{demand}\leftarrow X(X\in {ℝ}^{N\times F^0})$$; Inpu{t}_{demand}\leftarrow adj\_dis$$;Inpu{t}_{demand}\leftarrow adj\_poi$

$\left(\mathrm{b}\right) Inpu{t}_{\mathrm{exogenous}}\leftarrow Wea(Wea\in {ℝ}^{N\times F^0})$$; Inpu{t}_{\mathrm{exogenous}}\leftarrow P(P\in {ℝ}^{N\times F^0})$

$\left(\mathrm{c}\right) \mathrm{Put} \mathrm{the} \mathrm{training} \mathrm{sample} (Inpu{t}_{demand},Inpu{t}_{\mathrm{exogenous}})$$\mathrm{into}Input$

$\left(\mathrm{d}\right) Output\leftarrow [N_{t+k}]$

2: end for

3: Repeat

4: Optimizing parameters using MSE loss

5: Until stopping criteria are met

6: output the learned MGCN-Transformer model

4.2. GCN

The parking demand between parking lots within the area will be affected by spatial distance. To improve the accuracy of parking demand prediction tasks, it is necessary to accurately capture the spatial characteristics of parking demand within the area. This paper adopts the GCN deep learning model to process and learn the connection graph structure between parking lots and adjacent parking lots, learn the spatial graph structure of each parking lot, and explore the spatial feature relationships of parking lots.

The input matrix consists of two parts: the spatial connection relationship matrix $A(A\in {ℝ}^{N\times N})$ between parking lots and the parking demand time series matrix $X(X\in {ℝ}^{N\times F^0})$, where $N$ is the number of parking lots in the area and $F^0$ is the historical parking demand of each parking lot. The calculation formula for the spatial feature relationship matrix $H(H\in {ℝ}^{N\times F^0})$ is

$$H=f({\hat{D}}^{-{\displaystyle \frac{1}{2}}}\tilde{A}{\hat{D}}^{-{\displaystyle \frac{1}{2}}}XW)$$

(4)

The formula is where $f()$ represents the activation function, $\hat{D}$ is the degree matrix, and $W$ is the weight matrix.

By computing the above formula, the GCN layer effectively combines the connectivity information of each parking lot in the graph structure with its parking demand, thereby producing a feature matrix $H(H\in {ℝ}^{N\times F^0})$ containing spatial relationships.

The GCN fusion layer integrates the outputs of multiple GCNs through weighted fusion, resulting in a dynamic sequence matrix $dat{a}_{total}$ with multiple spatial features.

$$dat{a}_{total}=\alpha \odot dat{a}_{dis}+(1-\alpha )\odot dat{a}_{POI}$$

(5)

where $\alpha$ is a learnable parameter used for the weighted fusion of the $dat{a}_{dis}$ and $dat{a}_{POI}$ feature parts, and $\odot$ represents the dot product operation, where $dat{a}_{dis}$ and $dat{a}_{POI}$ respectively denote the spatial connection relationship and the POI feature sequences fused with the parking demand time series.

4.3. Transformer

To better explore the temporal characteristics of parking demand in the region, as shown in Figure 6, this paper adopts a Transformer deep prediction model consisting of position encoding and several encoders and decoders stacked together. The encoder includes a multi-head attention mechanism layer, two residual connection layers, and a feed-forward neural network. The decoder comprises two multi-head attention mechanism layers, three residual connection layers, and a feed-forward neural network. Unlike the encoder, the first multi-head attention mechanism layer in the decoder incorporates a masking mechanism, primarily to prevent the model from using future information for prediction.

4.3.1. Positional Encoding

In the task of predicting parking demand, the time sequence of parking demand will greatly affect the accuracy of the model prediction task. Therefore, it is necessary to encode the position of the input sequence in advance to preserve the time characteristics in the parking demand time series. To ensure the effectiveness of the position encoding, the study adopts a combination of sine and cosine functions, and the calculation formula is as follows:

$$P{E}_{(POS,2m)}=\sin (pos/{10000}^{2m/d_{model}})$$

(6)

$$P{E}_{(POS,2m+1)}=\cos (pos/{10000}^{2m/d_{model}})$$

(7)

where $P{E}_{(POS,2m)}$ and $P{E}_{(POS,2m+1)}$ are elements of the positional encoding matrix; $pos$ represents the position order; $m$ represents the vector dimension index; and $d_{model}$ represents the vector dimension.

4.3.2. Multi-Heads Attention

The multi-head attention mechanism transforms the input sequence into query (Q), key (K), and value (V) representations through linear transformations, by using multiple sets of attention to combine each query with each key to perform the attention calculation. After the computation, the SoftMax function is used for normalization to obtain the attention weight matrix. Finally, the output of the multi-head attention mechanism is obtained by multiplying the attention weight matrix with the value matrix and summing up. The calculation formula is

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}})V$$

(8)

where $Q,K,V$ represent the query, key, and value vector matrices and $d_k$ is the feature dimension used for normalization processing.

5. Results and Analysis

5.1. Model Configurations and Evaluation Metrics

This study utilized a personal computer with an Intel^® Core™ i5-11400H CPU and an RTX 3050 Laptop GPU and employed the PyTorch open-source machine learning framework to build the deep learning model. The dataset was divided into 70% for training and 20% for testing.

As described in Chapter 4, the MGCN–Transformer model consists of four modules, three of which involve deep learning. The hyperparameter settings for each part are as follows:

There are two independently trained GCN layers, and the hyperparameters of these two GCN layers are determined based on the feature dimensions of the input matrix, with a hidden layer dimension of 512.

In the Transformer module, four attention heads are used, and both the encoder and decoder are configured with six layers.

In the fully connected layer module, the input feature dimension is 512, and the output feature dimension is 43, achieving a non-linear mapping of the feature dimension.

The study uses Mean Squared Error (MSE) as the model loss function.

The study employs the Adam optimization algorithm for stochastic gradient descent (SGD) to update the model parameters, with a learning rate set to 1 × 10⁻³.

To validate the model performance, the study employs three classic deep learning evaluation metrics for assessment, including the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Coefficient of Determination (R-squared). Their calculation formulas are as follows:

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N|y_i-{\widehat{y}}_i|}$$

(9)

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{(y_i-{\widehat{y}}_i)}^2}}$$

(10)

$$R^2=1-{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N{(y_i-{\widehat{y}}_i)}^2}}{{\displaystyle {\sum }_{i=1}^N{(y_i-\overline{y})}^2}}}$$

(11)

where $y_i$ represents the actual parking demand for parking lot $i$; ${\widehat{y}}_i$ represents the predicted parking demand for parking lot $i$; $\overline{y}$ represents the mean parking demand for all parking lots; and $n$ represents the total number of parking lots.

5.2. Results

5.2.1. Cross-Validation Results

To validate the optimal input and output results of the proposed MGCN–Transformer model, this study designed a cross-validation experiment using multiple sets of input and output steps for the 15 min interval data. The model’s predictive accuracy was verified by comparing the goodness of fit R-squared (R²) of models with different input and output steps. The predictive results of the model cross-validation experiment are shown in

Table 4. Model R² results under different input and output step lengths.

Prediction Step Length	1 Step	2 Steps	4 Steps	6 Steps	8 Steps	12 Steps
1 step	97.56%	94.45%	95.25%	93.92%	94.13%	94.27%
2 steps	97.46%	95.34%	95.44%	93.92%	94.71%	94.59%
4 steps	97.59%	94.65%	93.61%	93.72%	93.04%	94.72%
8 steps	97.61%	95.49%	93.80%	95.01%	94.80%	94.85%
16 steps	97.60%	94.34%	95.93%	93.48%	94.65%	93.95%
24 steps	97.61%	94.10%	93.89%	94.11%	94.97%	94.44%

.

The results of the cross-validation experiment with multiple input and output step lengths show the following:

(1)

The multi-step parking demand prediction model proposed by the study, MGCN–Transformer, demonstrates good predictive performance in different input and output prediction tasks. Even with short input step lengths (one step, two steps, and four steps), the model continues to show strong predictive capabilities. Additionally, the model’s accuracy is higher in single-step prediction tasks compared to multi-step prediction tasks. This can be attributed to the model’s ability to accurately capture short-term trends in the time series, leading to superior predictive performance in shorter and less complex prediction tasks;

(2)

In the prediction task, the model’s predictive accuracy gradually improves with an increase in input step length, until it reaches its peak at an input step length of 16 steps. In most combinations of prediction step lengths, the predictive performance is optimal with an input step length of eight steps. Furthermore, for single-step and multi-step prediction tasks, the most effective input step lengths are 8 steps and 16 steps, respectively, with the predictive goodness of fit reaching 97.61% and 95.93%. This demonstrates that in time series prediction tasks, the predictive model’s ability to recognize patterns within the time series is influenced by the recent data points rather than the entire time series. Therefore, increasing the input sequence step length may not necessarily provide additional effective data.

Based on the cross-validation results with multiple input and output step lengths, it can be concluded that the MGCN–Transformer model is effective in extracting spatiotemporal features and exogenous variables for time series prediction. It exhibits strong generalization capabilities in both single-step and multi-step prediction tasks. Additionally, in practical applications, it is possible to choose different input and output step lengths based on the data characteristics to achieve a better predictive performance.

To further demonstrate the predictive accuracy of the MGCN–Transformer model for parking demand at each parking lot within the region, we designed a parking demand prediction task with an eight-step input and a prediction length of 1 based on the cross-validation results. The predicted results for the No. 1 parking lot are shown in Figure 7. Additionally, considering that MAE and RMSE metrics are significantly influenced by the numerical values of historical parking transaction data, this study will utilize the goodness of fit R² to visually display the training errors for predicting parking demand at each parking lot within the region.

The goodness of fit R² results for parking demand at each parking lot are shown in Figure 8, with different colors corresponding to the magnitude of R² values. For the majority of parking lots, the proposed MGCN–Transformer model demonstrates an accurate prediction of future time steps of parking demand, with over 58% of parking lots having a goodness of fit above 0.98. However, there are still some parking lots with a goodness of fit below 0.60. This is mainly due to two factors. Firstly, the spatial relationships among parking lots play a role; in saturated areas, commuters tend to seek nearby parking lots for convenience, leading to a convergence effect in parking demand among parking lots in the region. Therefore, densely distributed parking areas exhibit better fitness in terms of parking demand, while isolated parking lots at the bottom of Figure 8 show a poor fitness of parking demand. Secondly, there is an impact from the scale effect of parking lot capacity. Parking lots with more spaces have better resistance to interference when facing continuous and smooth fluctuations in parking demand. Even in the central area of Figure 8, where parking lots are densely distributed, the impact of insufficient parking spaces hinders their ability to effectively resist fluctuations in parking demand.

5.2.2. Comparison of Baseline Models

To further explore the predictive performance of the MGCN–Transformer model, the study evaluates its effectiveness by comparing it with basic and advanced methods. Firstly, an introduction to the benchmark models for comparison is outlined as follows:

ARIMA: This classic time series analysis and forecasting model employs autoregressive, differencing, and moving average features to capture sequence trends and seasonal characteristics. Due to the limitations of its model structure, it requires using all training data as input features for model training tasks.

MC: This model is a classic parking demand forecasting model that can explain state transitions in simple time series data, analyze their stability and convergence, and effectively capture the transformation patterns of time series. However, due to limitations in its mathematical model structure, it still requires all training set data as input features.

LSTM: LSTM, as a special type of RNN, overcomes the vanishing and exploding gradient problems in the traditional RNN model through the design of its gated units. It effectively captures the temporal features of time series data and performs predictive tasks. In the comparative study of the models, a two-layer LSTM model with 128 neurons is constructed to perform single-step and multi-step prediction tasks.

MGCN-LSTM: This model integrates Graph Convolutional Neural Networks and Long Short-Term Memory networks to model spatial and temporal dependencies by capturing the spatial relationships between nodes and temporal characteristics. It processes time series data by combining the spatial distance matrix between parking lots with the time series, thereby executing prediction tasks.

Transformer: This model, as a revolutionary neural network framework, is specifically designed for handling sequence data. It efficiently performs prediction tasks by utilizing attention mechanisms to achieve sequence modeling.

To effectively evaluate the predictive performance of the proposed MGCN–Transformer model, this study optimized the parameters for all the benchmark models and conducted prediction tasks using the same dataset. Except for traditional mathematical models such as ARIMA and MC, which require the use of the entire training set data as input, the remaining models processed the time series data as training data containing the target values. Using 15 min interval time series data, this study evaluated the models’ performance through the design of single-step and multi-step prediction tasks. The experimental results are presented in

Table 5. Results of single-step and multi-step prediction comparisons for the models.

Prediction Step Length	Single-Step Prediction			Multi-Step Prediction
	R2	MAE	RMSE	R2	MAE	RMSE
ARIMA	72.88%	5.75	6.97	72.88%	5.75	6.97
MC	74.12%	5.10	6.73	74.12%	5.10	6.73
LSTM	88.03%	2.40	4.49	75.53%	3.77	6.42
MGCN-LSTM	90.33%	2.34	4.04	77.60%	3.71	6.14
Transformer	97.56%	0.06	0.10	95.02%	0.30	0.51
MGCN–Transformer	97.61%	0.05	0.08	95.93%	0.26	0.42

.

The comparative experimental results indicate the following:

(1)

The ARIMA and MC mathematical models demonstrate a consistent performance in both single-step and multi-step prediction tasks, with the performance metrics for the ARIMA model being 72.88%, 5.75, 6.97 and for the MC model, 74.12%, 5.10, 6.73. This is primarily because traditional mathematical models rely heavily on historical demand time series data for prediction tasks, while ignoring other feature labels and lacking the ability to fit non-linear relationships in time series data, resulting in a poor predictive performance;

(2)

The LSTM and MGCN-LSTM deep learning models outperform the mathematical models in both single-step and multi-step prediction tasks, with MGCN-LSTM demonstrating superior predictive accuracy compared to LSTM due to its ability to capture spatial relationships among parking lots. However, both models exhibit a poor performance in multi-step prediction tasks. This can be attributed to the operational principles of LSTM in multi-step prediction tasks, as it uses each step’s prediction result as input for the next step’s prediction, leading to accumulated prediction errors with increasing prediction steps and ultimately resulting in a significant decrease in predictive accuracy;

(3)

The MGCN–Transformer model achieves the best predictive performance in both single-step and multi-step prediction tasks. It demonstrates excellent predictive accuracy even in multi-step prediction tasks. Furthermore, compared to the Transformer model, the predictive model is expected to achieve better performance by capturing spatial feature relationships.

5.2.3. Feature Ablation Experiment

To explore the contribution of different input features to parking demand prediction tasks, this section utilizes the MGCN–Transformer prediction model to design feature ablation experiments. The input features for the study include parking demand time series, parking lot spatial distance features, POI features near parking lots, weather features, and holiday features. In addition to the parking demand time series, which serves as the evaluation metric for the experimental results, the study sequentially conducts experiments to compare the remaining input indicators. The experimental design is outlined in

Table 6. Differential ablation design.

Input Features	Parking Demand Features	Parking Lot Distance Features	POI Information Features	Weather Features	Holiday Features
Control Group (COG)	√	√	√	√	√
Experiment A	√	-	√	√	√
Experiment B	√	√	-	√	√
Experiment C	√	√	√	-	√
Experiment D	√	√	√	√	-

.

The results of the ablation experiment are shown in Figure 9. In both single-step and multi-step prediction experiments, the MGCN–Transformer model achieves optimal output when all input features are present. Each input feature contributes differently to the model’s predictive accuracy. In the single-step prediction experiment, the POI feature has the greatest impact on predictive accuracy, followed by the spatial distance feature, weather feature, and holiday feature. In the multi-step prediction experiment, the spatial distance feature has the most significant impact on predictive accuracy, followed by the weather feature, POI feature, and holiday feature. This suggests that the spatial distance feature and POI feature have the highest impact on predictive accuracy, possibly due to the commercial nature of Shenzhen’s Nanshan district, where the spatial relationships between parking lots and the nearby POIs have a more direct and significant impact on parking demand compared to the weather and holiday features.

From the experimental results, the proposed model achieves a good prediction accuracy in multi-step parking demand forecasting tasks. For urban parking systems, accurate multi-step parking demand forecasting not only provides travelers with effective parking space information and helps them plan their travel routes more efficiently, but it also enhances the utilization of urban parking spaces and improves the operational efficiency of the urban network.

6. Discussion and Conclusions

To address the issue of urban parking difficulties and effectively optimize parking space utilization, this study proposes a sophisticated deep learning approach known as MGCN–Transformer. This model integrates GCN, Transformer, and Dence network modules to simultaneously consider spatiotemporal dependency features and the impact of exogenous variables on parking demand prediction. Specifically, in terms of spatial features, multiple GCN layers are used to extract relevant information from various spatial network graphs, while for temporal features, a Seq2Seq architecture-based Transformer module is employed to capture time dependencies. Additionally, regarding exogenous variables, one-hot encoded data are embedded into the Transformer network using an embedding method to facilitate multi-step prediction tasks.

To validate the model’s predictive performance, this study conducted a validation using real historical parking transaction data from Shenzhen’s Nanshan district and compared it with several benchmark models based on three evaluation metrics: MAE, RMSE, and R². The benchmark models include traditional time series forecasting methods such as ARIMA, MC, LSTM, MGCN-LSTM, and Transformer models. Furthermore, to explore the contribution of different input features to the model’s predictive performance, the study designed ablation experiments. The case results show the following:

(1)

The proposed model achieved excellent predictive results in the cross-validation experiments with different input and output step lengths. Regarding the setting of the prediction input step length, the predictive accuracy does not increase unrestrictedly with an increase in the input step length due to the limitations of the recognition patterns of the time series prediction model. The optimal prediction input step lengths are 8 and 16, where the optimal input step length is 16 only when the output step length is 4. As for the prediction output step length, the overall predictive accuracy decreases as the prediction step length increases. This is because as the prediction step length increases, the predictions become more complex, and the model’s learning capacity weakens;

(2)

From the model comparison results, it is evident that the proposed MGCN–Transformer model outperforms the others in both single-step and multi-step prediction tasks. Unlike traditional recurrent neural networks, the model’s accuracy for multi-step prediction tasks does not significantly decrease compared to single-step prediction tasks. This is attributed to the superior model structure, which mitigates error accumulation during the prediction process;

(3)

The results of the ablation experiments indicate that incorporating spatial features and exogenous variables significantly enhances the model’s predictive performance. Furthermore, spatial distance relationships and POI features have a greater impact on parking demand prediction tasks compared to exogenous variables such as weather and holidays. Their spatial feature relationships play a crucial role in improving the accuracy of parking demand predictions.

For urban systems, accurate multi-step parking demand forecasting can not only help travelers achieve reasonable path planning, but also assist urban planners in accurately implementing parking lot planning and improving parking space utilization.