Attention-Based Multiple Graph Convolutional Recurrent Network for Traffic Forecasting
Abstract
:1. Introduction
- A novel AMGCRN model is proposed to capture the dynamic spatiotemporal correlations of the traffic flow by simultaneously learning global and local spatial features.
- The AMGCRN models both global and local spatial features through the use of attention mechanisms on node embeddings and input features, respectively.
- The proposed method can adaptively assign different learning weights to the neighbors of nodes, without requiring prior knowledge of the road structure, thereby improving the accuracy of traffic flow forecasting.
2. Related Works
2.1. Traditional Methods
2.2. Graph Based Methods
3. Preliminaries
3.1. Traffic Network
3.2. Traffic Flow Forecasting
4. Proposed Method
4.1. Global Graph Generation
4.2. Local Enhancement Graph Generation
4.3. Graph Fusion Mechanism
4.4. Spatio-Temporal Fusion Mechanism
5. Experiments
5.1. Datasets
5.2. Baselines
- HA: Historical Average is based on integrating and averaging historical information to predict future information.
- VAR [7]: Vector Auto-Regression predicts interconnected time series by capturing hidden relationships in time series.
- SVR [31]: Support Vector Regression is a supervised learning model with a related learning algorithm, which is used to analyze data used for classification and regression analysis.
- LSTM [32]: Long Short Term Memory networks are a special recurrent neural network, which solves the problem of long-term dependence in time series prediction by introducing forgetting gates.
- TCN [22]: Temporal Convolutional Neural Network achieves the effect of capturing long-term dependent information through causal convolution.
- DCRNN [5]: Diffusion Convolutional Recurrent Neural Network is a sequence-to-sequence structure that models traffic flow as a diffusion process on a directed graph, capable of capturing both spatial and temporal correlations.
- STGCN [4]: Spatio-Temporal Graph Convolutional Network combines graph convolution and gated time convolution, which can extract the most useful spatial features and continuously capture the most basic time features.
- ASTGCN [13]: Attention-based Spatio-Temporal Graph Convolutional Network designs a spatial attention mechanism and a temporal attention mechanism to simultaneously extract spatiotemporal correlation.
- STSGCN [17]: Spatial-Temporal Synchronous Graph Convolutional Network uses multiple local spatiotemporal feature extraction modules to capture the heterogeneity of long-term spatiotemporal network data.
- AGCRN [27]: Adaptive Graph Convolutional Recurrent Network proposes a method for adaptively constructing spatial correlation and adopting spatiotemporal embedding method for traffic prediction.
5.3. Metrics
5.4. Experiment Settings
- Data Preprocessing: The collected traffic data from the sensors is processed by being windowed and aggregated into 5-min intervals. For the purpose of predicting the traffic flow of the next hour based on the input data of the previous hour, the dimensions of each input and label data instance are , where N represents the number of nodes.
- Loss Function: In order to train the model, we adopt L1 loss function and apply Adam optimizer to optimize its convergence. The batch size is set as 32 for PeMSD4 dataset and 64 for PeMSD8 dataset, respectively. The learning rate is set to 0.003. The model is trained for 200 epochs on PeMSD4 and 300 epochs on PeMSD8.
- Hardware Support: The proposed model is implemented on a combination of one GeForce RTX 2080 Ti GPU and one Intel(R) Xeon(R) CPU E5-2678 v3 @ 2.50GHz.
5.5. Results
6. Ablation Study
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Lv, Z.; Xu, J.; Zheng, K.; Yin, H.; Zhao, P.; Zhou, X. Lc-rnn: A deep learning model for traffic speed prediction. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 13–19 July 2018. [Google Scholar]
- Fu, R.; Zhang, Z.; Li, L. Using LSTM and GRU neural network methods for traffic flow prediction. In Proceedings of the 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), Wuhan, China, 11–13 November 2016; pp. 324–328. [Google Scholar]
- Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv 2014, arXiv:1412.3555. [Google Scholar]
- Yu, B.; Yin, H.; Zhu, Z. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv 2017, arXiv:1709.04875. [Google Scholar]
- Li, Y.; Yu, R.; Shahabi, C.; Liu, Y. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv 2017, arXiv:1707.01926. [Google Scholar]
- Williams, B.M.; Hoel, L.A. Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results. J. Transp. Eng. 2003, 129, 664–672. [Google Scholar] [CrossRef] [Green Version]
- Zivot, E.; Wang, J. Vector autoregressive models for multivariate time series. In Modeling Financial Time Series with S-PLUS®; Springer: New York, NY, USA, 2006; pp. 385–429. [Google Scholar]
- Lippi, M.; Bertini, M.; Frasconi, P. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning. IEEE Trans. Intell. Transp. Syst. 2013, 14, 871–882. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Chen, Y.y.; Lv, Y.; Li, Z.; Wang, F.Y. Long short-term memory model for traffic congestion prediction with online open data. In Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, 1–4 November 2016; pp. 132–137. [Google Scholar]
- Ma, X.; Dai, Z.; He, Z.; Ma, J.; Wang, Y.; Wang, Y. Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors 2017, 17, 818. [Google Scholar] [CrossRef] [Green Version]
- Pan, Z.; Liang, Y.; Wang, W.; Yu, Y.; Zheng, Y.; Zhang, J. Urban traffic prediction from spatiotemporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 1720–1730. [Google Scholar]
- Guo, S.; Lin, Y.; Feng, N.; Song, C.; Wan, H. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; pp. 922–929. [Google Scholar]
- Liu, C.; Xiao, Z.; Wang, D.; Wang, L.; Jiang, H.; Chen, H.; Yu, J. Exploiting Spatiotemporal Correlations of Arrive-Stay-Leave Behaviors for Private Car Flow Prediction. IEEE Trans. Netw. Sci. Eng. 2021, 9, 834–847. [Google Scholar] [CrossRef]
- Li, M.; Zhu, Z. Spatial-temporal fusion graph neural networks for traffic flow forecasting. Proc. AAAI Conf. Artif. Intell. 2021, 35, 4189–4196. [Google Scholar] [CrossRef]
- Chen, W.; Chen, L.; Xie, Y.; Cao, W.; Gao, Y.; Feng, X. Multi-range attentive bicomponent graph convolutional network for traffic forecasting. Proc. AAAI Conf. Artif. Intell. 2020, 34, 3529–3536. [Google Scholar] [CrossRef]
- Song, C.; Lin, Y.; Guo, S.; Wan, H. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. Proc. AAAI Conf. Artif. Intell. 2020, 34, 914–921. [Google Scholar] [CrossRef]
- Guo, S.; Lin, Y.; Wan, H.; Li, X.; Cong, G. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Trans. Knowl. Data Eng. 2021, 34, 5415–5428. [Google Scholar] [CrossRef]
- Zheng, C.; Fan, X.; Wang, C.; Qi, J. Gman: A graph multi-attention network for traffic prediction. Proc. AAAI Conf. Artif. Intell. 2020, 34, 1234–1241. [Google Scholar] [CrossRef]
- Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv 2014, arXiv:1406.1078. [Google Scholar]
- Cui, Z.; Ke, R.; Pu, Z.; Wang, Y. Stacked bidirectional and unidirectional LSTM recurrent neural network for forecasting network-wide traffic state with missing values. Transp. Res. Part C Emerg. Technol. 2020, 118, 102674. [Google Scholar] [CrossRef]
- Lea, C.; Flynn, M.D.; Vidal, R.; Reiter, A.; Hager, G.D. Temporal convolutional networks for action segmentation and detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 156–165. [Google Scholar]
- Yao, H.; Wu, F.; Ke, J.; Tang, X.; Jia, Y.; Lu, S.; Gong, P.; Ye, J.; Li, Z. Deep multi-view spatial-temporal network for taxi demand prediction. Proc. AAAI Conf. Artif. Intell. 2018, 32. [Google Scholar] [CrossRef]
- Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral networks and locally connected networks on graphs. arXiv 2013, arXiv:1312.6203. [Google Scholar]
- Wang, X.; Bo, D.; Shi, C.; Fan, S.; Ye, Y.; Philip, S.Y. A survey on heterogeneous graph embedding: Methods, techniques, applications and sources. IEEE Trans. Big Data 2022, 1. [Google Scholar] [CrossRef]
- Han, L.; Du, B.; Sun, L.; Fu, Y.; Lv, Y.; Xiong, H. Dynamic and multi-faceted spatiotemporal deep learning for traffic speed forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Online, 14–18 August 2021; pp. 547–555. [Google Scholar]
- Bai, L.; Yao, L.; Li, C.; Wang, X.; Wang, C. Adaptive graph convolutional recurrent network for traffic forecasting. Adv. Neural Inf. Process. Syst. 2020, 33, 17804–17815. [Google Scholar]
- Yu, H.; Li, T.; Yu, W.; Li, J.; Huang, Y.; Wang, L.; Liu, A. Regularized graph structure learning with semantic knowledge for multi-variates time-series forecasting. arXiv 2022, arXiv:2210.06126. [Google Scholar]
- Liu, L.; Chen, J.; Wu, H.; Zhen, J.; Li, G.; Lin, L. Physical-virtual collaboration modeling for intra-and inter-station metro ridership prediction. IEEE Trans. Intell. Transp. Syst. 2020, 23, 3377–3391. [Google Scholar] [CrossRef]
- Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; Bengio, Y. Graph attention networks. arXiv 2017, arXiv:1710.10903. [Google Scholar]
- Drucker, H.; Burges, C.J.; Kaufman, L.; Smola, A.; Vapnik, V. Support vector regression machines. Adv. Neural Inf. Process. Syst. 1996, 9. [Google Scholar]
- Graves, A. Long short-term memory. In Supervised Sequence Labelling with Recurrent Neural Networks; Springer: Berlin/Heidelberg, Germany, 2012; pp. 37–45. [Google Scholar]
Datasets | Nodes | Edges | Time Steps | Time Range |
---|---|---|---|---|
PeMSD4 | 307 | 340 | 16992 | 01/01/2018–02/28/2018 |
PeMSD8 | 170 | 295 | 17856 | 07/01/2016–08/31/2016 |
Model | Dataset | PeMSD4 | PeMSD8 | ||||
---|---|---|---|---|---|---|---|
Metrics | MAE | RMSE | MAPE | MAE | RMSE | MAPE | |
HA | 38.03 | 59.24 | 27.88% | 34.86 | 52.04 | 24.07% | |
VAR | 24.54 | 38.61 | 17.24% | 19.19 | 29.81 | 13.10% | |
SVR | 28.70 | 44.56 | 19.20% | 23.25 | 36.16 | 14.64% | |
LSTM | 26.77 | 40.65 | 18.23% | 23.09 | 35.17 | 14.99% | |
TCN | 23.22 | 37.26 | 15.59% | 22.72 | 35.79 | 14.03% | |
DCRNN | 21.22 | 33.44 | 14.17% | 16.82 | 26.36 | 10.92% | |
STGCN | 21.16 | 34.89 | 13.83% | 17.50 | 27.09 | 11.29% | |
ASTGCN | 22.93 | 35.22 | 16.56% | 18.25 | 28.06 | 11.64% | |
STSGCN | 21.19 | 33.65 | 13.90% | 17.13 | 26.86 | 10.96% | |
AGCRN | 19.83 | 32.26 | 12.97% | 15.95 | 25.22 | 10.09% | |
AMGCRN (ours) | 19.52 | 31.76 | 12.90% | 15.85 | 25.32 | 10.18% |
Model | Dataset | PeMSD4 | PeMSD8 | ||||
---|---|---|---|---|---|---|---|
Metrics | MAE | RMSE | MAPE | MAE | RMSE | MAPE | |
w/Global | 20.20 | 32.63 | 13.30% | 18.55 | 29.57 | 12.60% | |
w/Local | 19.83 | 32.26 | 12.97% | 15.95 | 25.22 | 10.09% | |
AMGCRN (ours) | 19.52 | 31.76 | 12.90% | 15.85 | 25.32 | 10.18% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, L.; Cao, Y.; Dong, Y. Attention-Based Multiple Graph Convolutional Recurrent Network for Traffic Forecasting. Sustainability 2023, 15, 4697. https://doi.org/10.3390/su15064697
Liu L, Cao Y, Dong Y. Attention-Based Multiple Graph Convolutional Recurrent Network for Traffic Forecasting. Sustainability. 2023; 15(6):4697. https://doi.org/10.3390/su15064697
Chicago/Turabian StyleLiu, Lu, Yibo Cao, and Yuhan Dong. 2023. "Attention-Based Multiple Graph Convolutional Recurrent Network for Traffic Forecasting" Sustainability 15, no. 6: 4697. https://doi.org/10.3390/su15064697
APA StyleLiu, L., Cao, Y., & Dong, Y. (2023). Attention-Based Multiple Graph Convolutional Recurrent Network for Traffic Forecasting. Sustainability, 15(6), 4697. https://doi.org/10.3390/su15064697