Multi-Scale Graph Attention Network Based on Encoding Decomposition for Electricity Consumption Prediction

Abstract

Accurate electricity consumption forecasting is essential for power scheduling. In short-term forecasting, electricity consumption data exhibit periodic patterns, as well as fluctuations associated with production events. Traditional forecasting methods typically focus on sequential features of the data, which may lead to an over-smoothing issue for the fluctuations. In practice, the fluctuations of electricity consumption associated with these events tend to follow recognizable patterns. By emphasizing the impact of these experiential electricity consumption fluctuations on the current prediction process, we can capture the volatility variations to alleviate the over-smoothing problem. To this end, we propose an encoding decomposition-based multi-scale graph neural network (CMNN). The CMNN starts by decomposing the electricity data into various components. For the high-order components that exhibit approximate periodic behavior, the CMNN designs a Multi-scale Bi-directional Long Short-Term Memory (MBLSTM) network for fitting and prediction. For the low-order components that exhibit fluctuations, the CMNN transforms these components from one-dimensional time series into a two-dimensional low-order component graph to model the volatility of the low-order components, and proposes a Gaussian Graph Auto-Encoder to forecast the low-order components. Finally, the CMNN combines the predicted components to produce the final electricity consumption prediction. Experiments demonstrate that the CMNN enhances the accuracy of electricity consumption predictions.

1. Introduction

Accurately predicting the electricity consumption within a region is crucial for power production and scheduling, which in turn enhance the quality of power delivery within the system. Time series electricity consumption data generally exhibit approximate periodicity and a certain degree of volatility. Many scholars have conducted lots of research on electricity consumption prediction. Traditional prediction methods include regression analysis [1], Markov models [2], support vector regression [3], time series analysis models [4,5], twin support vector regression [6], and other time series models. With the development of artificial intelligence, deep learning methods such as attention models and convolutional neural networks [6,7] have become mainstream algorithms. Additionally, some researchers have introduced methods like signal decomposition and Fourier analysis for extracting the frequency features.

However, transforming sequential data into the frequency domain may lead to a reduction in temporal information. Moreover, while signal-decomposition-based prediction methods capture the periodic nature of the data, they often struggle to effectively model the volatility associated with electricity consumption. During certain industrial production activities, electricity consumption in a region generally shows a distinct upward trend, leading to fluctuations in the data. In these instances, time series forecasting methods often produce predictions with the lowest mean squared error. This is akin to maximum likelihood estimation, which effectively captures the periodicity of the data. However, these predictions may suffer from over-smoothing, hindering the accurate modeling of electricity data volatility.

This paper addresses electricity consumption forecasting by highlighting the significance of modeling both periodicity and volatility. The volatility in electricity consumption is closely related to industrial production events. Although we cannot directly incorporate these events into the prediction model to improve the prediction accuracy, changes in electricity consumption caused by industrial activities typically follow predictable patterns. First, the changes on an industrial scale within a fixed area are relatively slow and the production patterns of enterprises are relatively stable. Therefore, the amplitude of electricity consumption fluctuations caused by production events is within a predictable range. Moreover, the intervals between these production-related fluctuations do not change significantly, and these intervals can provide a time series basis, while the amplitude of consumption change can serve as a numerical reference for predictions. By considering the influence of past fluctuations on current predictions, we can reduce prediction errors and mitigate the issue of over-smoothing. Therefore, we propose an encoding decomposition-based multi-scale graph neural network (CMNN). A CMNN first applies Ensemble Empirical Mode Decomposition (EEMD) to decompose the data into multiple Intrinsic Mode Functions (IMFs). For higher-order components with periodicity, the CMNN incorporates additional weather and date information (holidays and weekdays) and employs a Multi-scale Bi-directional Long Short-Term Memory (MBLSTM) model to predict these component values. For lower-order components that mainly exhibit volatility, the CMNN enhances the dimensionality of these components to analyze the relationships between production events and consumption fluctuations, transforming these lower-order components from one-dimensional time series into a two-dimensional low-order component graph to model the volatility of the low-order components by analyzing the temporal correlation and numerical correlation from the fluctuating nodes to the current prediction. To extract these features, the CMNN adds weather and date information to the node features of the low-order component graph and utilizes a Gaussian Graph Attention Auto-Encoder on the constructed low-order component graphs to encode these features into Gaussian distributions of expectation and covariance to forecast the low-order component value. Finally, the CMNN combines the predicted values of all components to deliver the final electricity consumption forecasts.

The main contributions of this paper can be summarized as follows:

➢

This paper breaks down the electricity consumption forecasting task into two main components: periodicity prediction and volatility modeling. This paper proposes an encoding decomposition-based multi-scale graph neural network (CMNN), which utilizes Multi-scale Bi-directional Long Short-Term Memory (MBLSTM) and a Gaussian Graph Attention Auto-Encoder to handle different components, enhancing electricity consumption predictions and mitigating the challenges of over-smoothing.

➢

For modeling volatility, the CMNN enhances the dimensionality of the lower-order components to effectively represent the temporal characteristics of volatile data and their relationships with associated production events. It establishes a low-order component graph that leverages temporal correlations and numerical similarities, transitioning the modeling of data volatility from a one-dimensional time series framework to a two-dimensional graph structure. Furthermore, it constructs a Graph Attention Auto-Encoder to capture these data features.

➢

The paper designs experiments to verify three conclusions. First, the experiments substantiate the effectiveness of the proposed CMNN for electricity consumption forecasting through comparative analysis. Second, the constructed low-order component graph is demonstrated to produce lower prediction errors than conventional one-dimensional time series models in forecasting scenarios. Finally, the experiments validate the utility of the proposed Gaussian Graph Attention Auto-Encoder in electricity consumption forecasting tasks.

The organization of this paper is as follows: Section 2 introduces related work on electricity consumption forecasting; Section 3 proposes the encoding decomposition-based Multi-scale Electricity Consumption Prediction Neural Network; Section 4 verifies the effectiveness of the algorithm through experiments; and the Section 5 presents the conclusions.

2. Background

This paper focuses on short-term daily electricity consumption forecasting tasks. More granular forecasting tasks such as hourly or minute-level electricity consumption forecasting are closely related to the specific industry, the specific monitoring area, environmental conditions, and the current social backdrop. Due to varying production scales and patterns, different industries exhibit different electricity consumption data distribution. Electricity consumption data have approximate periodicity and volatility, with weekday consumption mainly influenced by industrial and commercial electricity use. In contrast, weekend and holiday electricity consumption decreases, primarily due to reduced industrial and commercial activities. From this temporal scale, the data reveal an approximate weekly periodicity, making temporality crucial in short-term electricity consumption forecasting. Meanwhile, industrial production within a period causes fluctuations in electricity consumption data, which vary among different industries but are relatively stable in terms of fluctuation amplitude, making the modeling of volatility and its amplitude another key issue in achieving accurate electricity consumption predictions. Additionally, different industries have varying statistical characteristics, some with less obvious periodicity and some with volatility triggered by production events.

Recently, many machine learning techniques have been developed for electricity consumption forecasting. Among them, deep learning methods such as attention models and convolutional neural networks [6,7] have attracted much attention. Recurrent Neural Networks (RNNs) [8], Long Short-Term Memory networks (LSTMs) and their variants have been widely applied in effective modeling of time series data [9], demonstrating their application in short-term electricity and load forecasting accuracy [10]. Reference [11] introduced a multi-scale LSTM model aimed at capturing the temporal features of electricity consumption data. Reference [12] followed the LSTM temporal feature modeling approach. Reference [13] introduced a Bi-directional Long Short-Term Memory (Bi-LSTM) network that enhances prediction accuracy by utilizing context information. Additionally, various deep learning models have been applied to electricity consumption forecasting tasks. Attention mechanisms, as a development of time series models in deep neural networks, have also been applied to electricity consumption forecasting tasks [14]. Furthermore, some researchers have introduced signal decomposition and Fourier analysis methods, commonly using techniques like Empirical Mode Decomposition (EMD) and Ensemble Empirical Mode Decomposition (EEMD). Fourier analysis introduces frequency domain information, extracting frequency domain features from signal components for prediction [15,16]. Typical methods include Fourier Neural Networks and Fourier LSTM models. Reference [17] introduced a spectral pooling method in the frequency domain, using the inverse Fourier transform for training. However, transforming data into the frequency domain can lead to the loss or weakening of time information. Additionally, wavelet analysis methods are used for multi-scale analysis to enhance prediction accuracy [18], but constructing effective wavelet functions is challenging, and complex time data often require deeper wavelet decomposition to extract features. Moreover, graph neural networks are also used for electricity consumption prediction. Graph neural networks (GNNs) can incorporate link information of the prediction targets, thereby expanding the applicability of electricity consumption forecasting [19,20]. Moreover, some optimizer-based methods are used for electricity consumption forecasting [21,22].

3. Proposed Model

To address the aforementioned issues, this paper proposes an encoding decomposition-based multi-scale graph neural network (CMNN). After decomposing electricity data using EEMD to obtain multiple components, the CMNN fits the high-order components and residuals with Multi-scale Bi-directional Long Short-Term Memory (MBLSTM); for the lower-order components that exhibit volatility, the CMNN constructs a low-order component graph based on temporal correlation and numerical similarity. Then, the CMNN builds a Gaussian Graph Attention Auto-Encoder based on this low-order component graph, which learns temporal features and numerical correlation features and encodes them as the expectations and covariance of a Gaussian distribution to predict the low-order components. Finally, the CMNN combines the predictions of all component levels to produce the final electricity consumption forecast results. The model structure of the CMNN is shown in the Figure 1:

Next, this paper details the model specifics.

3.1. Analysis of Periodic Characteristics of Electricity Consumption Data

To capture the periodic characteristics of electricity consumption data, the CMNN employs a data decomposition approach, which breaks down the electricity data into components exhibiting various periodic features. This allows for detailed feature extraction and the analysis of components that demonstrate significant periodicity. Specifically, the CMNN utilizes the EEMD technique to decompose the electricity consumption data and extract the corresponding IMF components. The purpose of the CMNN is to model the approximate periodic characteristics of electricity consumption data across different time resolutions by employing different-order IMF components. This facilitates the capture of the approximate periodicity of electricity data across multiple temporal scales and enables predictions for the upcoming time slices of the IMF components. To achieve this, the CMNN incorporates a MBLSTM to forecast both high-order IMF components and residuals, allowing it to learn the periodic features of the electricity data effectively.

Specifically, the CMNN first decomposes the electricity consumption data using EEMD, represented by Equation (1):

$$x\left(t\right)={\displaystyle {\sum }_{j=1}^J{C}_j\left(t\right)}+r_J\left(t\right)$$

(1)

where x is electricity consumption data, J is the index of the residual component of the IMF, r_J(t) represents the residual term of the generated Intrinsic Mode Function (IMF), and C_j(t) represents the j-th order component of the IMF. Then, for high-order IMF components and residuals, incorporating additional weather and date information, the CMNN constructs a composite input vector that includes this additional information. Based on this composite input vector, the CMNN designed a MBLSTM to learn the periodic features of the high-order IMF components and residuals and predict the values of the next time slice of high-order IMF components and residuals. The structure of the MBLSTM model is shown in Figure 2:

In Figure 2, the feature extraction process marked in blue is carried out on each node with 1-scale Bi-LSTM, the feature extraction process marked in purple is carried out on the adjacent two nodes using a 2-scale Bi-LSTM, and then the extracted features are combined for prediction. The MBLSTM can be achieved simply by integrating multiple different-scale bi-directional LSTMs. After features are extracted with these bi-directional LSTMs, the extracted features are merged to form a new feature vector. The feature extraction process for the j-th order IMF component is represented as Equation (2):

$$\begin{array}{cc}{f}_{IMF\_j}\hfill & =Multi\_scale\_Bi\_LSTM\left(\left[C_j\left(t\right), add\left(t\right)\right]\right)\\ & =\left\{\begin{array}{c}Bi\_LSTM\_1\left(\left[C_j\left(t\right), add\left(t\right)\right]\right),...,\\ Bi\_LSTM\_K\left(\left[C_j\left(t\right), add\left(t\right)\right]\right)\end{array}\right\}\end{array}$$

(2)

where Multi_scale_Bi_LSTM contains several Bi_LSTMs in different scales and, among them, Bi_LSTM_K is the Bi_LSTM used with a K scale, add(t) represents the encoded weather and date information at time t, C_j(t) represents the j-th order component of the IMF, and f_IMF__j represents the feature of the j-th order component of the IMF, where j > 1. Based on the extracted features, the CMNN predicts the values of high-order components, modeling the periodic features in the high-order components. The prediction result of the high-order components is represented as a combination of the outputs from bi-directional LSTMs across multiple scales in Equation (3):

$$y_{IMF\_1}=Line\left(\begin{array}{l}Bi\_LSTM\_1\left(\left[C_j\left(t\right), add\left(t\right)\right]\right),...,\\ Bi\_LSTM\_K\left(\left[C_j\left(t\right), add\left(t\right)\right]\right)\end{array}\right)$$

(3)

where Line is a simple linear layer with ReLU as the activation function.

To model the volatility of electricity consumption data, the CMNN constructs a low-order component graph based on temporal correlation and numerical similarity for the low-order IMF components that exhibit volatility, as well as a Gaussian Graph Attention Auto-Encoder based on the low-order component graph to learn the volatility features.

3.2. Volatility Characteristics Analysis of Electricity Consumption Data

After the decomposition of the electricity consumption data using EEMD, the volatility of the data is primarily reflected in the lower-order IMF components. The volatility of these IMF components is closely related to industrial production events. The proposed CMNN aims to model these variations in electricity usage brought about by volatility, thereby improving the accuracy of electricity consumption forecasts. To avoid over-smoothing in the forecasts for the lower-order components that primarily exhibit volatility, the CMNN introduces additional correlations among electricity usage by expanding the dimensionality of the lower-order components. This allows for the modeling of the correlations between production events embedded in the volatile electricity data, based on the correlations among electricity values. Consequently, the CMNN constructs a low-order component graph based on temporal correlation and numerical similarity. On this basis, the CMNN builds a Gaussian Graph Attention Auto-Encoder to extract temporal features and numerical correlation features, encoding them as the expectations and covariances of a Gaussian distribution to parameterize the modeling of data changes and volatility. Based on these parameterized data features, the CMNN predicts the low-order components.

The CMNN converts one-dimensional temporal relationships into a two-dimensional graph structure. To preserve both the temporal information and the correlations between electricity values within this two-dimensional framework, the CMNN defines temporal Cosine similarities to assign weights to the edges of the graph during the graph construction process. This approach effectively integrates both temporal dynamics and electricity value correlation information. This information can be represented as Equation (4):

$$\begin{array}{cc}edg{e}_{i,j}& =temporal\left(i,j\right)\times Similar\left(i,j\right)\hfill \\ & ={\displaystyle \frac{1}{sequence\left(i,j\right)}}Cosine\left(f_i,f_j\right)\hfill \end{array}$$

(4)

where temporal(i, j) is the temporal feature between nodes i and j, Similar(i, j) is the correlation feature between nodes i and j, and sequence(i, j) is the duration between time slices i and j. Longer intervals weaken the impact of temporality. For the correlation features between nodes i and j, the CMNN uses Cosine similarity for measurement, where f_i is the electricity data for node i. Then, the constructed low-order component graph structure is as shown in Figure 3:

In Figure 3, the one-dimensional temporal relations between the low-order components are transferred into a two-dimensional graph, and each edge in the graph is calculated based on the Cosine similarity. Based on the low-order component graph, the CMNN designs a Gaussian Graph Attention Auto-Encoder to extract temporal features and numerical correlation features, encoding them as the expectations and covariances of a Gaussian distribution for the parameterized modeling of data changes and volatility. Specifically, the CMNN uses the constructed low-order component graph as the graph input for the Gaussian Graph Attention Auto-Encoder, and the electricity data with additional information as node features, encoding the extracted features into the form of a parameterized Gaussian distribution in Figure 4:

In Figure 4, the Gaussian Graph attention is used for encoding, and a corresponding decoder is constructed to extract attention features. During the encoding computation, the CMNN defines a temporal Cosine attention mechanism to extract features. The temporal Cosine attention mechanism includes a similarity function, attention coefficients, and the construction of attention features. The similarity function measures the associations between electricity usages at the time slices represented by the nodes in the low-order component graph. To express the different impacts caused by different similarities between inputs, the CMNN defines a temporal Cosine similarity function. This function measures the similarity between two input vectors as Equation (5):

$$Similarit{y}_{ij}=SLP\left({\displaystyle \frac{1}{sequence\left(i,j\right)}}\times Cosine\left(A\times W\times f_i,A\times W\times f_j\right)\right)$$

(5)

where A is the adjacency matrix of the low-order component graph, W refers to learnable weights, SLP() is a single-layer neural network, and sequence(i, j) is the duration between time slices i and j. Longer intervals weaken the impact of temporality. The temporal Cosine similarity function calculates the angle between vectors. Based on this function, attention coefficients are defined as Equation (6):

$${\alpha }_{ij}=\mathrm{softmax}\left(Similarit{y}_{ij}\right)={\displaystyle \frac{\exp \left(Similarit{y}_{ij}\right)}{{\displaystyle {\sum }_j\exp \left(Similarit{y}_{ij}\right)}}}$$

(6)

where α_ij are the attention coefficients for f_i and f_j. Thus, the attention feature for the current node i can be expressed as Equation (7):

$$\begin{array}{l}featur{e}_i=N\left({\displaystyle {\sum }_j{\alpha }_{ij}\times V\times h_j},{\displaystyle {\sum }_j{\alpha }_{ij}\times U\times h_j}\right)\\ ={\displaystyle {\sum }_j{\alpha }_{ij}\times V\times h_j}+\delta \times {\displaystyle {\sum }_j{\alpha }_{ij}\times U\times h_j},\\ where,\delta \sim N\left(0,1\right)\end{array}$$

(7)

where V and U are learnable weights. Based on Equations (5)–(7), the Gaussian Graph Attention Auto-Encoder calculates electricity usage features for each time slice and encodes the low-order component graph. The decoding process of the Gaussian Graph Attention Auto-Encoder can be achieved using a standard attention layer. To effectively model the numerical relationships among electricity values and provide guidance for forecasts, the Gaussian Graph Attention Auto-Encoder introduces a reconstruction loss based on the low-order component graph, and its loss function is expressed as Equation (8):

$$Loss\_ae={\displaystyle \frac{1}{N}}{\displaystyle {\sum }_n\left\{\begin{array}{l}{\left(X_n-{\mathrm{Recon}}_n\right)}^2+\\ \frac{1}{\mathrm{Edge}Num}{\displaystyle {\sum }_{i,j}[cosine\left(featur{e}_i, featur{e}_j\right)-A_{i,j}{]}_n^2}\end{array}\right\}}$$

(8)

In Equation (8), Recon is the reconstructed data, EdgeNum is the number of edges, and the second term of the loss introduces a feature consistency function, ensuring that the extracted features are consistent with the low-order component graph in terms of similarity. Based on the extracted features of the low-order component graph, the CMNN uses an output layer to predict the results for the low-order components as Equation (9):

$$y_{IMF\_1}=Line\left(feature\right)$$

(9)

where Line is a simple linear layer with ReLU as the activation function. Finally, combining the predictions of the IMF components and residuals, the final predicted value for electricity data is obtained in Equation (10):

$$y=y_{IMF\_1}+{\displaystyle {\sum }_{j=2}^J{y}_{IMF\_J}+y_{r_J}}$$

(10)

For the effective training of the CMNN, this paper constructs a loss function based on the reconstruction loss, loss_ae, and the minimum mean squared error loss for predictions, expressed as Equation (11):

$$\begin{array}{ll}Loss& =Loss\_ae+Loss\_pre\hfill \\ & ={\displaystyle \frac{1}{N}}{\displaystyle {\sum }_n\left\{\begin{array}{l}{\left(X_n-{\mathrm{Recon}}_n\right)}^2+\\ \frac{1}{\mathrm{Edge}Num}{\displaystyle {\sum }_{i,j}[cosine\left(featur{e}_i, featur{e}_j\right)-A_{i,j}{]}_n^2}+\\ MSE\left(y_n-labe{l}_n\right)\end{array}\right\}}\hfill \end{array}$$

(11)

where Loss_pre represents the prediction loss, Recon is the reconstructed data, and MSE is the Mean Square Error. The loss function, Loss, can be optimized using the gradient descent method, and this paper uses the Adam method.

4. Experiments

In this section, experiments are designed to verify three conclusions. First, the experiments confirm that the proposed algorithm yields results comparable to or even better than state-of-the-art models. Second, the designed low-order component graph is more effective than traditional one-dimensional time series structures for prediction. Finally, the experiments validate the effectiveness of the proposed Gaussian Graph Attention Auto-Encoder in the electricity consumption prediction task. For the proposed CMNN, we decompose the data using EEMD with 5 IMFs, and then we use a multi-scale Bi-LSTM with scale 1 and scale 2, which has 200 units in its hidden layer for forecasting the 2-order IMF, 3-order IMF, 4-order IMF, and 5-order IMF. For the 1-order IMF, we construct the Gaussian Graph Attention Auto-Encoder with three hidden layers with 200 units for encoding. The models are trained using Early-stop to alleviate overfitting under a 200 training epochs setting. The batch size is set to 16, gradient updates use the Adam method with an initial learning rate of 1 × 10⁻³, and dropout is applied to prevent overfitting, with a dropout rate of 0.5.

4.1. Datasets

To validate the effectiveness of the proposed CMNN, this study utilizes three datasets. First, to verify that the CMNN model can capture the periodicity and volatility of data across different scales, a simulated dataset is constructed. This dataset consists of a superposition of periodic signal sequences at multiple time scales and random noise. The signal sequence adds Gaussian-distributed random noise to sine functions of varying periodic lengths:

$$\begin{array}{l}S\left(n\right)=3\times \sin \left(4\times T\right)+4\times \cos \left(9\times T\right)+\\ \sin \left(8.12\times T+1.2\right)+1.5\times \left(n\sim Gaussian\left(0,1\right)\right)\end{array}$$

Here, n represents the sample. In this experiment, 2000 sampling points are taken, and a sliding window of size 50 is used to process the dataset, resulting in sequence data of length 50. The data value of the next sample point is predicted based on the sequence data within this window.

The second dataset is the individual household electric power consumption dataset (Data_1) (https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption, accessed on 1 November 2024) [23], and the third dataset is the electricity consumption dataset from a region served by the Southern Power Grid (Data_2) (https://www.csg.cn/). For the second dataset, electricity data from a household in Paris, France, has been collected over nearly four years (from December 2006 to November 2010) with a sampling interval of one minute.

The attribute information for the dataset is as follows:

➢

Date: Format dd/mm/yyyy;

➢

Time: Format hh:mm:ss;

➢

Global Active Power: Average active power for the household per minute;

➢

Global Reactive Power: Average reactive power for the household per minute;

➢

Voltage: Average voltage per minute;

➢

Global Intensity: Average current intensity for the household per minute;

➢

Submetering_1: Energy submeter No. 1, corresponding to the kitchen, primarily including the dishwasher, oven, and microwave (the hot plate is gas-powered);

➢

Submetering_2: Energy submeter No. 2, corresponding to the laundry room, including the washing machine, dryer, refrigerator, and lights;

➢

Submetering_3: Energy submeter No. 3, corresponding to the electric water heater and air conditioner.

The last three attributes among the nine attributes are not the total power consumption for all circuits in the household. Other electricity consumption can be calculated using the following Equation (12):

$$\begin{array}{l}remainder={\displaystyle \frac{global\_active\_power\times 1000}{60}}-\\ (sub\_metering\_1+sub\_metering\_2+sub\_metering\_3)\end{array}$$

(12)

For comparison, the data from both datasets has been normalized. Since the approximate period is seven days, we group the data into 14-day segments to capture variation patterns and provide a numerical reference for the fluctuating data. A sliding window of size 7 is utilized to facilitate sequence grouping throughout the dataset.

This paper also analyzes the electricity consumption dataset from a specific region of the Southern Power Grid, as outlined in the introduction. The dataset spans from 1 January 2020 to 9 August 2020, covering data from seven regions with recordings taken every 24 h. It includes the total electricity consumption for region (A), as well as consumption figures for the primary (A1), secondary (A2), and tertiary (A3) industries, along with nine different industrial sectors. Figure 5 shows the partial industrial electricity consumption over June in a certain area of the Southern Power Grid.

As shown in Figure 5, the regional electricity consumption displays approximate periodicity and fluctuations over time, posing challenges for electricity consumption prediction tasks. To prepare the data for training, we first address outliers by identifying and replacing them with the average values of adjacent points. Following this, we normalize the dataset for both inputs and labels. Some industries demonstrate a noticeable periodicity in their electricity consumption, while others do not exhibit such patterns clearly.

To accurately predict electricity consumption, we propose the CMNN model, and the following experiments verify the model’s effectiveness on these three datasets.

4.2. Baselines

The benchmark models used in this study consist of three categories with ten models:

➢

The first category includes standard time series forecasting models: SVR [24], LSTM [8], GA_Bi-LSTM [25], and an attention-based deep neural network (Att_NN) [26]. Among them, SVR is a traditional time series analysis method based on support vectors, GA_Bi-LSTM is a bi-directional LSTM with a genetic algorithm for optimizing parameters, and the Att_NN is an attention-based prediction method.

➢

The second category features time series forecasting models based on time–frequency analysis: FTT_LSTM (Fourier Transform) [16], DWT_LSTM (Wavelet Transform) [18], and the multi-scale LSTM model MSD_LSTM. Among them, FTT_LSTM forecasts the electricity consumption data by decomposing the data with Fourier Transform to extract the frequency features using LSTM, DWT_LSTM decomposes the electricity consumption data with Discrete Wavelet Transform to forecast the electricity consumption using LSTM, and MSD_LSTM introduces several LSTMs with different scales for forecasting the electricity consumption.

➢

The third category comprises time series forecasting models based on signal decomposition: EMD-based decomposition and optimized SVR (EMD-SVRCKH) [27], EEMD-based decomposition and LSTM (EEMD-LSTM), and EEMD–Attention (an extension of EMD and attention mechanism models) [28]. Among them, EMD-SVRCKH decomposes the data with EEMD and then predicts the electricity consumption by using a chaotic krill herd-optimized SVR.

In this paper, we use Mean Square Error (MSE) and Mean Absolute Precision (MAP) (also called Mean Absolute Error, MAE) to measure the gap between predicted electricity consumption and actual electricity consumption. MSE is more sensitive to larger errors because it squares each prediction error. This means that larger mistakes will have a greater impact on the overall error, encouraging the model to reduce significant prediction biases during training. Since prediction tasks differ from classification tasks, measuring the magnitude of the difference between predicted values and actual values is crucial for prediction tasks. MSE and MAP can quantify this difference and are widely used in much of the machine learning and statistical literature, making them well-accepted and comparable. MSE and MAP are expressed as follows:

$$\begin{array}{l}MSE={\displaystyle \frac{1}{n}}\left({\displaystyle {\sum }_{i=1}^n{\left(Output\left(i\right)-True\left(i\right)\right)}^2}\right)\\ MAP={\displaystyle \frac{1}{n}}\left({\displaystyle {\sum }_{i=1}^n\left|Output\left(i\right)-True\left(i\right)\right|}\right)\end{array}$$

(13)

In Formula (13), the Output is the predicted electricity consumption and True is the actual electricity consumption.

4.3. Validating the Effectiveness of CMNN in Prediction Tasks

First, for the simulated dataset, which exhibits multi-scale periodicity and volatility, we can verify whether the designed CMNN model can capture these characteristics. The normalized sampling sequences and EEMD components of the dataset are shown in Figure 6:

As shown in Figure 6, the “Original sequence” represents the normalized raw data, with the horizontal axis recording 101 data points sampled within the range from 0 to 2Π. It is decomposed into five IMFs, and the high-order IMF components show approximate periodicity while the low-order component shows volatility. The vertical axis represents the function values corresponding to the horizontal axis, resulting in the sequence curve. It is evident that, as the IMF order increases, the IMF components become smoother, while the first-order IMF component shows significant volatility. This experiment validates the prediction capability of the CMNN on the simulated dataset, where the MBLSTM layer count is set to 2, scales are chosen as 1 and 2, the hidden layer node count is 200, and the Gaussian Graph Attention Auto-Encoder has three attention layers. The batch size is set to 16, gradient updates use the Adam method with an initial learning rate of 1 × 10⁻³, and dropout is applied to prevent overfitting, with a dropout rate of 0.5. In comparative experiments, the skip scales for each layer of MSD-LSTM are set to 2, 4, and 6. The testing MSE of the prediction results for the CMNN on the simulated dataset is as in Figure 7:

As shown in Figure 7, the testing MSE of the signal decomposition-based forecasting models outperforms that of the three kinds of prediction models listed in the baselines, while the proposed CMNN achieves the best testing MSE. This indicates that the CMNN effectively completes prediction tasks on periodic signal sequences with multiple time scales and noise, demonstrating its ability to capture periodic and volatility features more efficiently.

In the next experiment, we verify the effectiveness of the periodic and volatility features extracted with the CMNN, using the individual household electricity consumption dataset Data_1. This electricity data can also be viewed as a combination of periodic and volatile data, with the original sequence and its decomposed components shown in Figure 8:

As shown in Figure 8, the “Original sequence” represents the normalized raw data, with the horizontal axis recording 101 data points. The high-order components show approximate periodicity, while the low-order component shows volatility.

Table 1. Test MSE of CMNN and other models proposed in Table 1 on Data_1.

	SUN	MON	TUE	WED	THR	FRI	SAT
LSTM	0.0572	0.0379	0.0366	0.0566	0.0381	0.0561	0.0713
SVR	0.0631	0.0392	0.0389	0.0574	0.0399	0.0593	0.0722
Att_NN	0.0544	0.0355	0.0358	0.0535	0.0351	0.0522	0.0709
GA_Bi-LSTM	0.0527	0.0352	0.0344	0.0513	0.0347	0.0507	0.0704
DWT_LSTM	0.0503	0.0333	0.034	0.0501	0.0341	0.0505	0.0693
FFT_LSTM	0.0502	0.0331	0.0324	0.0494	0.0338	0.0497	0.0681
MSD_LSTM	0.0493	0.0328	0.0335	0.0487	0.0333	0.0477	0.0644
EMD-SVRCKH	0.0448	0.0309	0.0279	0.0444	0.0317	0.0436	0.0623
EEMD-LSTM	0.0455	0.0311	0.0312	0.0444	0.0301	0.0437	0.0639
EEMD-Attention	0.0437	0.0314	0.0273	0.0433	0.0322	0.0439	0.0625
CMNN	0.0422	0.0291	0.0274	0.0393	0.0289	0.0431	0.0613

and

Table 2. Test MAP of CMNN and other models proposed in Table 1 on Data_1.

	SUN	MON	TUE	WED	THR	FRI	SAT
LSTM	0.222	0.181	0.180	0.218	0.175	0.219	0.247
SVR	0.249	0.192	0.183	0.223	0.179	0.221	0.252
Att_NN	0.224	0.183	0.178	0.216	0.174	0.215	0.247
GA_Bi-LSTM	0.221	0.181	0.178	0.215	0.174	0.215	0.247
DWT_LSTM	0.218	0.179	0.175	0.215	0.172	0.215	0.244
FFT_LSTM	0.217	0.177	0.173	0.212	0.173	0.212	0.241
MSD_LSTM	0.216	0.174	0.170	0.210	0.173	0.213	0.237
EMD-SVRCKH	0.207	0.170	0.171	0.206	0.171	0.207	0.234
EEMD-LSTM	0.206	0.171	0.169	0.209	0.168	0.209	0.236
EEMD-Attention	0.201	0.171	0.164	0.198	0.169	0.209	0.233
CMNN	0.197	0.167	0.162	0.194	0.164	0.204	0.227

record the prediction results for the individual household electricity consumption dataset, presenting the electricity consumption forecasts for each day of the week.

In Table 1 and Table 2, the DWT-LSTM model integrates a three-level Discrete Wavelet Transform (DWT) for preprocessing sequential data within an LSTM framework. The MSD-LSTM model is characterized by its multi-scale approach, specifically designed with sequence lengths of 2, 3, and 4. In contrast, the Bi-LSTM model processes data in a bi-directional manner. Our analysis reveals that the proposed CMNN achieves the lowest MSE and MAP compared to other forecasting models. Table 1 and Table 2 indicate that the MSE and MAP for predicting electricity consumption on Friday, Saturday, and Sunday is significantly higher than on the other weekdays. This observation can be linked to the heightened fluctuations in household electricity usage during these weekend days, corroborating our initial findings. Additionally, the CMNN model demonstrates superior performance over EEMD-based time series forecasting models, wavelet analysis methods, Fourier analysis techniques, and traditional time series forecasting approaches.

In the following experiments, we assess the overall predictive performance of the CMNN model using both the individual household electricity consumption dataset and the electricity consumption data from the Southern Grid region. We also perform predictive analyses regarding electricity consumption across different areas and industries within the Southern Grid. Next, we present the distribution of certain electricity consumption data along with the decomposed IMF components from a specific area in the Southern Grid in Figure 9.

As shown in Figure 9, the “Original sequence” represents the normalized raw data, with the horizontal axis recording 101 data points, and the high-order components show approximate periodicity, while the low-order component shows volatility.

Table 3. MSE of total electricity consumption in individual household electricity dataset.

	Data_1	Data_2
LSTM	0.0792	0.0406
SVR	0.0807	0.0413
Att_NN	0.0753	0.0409
GA_Bi-LSTM	0.0757	0.0405
DWT_LSTM	0.0788	0.0401
FFT_LSTM	0.0775	0.0407
MSD_LSTM	0.0777	0.0402
EMD-SVRCKH	0.0696	0.0393
EEMD-LSTM	0.0684	0.0391
EEMD-Attention	0.0689	0.0387
CMNN	0.0677	0.0379

presents the overall performance of the proposed CMNN model compared to other common machine learning models on these two datasets, with results given as testing MSE.

As shown in Table 3, the proposed CMNN achieves the best testing Mean Square Error (MSE) on both datasets. Next, we will discuss the performance of the CMNN on the electricity consumption dataset from a specific area of the Southern Grid in China. For this dataset, the daily electricity consumption forecasting task is our focus, and experiments are conducted for this task.

Table 4. MSE test of total electricity consumption in seven regions of China’s Southern Power Grid.

	Region 1	Region 2	Region 3	Region 4	Region 5	Region 6	Region 7
LSTM	0.0412	0.0394	0.0607	0.0316	0.0417	0.0308	0.0327
SVR	0.0425	0.0412	0.0792	0.0417	0.0511	0.0419	0.0392
Att_NN	0.0416	0.0389	0.0595	0.0312	0.0403	0.0306	0.0309
GA_Bi-LSTM	0.0411	0.0385	0.0591	0.0310	0.0409	0.0303	0.0307
DWT_LSTM	0.0399	0.0390	0.0598	0.0308	0.0401	0.0305	0.0309
FFT_LSTM	0.0407	0.0394	0.0602	0.0311	0.0418	0.0307	0.0321
MSD_LSTM	0.0403	0.0390	0.0596	0.0304	0.0401	0.0301	0.0303
EMD-SVRCKH	0.0386	0.0379	0.0529	0.0301	0.0399	0.0302	0.0305
EEMD-LSTM	0.0385	0.0382	0.0502	0.0299	0.0384	0.0303	0.0304
EEMD-Attention	0.0384	0.0386	0.0507	0.0301	0.0389	0.0301	0.0303
CMNN	0.0373	0.0373	0.0493	0.0293	0.0374	0.0292	0.0297

displays the testing MSE for this dataset and

Table 5. MAP test of total electricity consumption in seven regions of China’s Southern Power Grid.

	Region 1	Region 2	Region 3	Region 4	Region 5	Region 6	Region 7
LSTM	0.207	0.209	0.225	0.186	0.209	0.185	0.185
SVR	0.222	0.215	0.229	0.191	0.221	0.189	0.188
Att_NN	0.202	0.205	0.221	0.183	0.204	0.183	0.182
GA_Bi-LSTM	0.203	0.203	0.219	0.182	0.202	0.179	0.181
DWT_LSTM	0.199	0.203	0.221	0.183	0.206	0.182	0.181
FFT_LSTM	0.199	0.199	0.219	0.181	0.203	0.181	0.178
MSD_LSTM	0.196	0.197	0.217	0.179	0.198	0.180	0.177
EMD-SVRCKH	0.194	0.196	0.215	0.176	0.199	0.177	0.174
EEMD-LSTM	0.194	0.192	0.211	0.174	0.196	0.175	0.174
EEMD-Attention	0.193	0.194	0.212	0.172	0.194	0.174	0.172
CMNN	0.187	0.185	0.206	0.167	0.188	0.166	0.168

shows the test MAP.

From Table 4 and Table 5, it is evident that the proposed CMNN model shows an improvement compared to other time series forecasting models. The experiment demonstrates that the MSE and MAP of the proposed algorithm are lower than those of other algorithms, indicating that the CMNN model has superior data fitting and prediction capabilities compared to commonly used electricity forecasting models. Additionally, we test the prediction capabilities of the CMNN in Region 1 across three industries, focusing on the main sectors—industry, manufacturing, and residential electricity consumption. The test results are listed in

Table 6. MSE test of electricity consumption in a certain region of Southern Power Grid.

	Primary Industry	Secondary Industry	Tertiary Industry	Industry	Manufacturing Industry	Residential
LSTM	0.0363	0.0387	0.0403	0.0408	0.0411	0.0478
SVR	0.0372	0.0429	0.0429	0.0474	0.0423	0.0579
Att_NN	0.0369	0.0388	0.0409	0.0408	0.0409	0.0476
GA_Bi-LSTM	0.0364	0.0386	0.0401	0.0402	0.0405	0.0454
DWT_LSTM	0.0368	0.0385	0.0395	0.0404	0.0403	0.0451
FFT_LSTM	0.0368	0.0387	0.0392	0.0407	0.0403	0.0449
MSD_LSTM	0.0372	0.0383	0.0397	0.0393	0.0407	0.0445
EMD-SVRCKH	0.0353	0.0376	0.0384	0.0393	0.0382	0.0432
EEMD-LSTM	0.0347	0.0377	0.0383	0.0394	0.0392	0.0429
EEMD-Attention	0.0343	0.0378	0.0383	0.0391	0.0387	0.0427
CMNN	0.0345	0.0372	0.0374	0.0388	0.0378	0.0419

and

Table 7. MAP test of electricity consumption in a certain region of Southern Power Grid.

	Primary Industry	Secondary Industry	Tertiary Industry	Industry	Manufacturing Industry	Residential
LSTM	0.196	0.202	0.204	0.202	0.198	0.211
SVR	0.199	0.207	0.209	0.209	0.205	0.220
Att_NN	0.193	0.198	0.201	0.202	0.196	0.211
GA_Bi-LSTM	0.189	0.191	0.199	0.198	0.194	0.207
DWT_LSTM	0.191	0.196	0.202	0.204	0.196	0.207
FFT_LSTM	0.189	0.194	0.201	0.200	0.197	0.209
MSD_LSTM	0.185	0.191	0.197	0.198	0.194	0.207
EMD-SVRCKH	0.181	0.188	0.196	0.194	0.189	0.203
EEMD-LSTM	0.180	0.192	0.196	0.193	0.190	0.203
EEMD-Attention	0.177	0.189	0.195	0.191	0.190	0.201
CMNN	0.179	0.182	0.187	0.185	0.183	0.193

:

From Table 6 and Table 7, it can be seen that the proposed CMNN performs relatively consistently in the sub-data prediction tasks within a specific area of the Southern Grid in China. It effectively predicts approximate periodic data, electricity data with less obvious periodicity, and instantaneous changes in data. Furthermore, the CMNN outperforms other comparative algorithms on most sub-datasets.

4.4. CMNN Ablation Experiment

In the CMNN model, we extend 1D time series data to a 2D graph structure to model the correlations between fluctuating data and the temporal characteristics of electricity consumption. To validate the effectiveness of the graph structure, we design an ablation experiment, referring to the CMNN model that uses a time series structure to model the low-order IMF components as CMNN_MBiLSTM. Additionally, we introduce an Auto-Encoder loss to model the amplitude of fluctuating data. To test the effectiveness of this Auto-Encoder loss, we retain the graph structure but remove the Auto-Encoder loss in the CMNN model, labeling it CMNN_nonRescon. To validate the efficacy of the Multi-scale Bi-directional LSTM used, we create a CMNN model that replaces the Multi-scale Bi-directional LSTM with a standard bi-directional LSTM, referred to as CMNN_BiLSTM. The results of the ablation experiment are shown in Figure 10:

As shown in Figure 10, the proposed CMNN achieves lower prediction MSEs on both electricity consumption datasets, indicating that the designed low-order component graph and the reconstruction loss based on this graph are effective for electricity consumption forecasting tasks. Next, we show the test MSEs in Region 1 across three industries, focusing on the main sectors—industry, manufacturing, and residential electricity consumption. The test results are listed in Figure 11:

In Figure 11, the ablation models used are the same as those in Figure 10. The CMNN performs better than the ablation models on the test MSEs in Region 1 across three industries, focusing on the main sectors—industry, manufacturing, and residential electricity consumption. It verifies that the designed low-order component graph and the reconstruction loss based on this graph are effective for electricity consumption forecasting tasks.

5. Conclusions

In this study, we present advancements in electricity consumption forecasting through the development of our proposed model, the encoding decomposition-based multi-scale graph neural network (CMNN). Our key contributions lie in the innovative approach of extending low-order components from one-dimensional time series data to a multi-dimensional graph representation and the corresponding Gaussian Graph Attention Auto-Encoder. By constructing a low-order component graph that captures temporal correlations and numerical similarities, the CMNN provides an understanding of the underlying patterns in electricity consumption data. The use of the Gaussian Graph Attention Auto-Encoder allows for more accurate modeling of the complex relationships between variables, enhancing predictive capabilities. Moreover, our experiments demonstrate that the low-order component graph is superior to conventional one-dimensional time series structures, highlighting the necessity for innovative approaches to data representation for forecasting tasks. The effectiveness of the Gaussian Graph Attention Auto-Encoder further underscores the potential of graph-based models in addressing complex forecasting challenges. In summary, the proposed CMNN contributes to the field by providing a methodology for electricity consumption forecasting, while also offering insights into the advantages of using graph neural networks for time series analysis. These findings not only advance the current understanding of forecasting methodologies but also pave the way for future research that can build on our approach to enhance predictive accuracy in various domains.