Enhancing Short-Term Load Forecasting Accuracy in High-Volatility Regions Using LSTM-SCN Hybrid Models

Abstract

Short-Term Load Forecasting (STLF) is essential for the efficient management of power systems, as it improves forecasting accuracy while optimizing power scheduling efficiency. Despite significant recent advancements in STLF models, forecasting accuracy in high-volatility regions remains a key challenge. To address this issue, this paper introduces a hybrid load forecasting model that integrates the Long Short-Term Memory Network (LSTM) with the Stochastic Configuration Network (SCN). We first verify the Universal Approximation Property of SCN through experiments on two regression datasets. Subsequently, we reconstruct the features and input them into the LSTM for feature extraction. These extracted feature vectors are then used as inputs for SCN-based STLF. Finally, we evaluate the performance of the LSTM-SCN model against other baseline models using the Australian Electricity Load dataset. We also select five high-volatility regions in the test set to validate the LSTM-SCN model’s advantages in such scenarios. The results show that the LSTM-SCN model achieved an RMSE of 56.970, MAE of 43.033, and MAPE of 0.492% on the test set. Compared to the next best model, the LSTM-SCN model reduced errors by 6.016, 8.846, and 0.053% for RMSE, MAE, and MAPE, respectively. Additionally, the model consistently outperformed across all five high-volatility regions analyzed. These findings highlight its contribution to improved power system management, particularly in challenging high-volatility scenarios.

1. Introduction

Power load forecasting is an important area of research within the field of power systems. The field has received increasing academic attention due to its critical role in the efficient and economical operation of power systems [1]. Electrical load forecasting can be categorized into three categories based on the forecasting horizon [2]: Short-Term Load Forecasting (STLF), Medium-Term Load Forecasting (MTLF), and Long-Term Load Forecasting (LTLF). STLF, which ranges from one hour to one week, is significantly influenced by weather conditions and recent load data. These forecasts play a crucial role in real-time grid management. MTLF, typically spanning from one week to several months, is influenced by factors such as seasonal variations in electricity consumption, holidays, and working days. These forecasts are essential for maintenance scheduling and fuel reserve management. LTLF, which extends beyond one year, is influenced by factors such as demographic changes, economic growth, and energy policies. These forecasts are crucial for system planning and optimization. This paper focuses on STLF. The proposed model leverages deep learning techniques, specifically integrating the Stochastic Configuration Network (SCN), to achieve high accuracy in load forecasting results.

With the growth in global electricity demand, improving the accuracy of load forecasting has become increasingly critical [3]. Many factors affect electricity load, such as regional differences, socio-economic activities, weather, and prices [4]. Therefore, power load data are characterized by randomness, volatility, periodicity, and diversity. Extracting the intrinsic patterns of load change from historical power load data and developing an accurate forecasting method are key to successful load forecasting [5]. Various power load forecasting methods have been proposed to address specific challenges, each offering unique advantages based on different technologies, algorithms, or data types. Currently, power load forecasting methods are mainly divided into four categories [6]: statistical methods, artificial intelligence techniques, knowledge-based expert systems, and hybrid approaches.

Statistical methods require the construction of explicit data models to represent the relationship between the electric load and contributing factors. Classical statistical methods include multiple regression analysis, exponential smoothing, and stochastic time series, among others. Krstonijević [7] proposed an adaptive load forecasting method based on the Generalized Additive Model (GAM) and big data estimation techniques. Shi et al. [8] introduced a very short-term bus load forecasting model that utilizes Phase Space Reconstruction (PSR) and a Deep Belief Network (DBN). Barta et al. [9] aimed to establish a national energy consumption forecasting framework using open-access data from the European Network of Transmission System Operators for Electricity (ENTSO-E). To construct the forecast density, they employed Gradient Boosting Regression Trees (GBRTs) and conducted benchmarking based on actual load data and forecasts provided by each country. Wijaya et al. [10] extended the Generalized Additive Model (GAM) to GAM2, where a second GAM is applied to the squared residuals. Many statistical methods rely on linear models, which limit their ability to handle nonlinear relationships and complex patterns. Therefore, the nonlinear characteristics of complex electric loads cannot be precisely characterized using traditional methods [11].

Artificial intelligence methods used in power load forecasting include Artificial Neural Networks (ANNs), fuzzy logic, neuro-fuzzy systems, and Support Vector Machines (SVMs) [12,13,14]. Andriopoulos et al. [15] leveraged Convolutional Neural Networks (CNNs) to optimize neural network hyperparameters for power load forecasting. Duan et al. [16] proposed a power load forecasting model based on the Sparrow Search Algorithm (SSA), Variational Mode Decomposition (VMD), attention mechanism, and Long Short-Term Memory (LSTM). Initially, the SSA is used to optimize VMD parameters; then, LSTM is employed for load forecasting, and an attention mechanism is introduced to enhance the model. Pavlatos et al. [17] applied Bidirectional Long Short-Term Memory (BiLSTM) to power load forecasting and proposed combining bidirectional memory with advanced neural network architectures. Shi et al. [18] applied a Pooling-Based Deep Recurrent Neural Network (PDRNN) to household load forecasting and suggested that adding more hidden layers to the neural network could improve forecasting performance. Chen et al. [19] applied an improved deep residual network to power load forecasting to enhance prediction results. Although the above models show good performance on their respective datasets, they suffer from limited applicability and poor generalization ability [1].

Expert systems, a notable achievement in artificial intelligence, rely on rule-based logic to mimic the decision-making processes of domain experts. These systems are commonly used as decision support tools. Qiu et al. [20] applied expert system theory to analyze load transfer in regional power grids during transformer, busbar, and line faults, providing load transfer schemes. A study utilized a knowledge-based reasoning expert system to assist decision-makers in selecting the most appropriate load forecasting model for long-term planning in power systems [21]. Additionally, researchers have developed a rule-driven method that incorporates prior knowledge from experts on load curves, integrating this knowledge into statistical models to enhance forecasting accuracy [22]. These research outcomes demonstrate that expert systems play a significant role in power load forecasting, enhancing the credibility and precision of forecasting results by combining expert insights with statistical analysis.

Electricity load data are characterized by temporal correlation, volatility, and uncertainty, with volatility leading to significant forecasting errors in models [23]. Statistical methods and traditional machine learning approaches often fail to account for these characteristics simultaneously, resulting in insufficient load forecasting accuracy, indicating room for further improvement [24]. A single method to deal with power load data will result in low computational efficiency, high computational complexity, and high error rates. Over the years, numerous scholars have developed hybrid load forecasting models with the goal of achieving higher forecasting accuracy and lower error rates. Shen et al. [23] proposed a hybrid model based on the Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN), combined with CNN, BiLSTM, and Self-Attention (SA) techniques, along with Wavelet Domain Denoising (WDD) to enhance data smoothing, applied to household electricity load data. This model leverages the spatial feature extraction capabilities of the CNN, the bidirectional temporal sequence extraction capabilities of BiLSTM, and the ability of SA to focus on key historical time points to reduce information loss. Ma et al. [25] proposed a hybrid model based on the CNN, Improved Chaotic Particle Swarm Optimization (ICPSO), and LSTM for electric load forecasting. CNNs are used for feature extraction, ICPSO is employed to optimize LSTM parameters, and LSTM is then used for load forecasting. Goh et al. [26] proposed a forecasting model that combines the CNN and LSTM, where CNNs are used to identify and extract local features from time series data, which are then input into LSTM for STLF. Chen et al. [27] proposed a hybrid load forecasting model integrating a Residual Neural Network (ResNet) and LSTM, using a ResNet for feature extraction and LSTM for STLF. Zhou et al. [28] proposed an improved ARIMA-LSTM model that allocates weights based on the forecasting errors of the two models in the training set. Shin et al. [29] proposed a hybrid approach combining Variational Mode Decomposition (VMD) to decompose complex data into Intrinsic Mode Functions (IMFs) and an RVFL network to predict each IMF, thereby improving load forecasting accuracy and robustness.

The SCN, a stochastic learning method, was proposed by Wang et al. [30] in 2017. Common stochastic learning algorithms include Radial Basis Function (RBF) networks [31] and RVFL networks [32]. The SCN learning algorithm outperforms other methods due to its supervision mechanism [33,34,35,36,37]. However, the lack of supervision mechanisms and the challenge of setting appropriate parameter ranges can lead to poor model performance [38], as demonstrated mathematically in [39]. The SCN exhibits faster convergence and superior forecasting ability compared to other stochastic networks [31]. The SCN has achieved great success in various fields, including software implementation, computer vision, medical data analysis, fault detection and diagnosis, and the modeling and forecasting of various systems. In the realm of power load forecasting, Wang et al. [40] developed a recurrent version of the SCN for time series problems. However, when dealing with power load data, comparing it with only a single model does not sufficiently demonstrate its superiority in handling such data. Our aim is to contribute an effective and accurate hybrid load forecasting model to the field.

The proposed LSTM-SCN approach offers distinct advantages over traditional statistical methods. Conventional statistical models typically assume linearity or stationarity in the data and focus on capturing dependencies and seasonal variations in time series, making them more suitable for simpler load forecasting scenarios. In contrast, LSTM-SCN, as a hybrid model, is capable of handling complex nonlinear relationships and data heterogeneity, enabling it to deliver more accurate predictions in challenging power load forecasting tasks. Compared to existing hybrid models, the LSTM-SCN approach introduces the SCN model innovatively to enhance predictive performance.

The main contributions of this paper are as follows: (1) We propose an innovative fusion method for STLF by combining LSTM and the SCN. We extract features using LSTM, which are then fed into the SCN for forecasting. (2) We introduce the SCN as an alternative to traditional forecasting methods, leveraging its Universal Approximation Property to enhance forecasting accuracy in STLF. (3) We contribute to the literature by comparing the LSTM-SCN model with other baseline models, particularly examining its performance in high-volatility regions, and thereby highlighting its superior predictive accuracy.

The rest of the paper is organized as follows: Section 2 introduces the principles of the SCN, LSTM, and the proposed LSTM-SCN hybrid model. Section 3 validates the Universal Approximation Property of the SCN using two regression datasets. Section 4 applies the LSTM-SCN model to the Australian electricity load dataset and analyzes its performance in comparison with other baseline models. Section 5 concludes the paper.

2. Methodologies

In this section, we provide a comprehensive overview of the underlying principles of the SCN, LSTM, and LSTM-SCN models. To enhance understanding, this glossary provides definitions of symbols and abbreviations frequently used throughout the paper (

Table 1. Glossary of symbols and abbreviations.

Symbol/ Abbreviation	Definition
$w_j$	$\mathrm{The}\mathrm{input}\mathrm{weight}\mathrm{of}\mathrm{the}j$-th node in the hidden layer
$b_j$	$\mathrm{The}\mathrm{input}\mathrm{bias}\mathrm{of}\mathrm{the}j$-th node in the hidden layer
$f_{L-1}\left(X\right)$	$\mathrm{The}\mathrm{network}\mathrm{output}\mathrm{result}\mathrm{when}\mathrm{the}\mathrm{number}\mathrm{of}\mathrm{hidden}\mathrm{layers}\mathrm{is}L$− 1
${\beta }_j$	$\mathrm{The}\mathrm{output}\mathrm{weight}\mathrm{corresponding}\mathrm{to}\mathrm{the}\mathrm{hidden}\mathrm{layer}\mathrm{node}j$
$g(·)$	Activation function
${\epsilon }_{L-1}$	$\mathrm{The}\mathrm{network}\mathrm{residual}\mathrm{vector}\mathrm{when}\mathrm{the}\mathrm{number}\mathrm{of}\mathrm{hidden}\mathrm{layers}\mathrm{is}L$− 1
ε	Training tolerance
$L_{max}$	The maximum number of nodes in the hidden layer
$h_L$	$\mathrm{The}\mathrm{output}\mathrm{of}\mathrm{the}\mathrm{hidden}\mathrm{layer}\mathrm{node}\mathrm{L}$
$T_{max}$	The number of candidates in the hidden layer
SVR	Support Vector Regression
RVFL	Random Vector Functional Link Networks
FC	Fully Connected Layer
LR	Linear Regression
XGBoost	Extreme Gradient Boosting
GBRT	Gradient Boosting Regression Trees
SCN	Stochastic Configuration Network
LSTM	Long Short-Term Memory
BiLSTM	Bidirectional Long Short-Term Memory
GRU	Gated Recurrent Unit

).

2.1. Stochastic Configuration Network (SCN)

The SCN, an incremental neural network, was proposed by Wang et al. [30] in 2017. The network construction begins with a single node in the hidden layer. In a supervised manner, the input weights and biases of the hidden layer nodes are randomly initialized. The number of hidden layer nodes is then gradually increased, and the output weights are computed using least squares. The SCN consists of input, hidden, and output layers, as shown in Figure 1.

SCN’s construction process is illustrated in Figure 2. Section 2.1.1 and Section 2.1.2 detail the specific steps of the SCN construction, while Section 2.1.3 presents the Universal Approximation Theorem.

2.1.1. Related Work

Given a training set $\{X,Y\}$, where $X=\{x_1,x_2,\cdots ,x_N$} represents the input data and $Y=\{y_1,y_2,\cdots ,y_N$} represents the corresponding labels. Specifically, $x_i={[x_{i,1},x_{i,2},\cdots ,x_{i,d}]}^T\in R^d$, $y_i={[y_{i,1},y_{i,2},\cdots ,y_{i,m}]}^t\in R^m$, $i=1,2,\cdots ,N$. Given a target function $f:R^d\to R^n$, suppose that an SCN with $L_n-1$ hidden nodes has already been constructed. Set the default error to ${\epsilon }^{\prime }$ and the maximum hidden layer node to $L_{max}^{\left(n\right)}$. The output of the current network is calculated using Formula (1).

$$f_{L-1}\left(X\right)={\sum }_{j=1}^{L-1}{\beta }_j{g}_j\left({\omega }_j^{T}X+b_j\right),L=1,2,\dots ,L_{max},f_0=0$$

(1)

Let ${\beta }_j$ denote the output weight of the $j$-th node in the implicit layer, and $g\left(·\right)$ denote the activation function. Let $w_j$ and $b_j$ denote the input weights and bias, respectively, of the $j$-th node in the implicit layer.

The formula for calculating the residual vector of the current network is as follows:

$${\epsilon }_{L-1}=f-f_{L-1}\left(X\right)={\left[{\epsilon }_{L-1,1}\left(X\right),\dots ,{\epsilon }_{L-1,\mathrm{m}}\left(X\right)\right]}^T$$

(2)

2.1.2. Network Model Construction

If ${\left|\right|{\epsilon }_{\mathrm{L}-1}\left|\right|}^2>\epsilon or L<L_{max}$, then add the $L_n$ node. The input weights and biases are determined by the Universal Approximation Property according to formula Equation (3), as follows:

$$h_L={\left[g_L\left({\omega }_L^T{x}_1+b_L\right),g_L\left({\omega }_L^T{x}_2+b_L\right),\dots ,g_L\left({\omega }_L^T{x}_N+b_L\right)\right]}^T$$

(3)

$${\xi }_{L,q}={\displaystyle \frac{{〈{\epsilon }_{L-1,q}^T,h_L〉}^2}{{h_L}^T,h_L}}-\left(1-r-{\mu }_L\right){‖{\epsilon }_{L-1,q}‖}^2$$

(4)

where $q=1,2,\dots ,m;r\in \left(0,1\right)$, $h_L$ denotes the output of the $L$-th node in the implicit layer. Let $w_j$ and $b_j$ represent the candidate parameters of the $L$-th node. Given a non-negative real number sequence $\left\{{\mu }_L\right\}$with $\underset{L\to +\infty }{\lim }‖f-f_L‖=0$ and ${\mu }_L\le 1-r$. The parameter of the candidate node that satisfies the condition ${\xi }_L={\sum }_{q=1}^m{\xi }_{L,q}\ge 0$ with the maximum value is used as the parameter for the $L$ node.

The output weight is determined through a least squares evaluation, as follows:

$$\beta =arg \underset{\beta }{\min }{\left|\right|H\beta -Y\left|\right|}^2=H^{+}Y$$

(5)

where ${‖·‖}_F^2$ denotes the Frobenius norm, while $H^{+}$ represents the Moore–Penrose generalized inverse matrix of $H$.

2.1.3. Universal Approximation Theorem

Let $\mathsf{\Gamma }=\left\{g_1,g_2,g_3,\dots \right\}$ represent a set of real-valued functions, and let $span\left(\mathsf{\Gamma }\right)$ denote the function space spanned by $\mathsf{\Gamma }$. Assume that $span\left(\mathsf{\Gamma }\right)$ is dense in the $L_2$ and $\forall g\in \mathsf{\Gamma },0<‖g‖<b_g,b_g\in R^{+}$.

For $L=1,2,3,\dots$, define ${\delta }_L={\sum }_{q=1}^m{\delta }_{L,q},{\delta }_{L,q}=\left(1-r-{\mu }_L\right){‖e_{L-1,q}‖}^2$, where, $0<r<1$; ${\mu }_L\le 1-r,\underset{L\to +\mathrm{\infty }}{\lim }{\mu }_L=0$.

For the random basis function $g_L$, the following inequality constraints are satisfied:

$${〈e_{L-1,q},g_L〉}^2\ge b_g^2{\delta }_{L,q},q=1,2,\dots ,m$$

(6)

The output weights of the hidden layer nodes are as follows:

$$\beta =\left[{\beta }_1,{\beta }_2,\dots ,{\beta }_L\right]=argmin‖f-{\sum }_{j=1}^L{\beta }_j{g}_j‖$$

(7)

Then, $\underset{L\to +\infty }{\lim }‖f-f_L‖=0$.

2.2. Long Short-Term Memory (LSTM)

LSTM is a specialized type of Recurrent Neural Network (RNN) architecture that excels in time series analysis and forecasting. LSTM’s ability to learn long-term dependencies enables it to outperform traditional RNNs in capturing such dependencies within sequential data. LSTM was originally designed to address the issues of gradient vanishing and explosion in traditional RNNs when processing long sequences. By introducing a special memory unit, LSTM can retain information from previous states.

The LSTM cell incorporates three gating mechanisms: the forget gate, the input gate, and the output gate. The role of the forget gate is to determine whether information should be discarded or retained in the cell state, based on the hidden state from the previous time step and the current input. The input gate consists of a sigmoid layer and a tanh layer. The sigmoid layer decides which values to update, and the tanh layer generates a new vector of candidate values for updating the cell state. The output gate determines the value of the next hidden state. The interaction of these three gates enables LSTM to efficiently capture long-term dependencies in time series data while mitigating the effects of noise. The LSTM unit is depicted in Figure 3. The mathematical representation of the LSTM is presented below.

$$\left\{\begin{array}{c}\begin{array}{c}{f}_t=\sigma \left(W_f·\left[{h_{t-1},x}_t\right]+b_f\right)\\ i_t=\sigma \left(W_i·\left[{h_{t-1},x}_t\right]+b_i\right)\end{array}\\ \begin{array}{c}{C}_t^{\prime }=tanh\left(W_C·\left[{h_{t-1},x}_t\right]+b_C\right)\\ C_t=f_t·C_{t-1}+i_t·C_t^{\prime }\\ O_t=\sigma \left(W_O·\left[{h_{t-1},x}_t\right]+b_O\right)\end{array}\\ h_t=O_t·tanh\left(C_t\right)\end{array}\right.$$

(8)

2.3. LSTM-SCN

The LSTM-SCN model proposed in this paper consists of two main components: the LSTM, which acts as a feature extractor, and the SCN, which performs load forecasting using these features. The structure of the model is illustrated in Figure 4.

Data are preprocessed before being input into the model. Missing values, outliers, and noise in the data can affect the accuracy of the results, necessitating preprocessing operations. Additionally, data normalization can improve the performance of the LSTM model and prevent issues with vanishing or exploding gradients.

The proposed model employs an LSTM network for feature extraction. The internal architecture of the LSTM network consists of two layers [27]: the first layer contains 1024 units, and the second layer contains 256 units. To mitigate overfitting, dropout is applied between the two LSTM layers. The output from the second layer, representing the extracted features, serves as the final output of the model.

This paper dispenses with the fully connected layers commonly used in most models, opting instead for an innovative approach that employs the SCN for forecasting. In traditional models that use fully connected layers for forecasting, it is necessary to frequently set and adjust the number of hidden layers. The SCN allows for a relatively large number of hidden layer nodes, enabling the model to autonomously identify the optimal network structure within the permissible range of node configurations. This process not only consumes time but also requires significant computational resources. In contrast, the SCN structure is determined by setting a maximum number of hidden nodes and a tolerance level. The number of hidden nodes in the SCN will incrementally increase until the specified tolerance is achieved or the maximum node limit is reached. The SCN is set with the following parameters: the maximum number of hidden layers $L_{max}$= 200, weights scale sequence Lambdas = [0.5, 1, 5, 10, 30, 50, 100, 150, 200, 250], training tolerance ε = 0.001, contractive sequence r = [0.9, 0.99, 0.999, 0.9999, 0.99999, 0.999999], and maximum number of candidate nodes $T_{max}$= 100.

3. Experimental Confirmation of SCN’s Universal Approximation Property

In this section, we validate the universal approximation property of the SCN through experiments. The common predictive methods employed in existing models for power load data processing include Support Vector Regression (SVR) [41], Random Vector Functional Link Networks (RVFLs) [29], and Fully Connected Layers (FCs) [27]. In addition, we include Linear Regression (LR), Extreme Gradient Boosting (XGBoost), and Gradient Boosting Regression Tree (GBRT) models for comparison. Wang et al. [30] have demonstrated the universal approximation property of the SCN in their studies on three regression datasets: Stock, Concrete, and Compactiv. These three datasets are all from the KEEL (KEEL. Available online: http://www.keel.es/ (accessed on 22 June 2024)) database. In this study, we validate the universal approximation property of the SCN on the Stock and Concrete regression datasets, and compare the performance of various models on these datasets. Specifically, we randomly selected 80% of the samples in each dataset as the training set, and the remaining 20% as the test set.

3.1. Evaluation Metric

The performance of the proposed model is evaluated using three commonly utilized metrics: the Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). These metrics are widely recognized for assessing the accuracy of forecasting models. In the fourth part of the experiment, these metrics are also applied to further evaluate the model’s performance. The RMSE quantifies the magnitude of discrepancies between predicted and actual values, the MAE calculates the mean of absolute differences, and the MAPE measures percentage errors. The formulas for these metrics are as follows:

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2$$

(9)

$$\mathrm{M}\mathrm{A}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{I=1}^N{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(10)

$$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|{\displaystyle \frac{y_i-{\widehat{y}}_i}{y_i}}\right|$$

(11)

3.2. Parameter Settings

In this section, we use the sigmoid activation function for the SCN, RVFL, and FC models. The parameter settings for each model are as follows [30]: for the FC model, we set the number of iterations to 200 and specify 200 nodes in the hidden layer. For the RVFL model, we also set the number of iterations to 200 and use 200 hidden nodes. For SVR, we determine the optimal parameter combinations through a grid search method, and then apply these optimal parameters to perform regression analysis. For GBRTs, we set the number of trees to 200.

3.3. Experimental Results

After conducting 50 independent experiments on each model, we obtained the average performance of these models on the two datasets, as shown in

Table 2. Evaluation indicator values for each model.

Dataset	Model	RMSE	MAE	MAPE
Stock	SVR	0.641	0.486	1.042%
	RVFL	32.291	30.521	65.303%
	FC	1.645	1.645	2.759%
	LR	2.335	1.795	3.857%
	XGBoost	0.783	0.585	1.256%
	GBRT	0.900	0.685	1.461%
	SCN(our)	0.540	0.540	0.876%
Concrete	SVR	9.610	6.245	22.595%
	RVFL	10.793	8.345	30.080%
	FC	6.612	4.925	16.943%
	LR	2.615	2.032	4.411%
	XGBoost	0.893	0.655	1.412%
	GBRT	0.963	0.742	1.599%
	SCN(our)	1.698	0.194	0.568%

. The performance of the SVR model is based on the optimal parameters obtained by the grid search method.

Table 2 presents the average performance of various models across two datasets, obtained after 50 independent experiments, with the performance values for the SVR model derived from the optimal parameters obtained through the grid search method. The experimental results demonstrate that on the Stock dataset, the SCN model achieves the best performance in both RMSE and MAPE metrics, being 0.101 and 0.166% lower, respectively, than those of the second-best model. In terms of the MAE metric, the SCN ranks second, with a value only 0.054 higher than that of the top-performing model. On the Concrete dataset, the SCN achieves the best performance in terms of the MAE and MAPE metrics. In conclusion, the superior forecasting performance of the SCN demonstrates the rationality and innovativeness of the LSTM-SCN model proposed in this paper.

To visually highlight the advantages of the SCN in forecasting, we present the prediction plots, performance indicator boxplots, and loss function plots for the SCN and other models on the Stock dataset. Figure 5 compares the predicted and actual values of the SCN with those of the second-best performing model, SVR, in the test set. It is evident that the SCN forecasts the data more accurately than SVR.

Figure 6 presents the box plots of the RMSE, MAE, and MAPE results for each model over 50 experiments, as shown in Figure 6a–c. Due to poor performance, the RVFL results are not displayed. The plots show that the SCN consistently achieves the lowest values across all metrics, indicating superior predictive accuracy and strong stability with an approximately symmetric distribution of its values.

Among all the models, the SCN, FC, and RVFL require iterative training to achieve optimal performance, and these three models are commonly used in hybrid electric load forecasting models. Therefore, we compared the training errors of the SCN with those of FC and RVFL. Figure 7 illustrates the error plots for the SCN, FC, and RVFL on the training and testing sets. The SCN model optimizes its network structure by progressively increasing the number of hidden layer nodes, where each addition of a hidden node corresponds to a training iteration. Therefore, for the SCN, the x-axis represents the number of hidden layer nodes. The experimental results indicate that the SCN and FC models exhibit roughly comparable convergence speeds during training, but the SCN achieves a lower loss function value at the end of training. This suggests that the SCN is more effective in finding the optimal network structure. In contrast, while the RVFL model has a simpler training process, it converges more slowly and ends with a higher loss function value than the SCN and FC models.

4. Experiments

This section utilizes the Queensland electricity load dataset (Australia Load. Available online: https://github.com/weiran4/AustraliaData (accessed on 22 June 2024)) to evaluate the performance of the proposed model. The effectiveness of the model in predicting the electricity load is also compared against other established baseline models.

4.1. Experimental Setup

The experiments presented in this paper were conducted on the Ubuntu 22.04 LTS platform within an experimental environment that included Python 3.10 and PyTorch 2.1.0. The precise specifications of the experimental setup are detailed in

Table 3. Experimental environment configuration.

Experimental Environment	Experimental Setup
OS	Ubuntu 22.04
Development Environment	VS code
Experimental Setup	Intel(R)Xeon® Silver 4116 CPU @ 2.10 GHz
Graphics Card Model	NVIDIA GeForce RTX 3090, RTX(24GB)
Programming Language	Python3.10
Deep Learning Framework	Pytorch

.

4.2. Dataset

The dataset contains electrical load data from Queensland, Australia, spanning from 1 January, 2006, to 31 December, 2010. Sampled at 30 min intervals, the dataset provides 48 sampling points per day and contains 87,649 entries. The dataset includes tariff, temperature, humidity, and power load information, as detailed in

Table 4. Dataset characteristics.

Feature Name	Unit	Description
Power Load	MW	The rate of electrical energy consumption of electrical devices
Dry Bulb Temperature	°C	The air temperature measured by a conventional thermometer
Dew Point Temperature	°C	The temperature at which water vapor in the air condenses into dew
Wet Bulb Temperature	°C	Wet bulb temperature indicates humidity and cooling potential
Humidity	%	The amount of water vapor in the air
Price	$/MWh	The cost per unit of electricity consumed by the user per hour

. In this section, data from 1 January, 2006, to 30 June, 2009, are used as the training set; data from 1 July, 2009, to 31 December, 2009, are used as the validation set; and data from the year 2010 are used as the test set.

4.3. Data Preprocessing

We preprocessed the dataset to ensure the accuracy and reliability of subsequent analyses. This involved cleaning the data to remove inconsistencies or errors, extracting time series features to capture temporal patterns, and normalizing the data to standardize the scale of variables.

4.3.1. Data Cleaning

In the data cleaning section, we inspected the data to confirm that there were no missing values in the dataset. Additionally, we identified potential outliers by calculating the interquartile range (IQR) and applying the 1.5 × IQR rule. For detected outliers, we replaced them with the mean of the preceding and succeeding data points. This approach reduces the impact of outliers on the overall data distribution while maintaining data continuity and stability.

4.3.2. Time Series Feature Construction

It is recognized that power load data exhibit significant time series characteristics and interdependence between neighboring data points. Therefore, in addition to selecting environmental factors such as weather and humidity as features, we employ a sliding window method to construct features from the power load data. Specifically, we define a sliding window of 48 time units [28] that moves step by step along the time series. Consequently, the feature vector at each time point includes data from the previous 48 time units, capturing the dynamic changes in the historical data at that point. This process is illustrated in Figure 8. For instance, the feature vector for the nth data point is ${\alpha }_n=\left[{Load}_{n-48},{Load}_{n-47},\dots ,{Load}_{n-1}\right]$. Using the sliding window method to construct features enhances the model’s ability to capture temporal dependencies and reveals potential patterns and trends in the data. This approach provides richer and more accurate feature information for electricity load forecasting. The feature vectors for the first 48 data points are incomplete due to missing values within the sliding window. Therefore, we remove the first 48 records from the dataset after feature construction to prevent any potential impact of these missing values on subsequent data analysis and model training. The 48 load features we constructed, combined with factors such as weather and humidity, result in a total of 55 features for each electricity load value.

4.3.3. Data Normalization

For the power load data, we normalize both the training and test sets before inputting them into the model. We use the min–max normalization method to scale the data to the [0, 1] range with the following formula:

$$X_{norm}={\displaystyle \frac{X-X_{min}}{X_{max}-X_{min}}}$$

(12)

The normalized value $X_{norm}$ is calculated from the original value $X$, where $X$ is the original value, $X_{max}$ is the maximum value, and $X_{min}$ is the minimum value in the dataset.

In this section, we back-normalize the model forecasts to revert the data to their original scale. All model performance metrics are calculated on the back-normalized data, and the visualization of results is also based on these data. Performing the inverse normalization provides a more intuitive and clearer view of the model’s forecasting performance. The inverse normalization formula is as follows:

$$X=X_{norm}\times \left(X_{max}-X_{min}\right)+X_{min}$$

(13)

4.4. Experimental Results

We compared the proposed LSTM-SCN model with other baseline models, and

Table 5. Performance metrics for each model on the forecasting set.

Model	RMSE	MAE	MAPE
LSTM [42]	93.797	72.175	0.819%
BiLSTM [17]	124.220	100.635	1.179%
GRU [43]	69.620	51.879	0.593%
SCN	97.626	74.071	0.847%
CNN-LSTM [5]	62.986	57.829	0.545%
LSTM-RVFL [29]	187.695	167.114	1.866%
LSTM-SCN (our)	56.970	43.033	0.492%

presents the performance metrics of each model on the forecasting set. The LSTM-SCN model demonstrated the best performance across all metrics, with an RMSE of 56.970, an MAE of 43.033, and a MAPE of 0.492%. The RMSE and MAPE values of the LSTM-SCN model are reduced by 6.016 and 0.053%, respectively, compared to the second-best performing CNN-LSTM model. Additionally, the MAE of the LSTM-SCN model is decreased by 8.846 compared to the second-best performing GRU model. These results demonstrate the superior performance of the LSTM-SCN model compared to the others.

In the field of electricity load forecasting, while most models achieve good forecasting performance in low-volatility regions, their true performance is often revealed in high-volatility regions. To fully evaluate LSTM-SCN’s performance, we selected five high-volatility periods from the last month of the test set, each with 40 data points. Figure 9 and

Table 6. Data ranges for selected high-volatility regions.

Region Name	Region 1	Region 2	Region 3	Region 4	Region 5
Data Range	[200, 240)	[540, 580)	[875, 915)	[1165, 1205)	[1310, 1350)

detail the selection of these regions and their data ranges, respectively. This detailed analysis enables us to more accurately assess the model’s applicability and accuracy in real-world scenarios with high volatility.

Table 7. Comparative performance indicators of models across different regions.

Region Name	Model	RMSE	MAE	MAPE
Region 1	LSTM	69.503	56.357	0.723%
	BiLSTM	43.366	34.632	0.453%
	GRU	49.503	40.009	0.511%
	SCN	82.894	69.422	0.894%
	CNN-LSTM	48.532	36.188	0.454%
	LSTM-RVFL	120.833	111.067	1.391%
	LSTM-SCN (our)	5.465	4.851	0.062%
Region 2	LSTM	77.431	55.698	0.666%
	BiLSTM	63.474	46.654	0.566%
	GRU	47.208	36.597	0.451%
	SCN	58.148	47.428	0.591%
	CNN-LSTM	52.001	41.473	0.517%
	LSTM-RVFL	158.725	149.760	1.853%
	LSTM-SCN (our)	7.285	5.095	0.062%
Region 3	LSTM	66.255	48.467	0.632%
	BiLSTM	48.305	36.094	0.477%
	GRU	53.502	43.131	0.573%
	SCN	86.465	81.428	0.794%
	CNN-LSTM	53.183	41.249	0.542%
	LSTM-RVFL	154.871	146.592	1.947%
	LSTM-SCN (our)	6.104	6.104	0.057%
Region 4	LSTM	109.402	93.140	1.262%
	BiLSTM	56.048	37.506	0.506%
	GRU	71.450	54.467	0.733%
	SCN	93.136	82.856	1.492%
	CNN-LSTM	42.403	30.666	0.412%
	LSTM-RVFL	156.803	148.207	2.012%
	LSTM-SCN (our)	7.452	5.932	0.080%
Region 5	LSTM	95.331	84.480	1.166%
	BiLSTM	41.727	30.050	0.406%
	GRU	60.292	46.030	0.625%
	SCN	79.458	69.475	0.916%
	CNN-LSTM	58.118	45.542	0.617%
	LSTM-RVFL	112.343	104.097	1.418%
	LSTM-SCN (our)	7.175	5.976	0.081%

displays the performance metrics for each model in the five selected high-volatility regions. The analysis results indicate that the LSTM-SCN model demonstrates optimal performance across all five high-volatility regions. In Region 1, we compare the LSTM-SCN model to the second-best performing model. The LSTM-SCN model achieves reductions of 37.901 in RMSE, 29.781 in MAE, and 0.391% in MAPE, respectively. In Regions 2, 3, 4, and 5, the reductions in these three metrics for the LSTM-SCN model are as follows: 39.923 in RMSE, 31.502 in MAE, and 0.389% in MAPE for Region 2; 42.201 in RMSE, 29.990 in MAE, and 0.420% in MAPE for Region 3; 34.951 in RMSE, 24.734 in MAE, and 0.332% in MAPE for Region 4; and 34.552 in RMSE, 24.074 in MAE, and 0.325% in MAPE for Region 5. These results demonstrate that the LSTM-SCN model exhibits excellent performance in handling high-volatility regions, thereby highlighting the advantages of the LSTM-SCN model in the field of power load forecasting.

To explain why the LSTM-SCN model exhibits optimal performance across the five high-volatility regions, we offer the following analysis:

(1)

Adaptive Dynamic Network Structure: The LSTM-SCN model relies on a supervision mechanism during prediction, whereas other models depend on the backpropagation mechanism of fully connected layers. Backpropagation requires gradient optimization and relies on a fixed network structure, while the supervision mechanism can dynamically adjust the network structure based on data and task requirements. This adaptability allows the model to construct an optimal network architecture during learning, avoiding overfitting and unnecessary complexity.

(2)

Robustness to Gradient Issues: Backpropagation in deep networks often faces challenges like vanishing or exploding gradients, which hinder model convergence and learning efficiency. In contrast, the supervision mechanism does not rely entirely on gradients but adjusts through a feedback-driven process, making the model more robust to these issues.

(3)

Enhanced Interpretability: The supervision mechanism offers higher interpretability. Its dynamic structural adjustments can reveal the relationship between data features and model behavior, providing a more transparent and efficient learning process.

Figure 10 illustrates the forecasting performance of each model in Region 1. In the forecast plots for Region 1, the LSTM-RVFL model exhibits a significant deficiency in forecasting performance compared to other models. The data in this region are highly volatile, which results in most models being unable to accurately predict the changing trend of the power load at the turning points. However, the LSTM-SCN model effectively captures the changes in power load, with its predicted curve closely matching the actual curve.

Figure 11 depicts the forecasting performance of each model in Region 2. Although the data volatility in Region 2 is lower than in Region 1, the forecast curves of all models except the LSTM-SCN deviate at the data turning points, indicating a deficiency in capturing the data trend. The forecasting curve of the LSTM-SCN model is essentially consistent with the actual curve, demonstrating its superior predictive capability.

Figure 12, Figure 13 and Figure 14 illustrate the forecasting performance of each model in regions 3, 4, and 5, each of which represents distinct high-volatility characteristics. In these regions, the LSTM-SCN model consistently demonstrates superior predictive performance, with its predicted curve closely aligned with the actual curve and effectively capturing the underlying data trends. This result corroborates findings from the first two regions, providing further evidence of the LSTM-SCN model’s robustness and dependability in accurately forecasting the power load in high-volatility regions. The model’s ability to reliably track fluctuations across diverse regions reinforces its applicability and effectiveness in handling complex, volatile power load scenarios.

5. Conclusions

With the growing demand for higher accuracy in STLF in power systems, particularly in high-volatility regions, this study introduces a hybrid forecasting model based on the LSTM-SCN. Abandoning the traditional fully connected layer forecasting method, this study innovatively adopts the SCN, known for its universal approximation property, for the forecasting process. This approach not only reduces the time and effort required to set up the network structure by eliminating the need for repeated attempts but also enables the identification of the optimal network structure among various configurations. The LSTM-SCN model utilizes the LSTM to extract data features, which are then used as inputs to the SCN for forecasting. Before commencing the experiments, we validated the Universal Approximation Property of the SCN on two regression datasets. In this study, we selected the Australian Electricity Load dataset for the experiments and used three metrics—the RMSE, MAE, and MAPE—to evaluate the model’s performance. The RMSE, MAE, and MAPE values of the LSTM-SCN model on the Australian dataset are 56.970, 43.033, and 0.492%, respectively, outperforming other models. To evaluate the predictive performance of the LSTM-SCN model in high-volatility regions, five high-volatility areas were selected from the test set for analysis. The LSTM-SCN model achieved the best performance metrics across all selected regions. In Region 1, the RMSE, MAE, and MAPE values for the LSTM-SCN model were 5.465, 4.851, and 0.062%, respectively. Compared to the second-best model, the LSTM-SCN model achieved reductions in errors of 37.901, 29.781, and 0.391% for the RMSE, MAE, and MAPE, respectively. Similarly, in Region 2, the LSTM-SCN model obtained RMSE, MAE, and MAPE values of 7.285, 5.095, and 0.062%, respectively, with reductions in errors of 39.923, 31.502, and 0.389% compared to the second-best model. For the remaining three regions, the LSTM-SCN model consistently outperformed all other models across the three evaluation metrics.

In summary, the proposed LSTM-SCN hybrid model demonstrates strong performance in short-term power load forecasting, particularly in capturing data variation trends in high-volatility regions, thereby improving forecasting accuracy in these challenging areas. Additionally, the LSTM-SCN model offers a novel solution for STLF, with significant practical application value and promising development potential. This model has the potential to drive new breakthroughs in the field of power load forecasting.