Solar Irradiation Forecasting Using Ensemble Voting Based on Machine Learning Algorithms

Abstract

This paper proposes an ensemble voting model for solar radiation forecasting based on machine learning algorithms. Several ensemble models are assessed using a simple average and a weighted average, combining the following algorithms: random forest, extreme gradient boosting, categorical boosting, and adaptive boosting. A clustering algorithm is used to group data according to the weather, and feature selection is applied to choose the most-related inputs and their past observation values. Prediction performance is evaluated by several metrics using a real-world Brazilian database, considering different prediction time horizons of up to 12 h ahead. Numerical results show the weighted average voting approach based on random forest and categorical boosting has superior performance, with an average reduction of 6% for MAE, 3% for RMSE, 16% for MAPE, and 1% for R² when predicting one hour in advance, outperforming individual machine learning algorithms and other ensemble models.

1. Introduction

Nowadays, solar energy is receiving significant attention as one of the main renewable energy sources, with great potential for contributing to the reduction of fossil fuel consumption and CO₂ emissions [1]. As reported by [2], solar energy achieved an increase of 137 GW (+19%) in 2021, reaching a total of 854 GW of capacity and accounting for 28% of the renewable generation portfolio.

Solar power generation depends on the amount of available solar irradiation. It is an intermittent resource of energy, sensitive to random and uncontrollable weather changes. It leads to many challenges in the appropriate integration of solar power generation into the power grid, especially under high penetration levels. Solar forecasting errors can cause transmission congestion, forced solar generation curtailment, or activation of an expensive set of generators, implying extra costs [3]. Accurate solar irradiation forecasting is therefore an essential task in guaranteeing the reliable and safe operation of power systems. Additionally, it helps to maintain power quality, avoid unexpected expenditure of expensive energy resources, reduces the need for large backup energy storage, and increases the penetration of solar-powered systems [4].

In general, forecasting methods can be classified into physical, statistical, and artificial intelligence (AI) methods [5]. In physical methods, forecasting is carried out based on numerical weather prediction (NWP), cloud observations by satellite, or total sky imagers (TSIs) and using physical data such as temperature, pressure, humidity, and cloudiness. In contrast, statistical methods are based on historical series of meteorological data, which is simpler than physical methods. Autoregressive moving average (ARMA), autoregressive integrated moving average (ARIMA), and exponential smoothing (ES) are examples of statistical methods used [6].

With the gradual emergence of AI, machine learning (ML) methods have become one of the most popular approaches for solar irradiation forecasting, presenting promising results. ML is a branch of AI that uses algorithms to automatically learn insights and recognize patterns from big datasets [7]. The most-used algorithms are neural networks (NN), long short-term memory (LSTM), support vector regression (SVR), random forest (RF), k-nearest neighbors (kNN), and decision trees (DT).

Several studies can be found in the literature using ML algorithms to forecast solar irradiation [8]. The authors in [9] compare three different ML algorithms to predict hourly solar irradiance: SVR, nonlinear autoregressive (NAR), and NN. They applied k-means clustering to classify data according to the weather and highlighted that the SVR model performed better. In [10], the authors examined the potential of different ML models to forecast hourly and daily solar radiation. The models used were NN, a recurrent neural network (RNN), gated recurrent units (GRUs), LSTM, and SVR. GRUs presented a slightly superior performance compared to the other models. The authors in [11] employed different ML algorithms to predict the hourly solar irradiance: NN, SVR, fuzzy inference system (FIS), and adaptive neuro-fuzzy inference system (ANFIS). The algorithms’ performance was verified using only past time-series values of solar radiation as input and meteorological data as explanatory variables. Results showed the algorithms performed better when using meteorological data as input. The authors in [12] proposed a hybrid deep learning model for hourly solar irradiance forecasting, combining wavelet packet decomposition (WPD), a convolutional neural network (CNN), LSTM, and multilayer perceptron (MLP). Results showed the proposed hybrid model has better prediction accuracy than the other methods tested. The authors in [13] presented a combination of auto-encoder (AE) and LSTM for long-term solar radiation forecasting. The results showed the proposed hybrid method has superior performance compared with state-of-the-art models such as LSTM, GRU, and RF.

More recently, ensemble methods have increasingly been used to improve algorithms’ prediction performance [14]. The ensemble methods combine multiple algorithms into one to make an enhanced predictor. The ensemble can be homogeneous when using the same type of base learning algorithms or heterogeneous when combining different types of algorithms. Different techniques can be used to combine the base algorithms, such as bagging, boosting, stacking, or voting.

Several ensemble approaches have been proposed to forecast solar irradiation. The authors in [15] proposed a multistep-ahead solar radiation forecasting model based on the light gradient boosting machine (LightGBM), which is a boosting homogeneous ensemble learning technique. They compared the results with several tree-based ensembles and deep learning methods, and the proposed model achieved better performance. The authors in [16] proposed a heterogeneous ensemble model based on stacking for day-ahead solar power forecasting, combining the following ML algorithms: RF, extreme gradient boosting (XGBT), adaptive boosting (AdaBoost) and extra trees regressor (ETR). The authors in [17] proposed a stacking heterogeneous ensemble model combining XGBT and deep neural networks (DNNs) to forecast hourly solar irradiance. The input dataset included meteorological parameters and clear-sky index. In [18], the authors investigated the performance of homogeneous ensemble models based on bagging and boosting for solar radiation forecasting, such as boosted trees, bagged trees, RF, and generalized random forest. In [19], the authors proposed a homogeneous ensemble model using RF to forecast solar generation. Cluster analysis was first applied, and predictions were weighted by ridge regression to obtain the final prediction. In [20], the authors proposed a hybrid model for solar generation prediction that combined a statistical method with ML models. The ML models included LSTM, GRU, AE LSTM, and AE GRU. Several ensembles were explored with simple averaging and weighted averaging using linear, non-linear, and inverse approaches.

Table 1. Literature review of recent papers using ML algorithms for solar forecasting.

Reference	Year	Forecasting Variable	Feature Selection	ML Algorithms	Cluster Analysis	Ensemble	Multi-Step Ahead Forecast
[9]	2016	Hourly solar irradiance	-	SVR, NAR and NN	✓	-	-
[10]	2019	Hourly and daily solar radiation	-	NN, RNN, LSTM, GRU and SVR	-	-	-
[11]	2018	Hourly solar radiation	-	NN, SVR, FIS and ANFIS	-	-	✓
[12]	2021	Hourly solar irradiance	-	hybrid WPD, CNN, LSTM, and MLP	-	-	-
[13]	2020	PV system energy	-	hybrid AE and LSTM	-	-	✓
[15]	2020	Hourly global solar radiation	✓	LightGBM	-	Bagging and boosting	✓
[16]	2022	PV generation	-	RF, XGBT, AdaBoost and ETR	-	Bagging, boosting, and stacking	-
[17]	2021	Hourly solar irradiance	✓	Ensemble XGBT and DNN	-	Bagging and stacking	-
[18]	2020	Global horizontal irradiance	-	Boosted trees, bagged trees, RF, and generalized RF	-	Bagging and boosting	-
[19]	2019	Hourly solar generation	-	Ensemble of RF	✓	Bagging	-
[20]	2019	Solar power generation	-	LSTM, GRU, AE LSTM, AE GRU, and Theta model	-	Voting	-
This paper	2023	Hourly global solar irradiation	✓	AdaBoost, RF, XGBT, CatBoost, and voting average	✓	Bagging, boosting, and voting	✓

summarizes the main differences between previous works and the proposed work in different aspects, including the forecasting variable, algorithms used, type of ensembles tested, cluster analysis, feature selection, and forecasting horizon.

In all references mentioned, the results show ensemble models offer superior prediction performance compared to individual regressor models. Although several papers have investigated ensemble methods based on ML algorithms to forecast solar irradiation, to the best of the authors’ knowledge, only one study has been conducted combining algorithms through ensemble voting [20]. However, this paper does not explore cluster analysis and feature selection. Furthermore, our literature review shows that some ML algorithms with great performance, such as categorical boosting (CatBoost), have not yet been applied to solar energy forecasting.

This paper attempts to address this knowledge gap in the literature by proposing an ensemble voting method based on several ML algorithms to forecast solar irradiation in a city in Brazil using historic meteorological data. A feature selection was applied to choose the most important inputs and their delay values, and a clustering algorithm was used to group data with similar weather patterns. The ensemble voting was constructed using the following algorithms: random forest, extreme gradient boosting, categorical boosting, and adaptive boosting. First, the performance of each ML algorithm was assessed with several commonly used metrics: MAE, MAPE, RMSE, and R². Then, the ensemble voting model was built, combining ML algorithms using two approaches: the simple average and the weighted average. Several ensembles were tested, combining all the ML algorithms and later discarding the algorithm with the lowest performance until the two algorithms that individually produced the best results remained. The prediction accuracy was evaluated for different forecast horizons, from 1 h up to 12 h ahead, and a well-known dataset for time-series forecasting was used to validate the results. Moreover, the Diebold–Mariano statistical test was applied to compare the proposed ensemble voting algorithm against the other methods and check whether accuracy differences between the models were statistically significant.

The key contributions of this paper can be summarized as follows:

• Propose an ensemble voting combining random forest, extreme gradient boosting, categorical boosting, and adaptive boosting, which had never before been implemented for solar irradiation forecasting;
• Apply a clustering algorithm to group data with similar weather patterns;
• Propose an ensemble feature selection method to select the most significant input variables and their delay values;
• Evaluate the performance of algorithms for different forecasting horizons.

This work is organized as follows: Section 2 presents the basic theory of the ML algorithms used, Section 3 presents the proposed methodology and the database, and Section 4 shows the results, followed by our main conclusions in Section 5.

2. Machine Learning Algorithms

ML is a field of AI that builds algorithms for automated data analysis, providing more comprehensive insights into data. ML helps to handle large amounts of data efficiently and effectively, and even acts based on the information, which increases its demand.

ML algorithms use historical data as input to find and learn patterns to predict new output values. The input historical dataset is divided into training and testing datasets. The training dataset has an output variable that needs to be predicted or classified. ML algorithms infer patterns from the training dataset and apply them to the test dataset for prediction or classification. The workflow of supervised ML algorithms is shown in Figure 1. The algorithms used in this paper to compose the voting ensemble are: RF, XGBT, CatBoost, and AdaBoost.

2.1. Random Forest (RF)

RF is a commonly used ML algorithm trademarked by Leo Breiman in [21], which combines the output of multiple DTs to reach a single result. RF is a set of individual trees in which each tree predictor is trained using a different random subset of the training set, sampled using bagging or pasting methods. The prediction of a regression tree is simply given by the mean target value of the training data reaching the leaf node, and the prediction of an RF regressor is obtained by averaging the predictions of the individual regression trees. Substantial improvements in classification and regression have been obtained.

2.2. Extreme Gradient Boosting (XGBT)

XGBT is a model that was first proposed by Tianqi Chen and Carlos Guestrin in 2011 and has been continuously optimized and improved in follow-up studies by many scientists [22]. The model is a learning framework based on boosting tree models. XGBT is a decision tree-based ensemble ML algorithm, meaning it is a predictor built out of many small predictors. It builds a sequential series of weak learners, in which each learner tries to complement the others and correct for the residuals in the predictions made by all previous learners. It handles missing data and employs regularization to prevent overfitting for individual predictors. It uses parallel processing, considerably improving the training time and making it an advanced algorithm.

2.3. Categorical Boosting (CatBoost)

CatBoost is a gradient-boosting framework developed by Prokhorenkova et al. [23] in 2017 and uses a binary decision tree as base predictor. CatBoost has two main differences compared with other boosting algorithms. It uses the concept of ordered boosting, which is a random permutation approach, to train the model with a subset of data while calculating residuals with another subset, thus preventing overfitting. Furthermore, the same splitting criterion is used at all nodes, always creating symmetric trees. These trees are balanced and less prone to overfitting, which significantly speeds up the model execution. CatBoost is, however, sensitive to hyperparameter tuning.

2.4. Adaptive Boosting (AdaBoost)

AdaBoost is the first boosting meta-learning algorithm, proposed by Freud and Schapire in [24]. It is based on the idea that a better model can be created by combining multiple “weak” models added sequentially, meaning the mistakes of earlier models are learned by their successors. Each model is trained using the same data set, with different weights assigned to each one based on its accuracy. Many researchers enhance the algorithm to obtain better performance, lower computation cost, and higher speed. However, AdaBoost is an algorithm with a convex loss function and is sensitive to noise and outliers in the data, and thus prone to overfitting.

2.5. Ensemble Voting

Ensemble voting can combine algorithms using a simple average or a weighted average [25]. In the simple averaging approach, the final forecasted irradiation is obtained by taking the mean value of the forecast results from individual ML models, as shown in (1):

$$\widehat{y}={\displaystyle \sum }_{j=1}^m\widehat{y_j}/m$$

(1)

where $m$ is the number of ML algorithms used in the ensemble, $\widehat{y_j}$ is the predicted value of the j-algorithm, and $\widehat{y}$ is the final prediction value of solar irradiation.

In the weighted average approach, the final forecasted irradiation is obtained based on the weighted arithmetic mean, assigning different weights to the ML algorithms based on their accuracy. In this case, weights are assigned using integers starting at 1 depending on the individual performance of the ML algorithm. Then, the forecasted irradiation is evaluated as shown in (2):

$$\widehat{y}=\frac{{{\displaystyle \sum }}_{j=1}^m\left(w_j\widehat{y_j}\right)}{{{\displaystyle \sum }}_{j=1}^m{w}_j}$$

(2)

where $w_j$ is the weight of the j-algorithm.

3. Proposed Methodology

This section describes the proposed methodology for solar irradiation forecasting. First, data is pre-processed with cleaning, normalization, correlation analysis, and splitting data into training, validation, and test sets. Then, a clustering technique is applied to group data with similar weather patterns. Next, feature selection is performed to choose the most important inputs and delay values. After that, ensemble learning methods are applied to forecast solar irradiation. Finally, results are analyzed using several performance metrics. Each step of the proposed approach is explained in detail next. The flowchart of the proposed method is presented in Figure 2.

3.1. Data Description

Simulations were conducted using real-world data collected from the Brazilian National Institute of Meteorology (INMET) [26]. INMET’s weather stations are equipped with devices to measure temperature (thermometer), wind (anemometer), rain (pluviometer), atmospheric pressure (barometer), and solar irradiation (pyranometer), installed in locations considered strategic and in areas of interest for each Brazilian state. The database used is from the city of Salvador, located on the Brazilian coast (12°58′28.9992″ S, 38°28′35.9940″ W) at 8 m altitude, as shown in Figure 3. It has a tropical climate characterized by high temperatures ranging from 22 °C to 31 °C, high humidity, and rainfall all year round. The period covered by the database is from 1 January 2015 to 23 August 2022, in sampling intervals of 1 h. Since global solar irradiation is measured while having sunlight on the sensor, only daytime samples were considered, from 7:00 a.m. to 5:00 p.m.

Table 2. Available Database.

Data	Abbrev.	Unity	Mean
Hour, day, month, year	Hr, D, M, Y	-	-
Global solar irradiation	R	MJ/m2	1.65
Maximum wind gust	Wg	m/s	5.66
Wind speed	Ws	m/s	1.60
Wind direction	Wd	°	130.80
Dry-bulb temperature	T	°C	27.36
Hourly maximum temperature	Tmax	°C	28.14
Hourly minimum temperature	Tmin	°C	26.50
Dew point temperature	Td	°C	21.53
Hourly maximum dew point temperature	Tdmax	°C	22.28
Hourly minimum dew point temperature	Tdmin	°C	20.85
Total precipitation	P	mm	0.20
Station atmospheric pressure	A	mb	1009.40
Hourly maximum atmospheric pressure	Amax	mb	1009.71
Hourly minimum atmospheric pressure	Amin	mb	1009.21
Relative humidity	H	%	71.32
Hourly maximum relative humidity	Hmax	%	75.05
Hourly minimum relative humidity	Hmin	%	68.11

shows all variables in the database.

Figure 4 depicts the time series of solar irradiation values over the period covered in this study. The graph indicates the strong seasonal pattern that can be found in solar irradiation data. For this reason, the clustering technique was applied to divide data according to seasonality, using solar and meteorological parameters.

3.2. Pre-Processing

Missing values and outliers were replaced using linear interpolation, and data normalization was applied using the min–max normalization method to scale the data into [0, 1]. The dataset was divided into training, validation, and test sets, with a split of 70%–10%–20%, respectively. The training set was used to construct the forecasting model, and the test set was used to evaluate the performance of the model. The validation set was used to evaluate the model while tuning the hyperparameters. Next, Pearson’s correlation analysis was performed [28]. Figure 5 shows the correlation matrix between input variables. There is a high linear correlation between variables and their minimum and maximum values. Therefore, the following variables were removed: maximum and minimum atmospheric pressure, maximum and minimum temperature, maximum and minimum dew point temperature, and maximum and minimum relative humidity.

3.3. Clustering

Clustering techniques consist of analyzing data to group similar samples into clusters. In this work, the k-means clustering algorithm was used to group data with similar weather patterns and capture seasonality [28]. Three indices were used to determine the number of clusters: Calinski–Harabasz, silhouette, and Davies–Bouldin. Figure 6 shows the results. The number of clusters k varied from 2 to 10. For Calinski–Harabasz and silhouette, the best number of clusters is the one with the highest value, which is achieved with three clusters. For Davies–Bouldin, the best clustering number is the one with the smallest value, which is also achieved with three clusters. The three metrics are coincident in the optimum number k = 3. Therefore, in this paper the dataset was divided into three clusters. Cluster 1 corresponds to 31.09% of the data, cluster 2 corresponds to 30.05%, and cluster 3 to 38.86%.

The average value of daily solar irradiation in Cluster 1 is 1.34 MJ/m², in Cluster 2 it is 1.46 MJ/m², and in Cluster 3 it is 2.03 MJ/m². As can be seen, Cluster 3 presents higher levels of solar irradiation, meaning it holds data with sunnier days, in contrast to Cluster 1, which presents lower levels of solar irradiation. Figure 7 shows the percentage of days per month in each cluster. Data was grouped into clusters based on meteorological conditions of each day, and some months have data belonging to two or three clusters. Thus, data are grouped differently compared to the traditional division that follows the seasons of the year.

3.4. Feature Selection

Feature selection is a key step to improve the prediction performance of ML algorithms, reducing data size and model complexity. In this paper, three algorithms were used to select the most important inputs and their delay values: RF [21], mutual information (MI), and relief [29,30]. Each algorithm evaluates and assigns an important value to each variable. These values are normalized, and the final variable importance ranking is achieved by calculating the average value of the three algorithms. The threshold between the selected and discarded variables is empirically found using the training and validation sets. Figure 8 shows the feature importance ranking for each cluster. The importance of several lags was tested for each variable, from 1 to 72 (X_{t − 1} … X_{t − 72}). The final data set with the selected variables and their lags are shown in

Table 3. Selected Set of Input Features for ML Algorithms.

Variable	Cluster 1	Cluster 2	Cluster 3
R	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 47, t − 48, t − 72
T	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72
H	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72	t − 1, t − 2, t − 23, t − 24, t − 25, t − 48, t − 49, t − 72
Ws	t − 1, t − 2, t − 24, t − 25	t − 1, t − 2, t − 24, t − 48	t − 1, t − 2, t − 24, t − 25
Wg	t − 1, t − 24	t − 1	t − 1, t − 2
Wd	t − 1, t − 2,	t − 1	t − 1, t − 2
Td	t − 1	-	-
A	t − 1, t − 2, t − 24	-	-

.

3.5. Hyperparameter Optimization

In ML, it is important to find the optimal hyperparameter values for a given algorithm, so that its performance is maximized. In this paper, the hyperparameters were selected using the GridSearchCV technique from the scikit-learn library, which combines grid search and cross-validation [31]. GridSearchCV tries all the combinations of hyperparameter values pre-defined by the user and evaluates the model for each combination using the cross-validation method. Cross-validation is a resampling method that uses different portions of the data to test and train the model. In this paper, K-fold cross-validation was adopted with k = 5. Thus, the set of hyperparameters that provided the highest accuracy was considered the best. The hyperparameters of the forecasting models are presented in

Table 4. Selected Hyperparameters for ML Algorithms.

Algorithm	Hyperparameter	Cluster 1	Cluster 2	Cluster 3
RF	Max_depth: depth of the tree	12	11	14
	n_estimators: number of trees	400	500	600
XGBT	learning_rate: weighting factor for learning	0.1	0.1	0.1
	max_depth: depth of the tree	4	6	5
	n_estimators: number of trees	80	80	80
	subsample: subsample ratio of the training set	0.9	0.9	0.6
CatBoost	depth: depth of the tree	6	6	8
	L2_reg: coefficient at the L2 regularization term of the cost function	4	4	2
	learning_rate: used to reduce the gradient step	0.05	0.05	0.05
	Iterations: maximum number of trees that can be built	2000	2000	2000
AdaBoost	learning_rate: weight applied to each regressor at each boosting iteration	0.1	0.2	0.2
	n_estimators: number of trees	50	30	30

.

4. Performance Metrics

The performance of algorithms is analyzed using several metrics, such as the mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and the coefficient of determination (R²). These metrics are evaluated as shown in Equations (3)–(6), where, $F_i$ is the forecasted value, $O_i$ is the observed value, $\overline{O}$ is the mean value of observations, and $n$ is the number of samples. The lower the number of errors, the better the prediction, and an R² equal to 1 indicates the model perfectly performs on unseen data.

Mean absolute error (MAE):

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|F_i-O_i\right|$$

(3)

Mean absolute percentage error (MAPE_%):

$${\mathrm{MAPE}}_{\%}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|\frac{F_i-O_i}{O_i}\right|\times 100$$

(4)

Root mean square error (RMSE):

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(F_i-O_i\right)}^2}$$

(5)

Coefficient of determination (R²):

$$\begin{array}{c}{\mathrm{R}}^2=1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(F_i-O_i\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(F_i-\overline{O}\right)}^2},\\ \overline{O}=\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}{F}_i\end{array}$$

(6)

5. Results and Discussion

In this section, the forecasting accuracy of each ML algorithm was first evaluated using the test dataset. Then, different ensemble models were constructed, combining the ML algorithms using the voting average approach and the weighted average approach. Finally, the ensemble models were tested for different prediction horizons.

5.1. Machine Learning Algorithms

Table 5. Forecasting Accuracy of ML Models.

	MAE	RMSE	MAPE (%)	R2
Cluster 1
CatBoost	0.299	0.426	35.505	0.798
RF	0.306	0.427	35.963	0.797
XGBT	0.308	0.430	38.393	0.794
AdaBoost	0.364	0.476	58.716	0.748
Cluster 2
CatBoost	0.241	0.352	25.773	0.852
RF	0.248	0.356	26.457	0.848
XGBT	0.249	0.361	27.068	0.844
AdaBoost	0.304	0.403	47.741	0.806
Cluster 3
CatBoost	0.235	0.359	17.571	0.887
RF	0.245	0.372	18.195	0.878
XGBT	0.243	0.363	18.373	0.884
AdaBoost	0.312	0.425	30.492	0.841

shows the performance metrics for each algorithm and cluster. The best results are highlighted in bold. Results show that CatBoost outperformed the other ML algorithms in all metrics and all clusters. AdaBoost presented the poorest performance in all metrics to all clusters.

Figure 9 shows the histogram of absolute forecasting errors obtained with the best and the worst models, CatBoost and AdaBoost. In CatBoost, the error distribution peak is centered around zero more sharply in all clusters, indicating small errors in most predictions. For the AdaBoost model, error distribution is clearly more dispersed, indicating larger prediction errors.

5.2. Voting Ensemble

This paper tested three approaches of voting average (VOA) and one approach of voting weighted average (VOWA). In the simple average voting approach, the first ensemble, VOA1, was built combining all the ML algorithms investigated. The second ensemble, VOA2, combined the ML algorithms investigated and discarded AdaBoost, which was the algorithm with the lowest performance. The third ensemble, VOA3, combined the ML algorithms investigated and discarded the two worst-performing algorithms, retaining CatBoost and RF. The weighted average approach, VOWA, combined the same two algorithms that produced the best results, which were CatBoost and RF. In summary, the following ensemble methods were explored:

• VOA1: simple average of CatBoost + RF + XGBT + AdaBoost
• VOA2: simple average of CatBoost + RF + XGBT
• VOA3: simple average of CatBoost + RF
• VOWA: weighted average of CatBoost + RF

It is important to evaluate the computational performance of an algorithm when dealing with real-world applications.

Table 6. Forecasting Accuracy of Ensemble Models.

	MAE	RMSE	MAPE (%)	R2	Learning Time (s)
Cluster 1
VOA 1	0.309	0.427	40.099	0.797	134.40
VOA 2	0.297	0.423	34.457	0.801	125.96
VOA 3	0.297	0.422	34.598	0.802	123.96
VOWA	0.296	0.422	34.394	0.801	49.43
Cluster 2
VOA 1	0.250	0.355	29.889	0.849	57.20
VOA 2	0.239	0.351	25.336	0.852	53.43
VOA 3	0.240	0.350	25.402	0.854	55.95
VOWA	0.239	0.350	25.269	0.854	58.16
Cluster 3
VOA 1	0.246	0.365	19.977	0.883	130.60
VOA 2	0.235	0.359	17.570	0.887	127.54
VOA 3	0.234	0.358	17.425	0.886	120.34
VOWA	0.233	0.358	17.314	0.888	105.37

shows the mean value of 10 runs of performance metrics and learning speed in seconds for each ensemble algorithm and cluster. All experiments were performed on a computer with an Intel i5-1035G1 CPU (1.19 GHz) and 8.0 GByte RAM.

The comparison of the developed models illustrates that VOWA outperformed the other ensembles, with lower error metrics except for R² in cluster 1. It also presented better results than the ML algorithms individually. VOWA’s average learning time was 69.4 s, which is acceptable for planning purposes.

Based on Table 6, all the forecasting ensemble models had a high correlation coefficient, close to 1, with an average of 0.85. The learning time for the VOWA model in cluster 3 was larger than in the other clusters, because cluster 3 had more samples from the dataset.

Figure 10 shows the observed and forecasted hourly solar irradiation in addition to the residuals obtained using the VOWA ensemble model. The results show that the forecast model can follow variations in solar irradiation. Since data are grouped into clusters based on the meteorological conditions of each day, they lose continuity. For this reason, the figure presents the last 30 days of the test set of each cluster.

Weights used in VOWA are shown in

Table 7. Weights of Ensemble Models.

VOWA	CatBoost	RF
Cluster 1	2	1
Cluster 2	2	1
Cluster 3	2	1

; the model with the best performance had two votes and the one with the lowest had one vote.

The forecasting errors obtained in Cluster 3 were lower compared to the errors obtained in the other clusters, because Cluster 3 had data with higher levels of solar irradiation: sunnier days with lower irradiance variability. Cluster 1 had data with lower levels of solar irradiation, referred to as cloudy days with higher irradiance variability. Although its absolute error curve had a higher error peak than the other clusters, at around 2 MJ/m², the error metrics presented in Table 6 indicate this cluster achieved lower error. Since there were no previous studies utilizing the voting regressor method, it is not possible to make comparisons with previous studies.

5.3. Statistical Analysis

In this section, the proposed voting ensemble method is compared against the other ML algorithms, using the widely used Diebold–Mariano (DM) statistical test [32]. This test analyzes whether any difference in accuracy between two forecasting models is statistically significant.

Define the forecast errors from two competing algorithms as:

$$e_{jt}={\widehat{y}}_{jt}-y_t, t=1\dots n$$

(7)

where ${\widehat{y}}_{jt}$ is the forecasted value of the $j$-algorithm ($j=1, 2$), $y_t$ is the observed value, and $n$ is the number of samples.

The loss function of the forecast error $g\left(e_{jt}\right)$ is usually taken as the squared error or the absolute error.

The DM test is based on the loss differential $d_t$ between the two competing forecasts, defined as:

$$d_t=g\left(e_{1t}\right)-g\left(e_{2t}\right)$$

(8)

The two forecasts have equal accuracy if, and only if, the loss differential has zero expectation for all $t$. The null hypothesis ($H_0$) states that the two forecasts have equivalent accuracy ($H_0: E\left(d_t\right)=0 \forall t$). The alternative hypothesis ($H_a$) states that the two forecasts have different levels of accuracy ($H_a: E\left(d_t\right) \ne 0 \forall t$). In the DM test, a significance level of $p=0.05$ is established. Then, the decision whether to reject the null hypothesis or not is based on the resulting $p$-value. If the p-value is greater than 0.05, it will fail to reject the null hypothesis, and the differences observed between the performance of the two forecasting models are not significant. Otherwise, if the p-value is less than 0.05, the null hypothesis will be rejected, and the differences observed between the performance of the two forecasting models are significant.

Table 8. Diebold-Mariano test.

p-Value
	Cluster 1	Cluster 2	Cluster 3
CatBoost-VOWA	4.33 × 10−5	0.006	0.003
RF-VOWA	0.003	0.001	1.98 × 10−4
XGBT-VOWA	3.45 × 10−4	3.33 × 10−6	0.008
AdaBoost-VOWA	1.14 × 10−36	3.80 × 10−27	3.31 × 10−25

shows the results of the DM test, comparing the performance of the proposed VOWA voting model with the other forecasting algorithms two by two. The $p$-values are less than the threshold value of 0.05 in all cases, thereby allowing the null hypothesis $H_0$ to be rejected. This indicates that the observed differences are significant, and the proposed VOWA model is significantly more accurate than the other models.

5.4. Different Forecast Horizons

The proposed VOWA ensemble model was evaluated for different forecast horizons, from 1 h to 12 h ahead. Figure 11 shows the results. As expected, the forecasting errors increase as the prediction horizon becomes larger and the coefficient of determination decreases. Note that performance indices vary more significantly for the forecast window of up to 3 h ahead. From this point, the forecasts do not deteriorate, which is positive. In addition, Cluster 1 shows more errors and higher deterioration in forecasting performance. This can be explained by the fact that this cluster holds data with rainier days, with a higher level of precipitation and a lower level of solar irradiation.

5.5. Comparison with Benchmark Dataset

In this section, a well-known dataset for time-series forecasting is used to validate the proposed algorithm. The temperature time series shows the mean monthly air temperature measured at Nottingham Castle from 1920 to 1939 [33].

The dataset is divided into training and test sets, with a split of 70% and 30%, respectively. The VOWA approach was obtained following the same procedure previously adopted: combining the two ML algorithms that produced the best results, in this case, AdaBoost and RF. The forecasting results are presented on

Table 9. Forecasting Accuracy of ML Models for Benchmark Dataset.

	MAE	RMSE	MAPE (%)	R2
VOWA	1779	2300	3759	0929
CatBoost	2518	3091	5206	0871
RF	2031	2592	4183	0909
XGBT	2187	2764	4567	0897
AdaBoost	1796	2365	3828	0924

. In the benchmark results, VOWA outperformed the other ML algorithms in all metrics and in all clusters, confirming superior performance.

6. Conclusions

This paper proposes an ensemble voting method using ML algorithms to forecast solar irradiation. Several ensemble models were evaluated using average voting and weighted voting based on RF, XGBT, CatBoost, and AdaBoost. Feature selection was performed to select inputs and their corresponding delay values, and a clustering algorithm was used to group data according to weather characteristics.

First, the performance of the ML models was tested individually. Results showed that CatBoost had the best forecasting performance against the other ML models, presenting the following average performance metrics among the three clusters: MAE of 0.259, RMSE of 0.379, MAPE of 26.283%, and R² of 0.845. Although AdaBoost has practical advantages, such as low implementation complexity and lower tuning parameters, it presented the worst forecasting performance, with the following average metrics among the three clusters: MAE of 0.327, RMSE of 0.435, MAPE of 45.650%, and R² of 0.798. Then, different voting ensemble models were tested. Weighted average voting based on CatBoost and RF presented superior accuracy compared to the single algorithms and other ensembles tested, with the following average metrics: MAE of 0.256, RMSE of 0.377, MAPE of 25.659%, and R² of 0.848. The Diebold–Mariano statistical test was applied to compare the weighted average voting algorithm against the other methods. Results showed that the proposed model is significantly more accurate than the other models.

The performance of weighted average voting was also tested for different forecast horizons, from 1 h to 12 h ahead. Results showed that accuracy deteriorates more significantly for the forecast window of up to 3 h ahead and remains almost the same from this point forward. A well-known dataset for time-series forecasting was used to validate the results and confirmed superior performance for the weighted average voting algorithm.

This study presented interesting results, but several issues still need to be further investigated. The selection of appropriate values for weights in the weighted average voting approach remains a challenging task, and optimization algorithms can be tested in future studies. Another direction for future work is to apply the proposed methodology in other datasets from places with different weather conditions. Furthermore, the proposed methodology can be applied to solving other forecasting problems, such as wind speed prediction.