Solar Power Generation Forecasting in Smart Cities and Explanation Based on Explainable AI

Highlights

What are the main findings?

• LightGBM, among the selected black-box models, was identified as requiring explanation due to its performance in solar power generation forecasting.
• “Distance from the Noon” and its interaction with “Sky Cover” were highlighted as the primary environmental factors influencing solar power generation.

What is the implication of the main finding?

• Understanding the key environmental factors enables more accurate placement and optimization of solar power stations in smart cities.
• The use of Explainable AI provides valuable insights that can guide policymakers and engineers in enhancing solar energy infrastructure.

Abstract

The application of black-box models, namely ensemble and deep learning, has significantly advanced the effectiveness of solar power generation forecasting. However, these models lack explainability, which hinders comprehensive investigations into environmental influences. To address this limitation, we employ explainable artificial intelligence (XAI) techniques to enhance the interpretability of these black-box models, while ensuring their predictive accuracy. We carefully selected 10 prominent black-box models and deployed them using real solar power datasets. Within the field of artificial intelligence, it is crucial to adhere to standardized usage procedures to guarantee unbiased performance evaluations. Consequently, our investigation identifies LightGBM as the model that requires explanation. In a practical engineering context, we utilize XAI methods to extract understandable insights from the selected model, shedding light on the varying degrees of impact exerted by diverse environmental factors on solar power generation. This approach facilitates a nuanced analysis of the influence of the environment. Our findings underscore the significance of “Distance from the Noon” as the primary factor influencing solar power generation, which exhibits a clear interaction with “Sky Cover.” By leveraging the outcomes of our analyses, we propose optimal locations for solar power stations, thereby offering a tangible pathway for the practical.

1. Introduction

In the face of rapid urbanization and the rise of smart city initiatives, the need for efficient, sustainable energy management systems has become increasingly urgent [1,2,3,4]. Solar power, a renewable and clean energy source, offers a promising solution to meet these needs. However, effectively incorporating solar power into smart city energy grids requires precise and understandable forecasts to optimize its use [5,6]. Accurate forecasting improves the reliability of solar power, supports real-time decision-making, reduces operational costs, and maximizes energy efficiency. Additionally, clear and interpretable forecasting models promote transparency, enabling stakeholders to comprehend the factors influencing predictions. This fosters trust and aids in making informed policy decisions. As smart cities continue to develop, enhancing these forecasting techniques is crucial to support their growth and sustainability goals [7,8,9]. The abbreviations used in this work can be found in

Table 1. Nomenclature and abbreviations.

Abbreviation	Explanation	Abbreviation	Explanation
XAI	Explainable Artificial Intelligence	ARIMA	Autoregressive Integrated Moving Average
XGBoost	Extreme Gradient Boosting	ANN	Artificial Neural Networks
LightGBM	Light Gradient Boosting Machine	CNN	Convolutional Neural Networks
CatBoost	Categorical Gradient Boosting	SHAP	Shapley Additive Explanations
RNN	Recurrent Neural Networks	Bi-RNN	Bidirectional RNN
LSTM	Long Short-Term Memory	Bi-LSTM	Bidirectional LSTM
GRU	Gated Recurrent Unit	Bi-GRU	Bidirectional GRU
GOSS	Gradient-Based One-Side Sampling	STBS	Symmetric Tree-Based Sampling
MLP	Multilayer Perceptron	IoT	Internet of Things

.

As AI systems continue to permeate various industries and domains, the need for explainability and transparency has become paramount. XAI [10,11,12,13], a field dedicated to developing AI models that can provide human-understandable explanations for their decisions and outputs, holds the key to unlocking the full potential of AI while ensuring its secure and responsible deployment. One of the primary obstacles hindering the widespread adoption of XAI is the scarcity of real-world application scenarios. By investigating the application of XAI in the solar power generation industry, this research aims to contribute to the advancement of XAI technology and foster its practical implementation.

Solar power, as a clean and renewable energy source, provides numerous environmental and economic benefits [14]. It effectively minimizes carbon emissions and enhances energy security by reducing reliance on fossil fuels. The optimization of energy trading, reduction in operational costs, improvement of system reliability, and facilitation of the integration of solar power into the grid [15] can be achieved through accurate long-term forecasting. Predictions for solar power generation can be made using statistical methods, traditional machine learning techniques, and deep learning (Figure 1). Statistical methods offer a straightforward approach, while traditional machine learning techniques like decision trees, shallow neural networks, and ensemble learning, provide more sophisticated methods. Deep learning [16], an evolution of shallow neural networks, is capable of handling complex and nonlinear tasks by increasing the network’s depth. Section 2 will discuss the differences and relationships between these approaches in detail.

Linear regression and ARIMA [17] are two commonly employed methodologies for time series forecasting. Linear regression is utilized to model the overall trend in data by assuming a linear association between input and output variables. However, this technique may not be suitable for adequately capturing the cyclicality or seasonality observed in the data, which presents a limitation when attempting to predict solar power generation. Conversely, ARIMA is extensively used in time series forecasting and possesses the capability to effectively capture both seasonal patterns and trends within the data. Nonetheless, obtaining a stationary dataset can pose challenges when dealing with solar power generation due to the inherent variability of solar energy, which constitutes a prerequisite for applying ARIMA. These limitations become particularly pronounced in long-term forecasting scenarios, thus motivating the application of machine learning and deep learning algorithms within this domain.

Ensemble learning [18] and deep learning techniques have proven to be highly effective for handling nonlinear and non-stationary data. Ensemble learning involves combining the predictions of multiple models in order to improve accuracy, while deep learning utilizes intricate network structures to identify relationships between input and output variables. These approaches hold particular significance in domains such as solar and wind energy forecasting [19], where nonlinear relationships and patterns are frequently observed. Ensemble learning encompasses Boosting and Bagging algorithms, with Boosting algorithms such as XGBoost [20], LightGBM [21], and CatBoost [22] being widely utilized. On the other hand, Deep Learning can be classified into three distinct types based on their network structure: ANN [23], CNN [24], and RNN [25].

XGBoost, LightGBM, and CatBoost have exhibited remarkable efficacy in time series forecasting competitions. Notably, XGBoost has been extensively employed in the M4 competition [26]. LightGBM has proven its effectiveness in handling vast datasets and achieving rapid training times, rendering it a popular selection in diverse competitions. In fact, LightGBM surpassed other frameworks and emerged victorious in the M5 forecasting competition [27], thus highlighting its exceptional performance in real-world scenarios. CatBoost is also widely utilized in Kaggle time series forecasting competitions. All three frameworks have demonstrated their potency for long-term time series forecasting, with each possessing unique advantages that render them suitable for distinct use cases [28,29]. Bae D.J. et al. [30] proposed an XGBoost-based load forecasting algorithm, which enhanced accuracy by 21% and 29% in 2019 and 2020, respectively, when compared with previous models. Zhang Y. et al. [31] assessed mainstream prediction methods across various datasets, including solar power generation. The experimental results indicated that LightGBM was the superior algorithm overall, outperforming others in the three datasets. As another notable Boosting algorithm, CatBoost also exhibits evident potential in solar power generation forecasting [32]. Deep learning techniques are frequently employed for time series forecasting, particularly ANNs [33], RNN-based models (RNN [34], LSTM [35]) and bidirectional RNN-based models [36] (Bi-RNN, Bi-LSTM and Bi-GRU). These models possess the capacity to automatically extract pertinent features from input data, providing an advantage over ensemble learning, which relies on manual feature engineering. ANN excels at capturing intricate nonlinear patterns in scenarios where the relationships between input features and target variables are unclear or nonlinear. While RNN-based and Bi-RNN-based models were initially devised for natural language processing [37], they have become extensively adopted in solar power generation forecasting due to their ability to capture temporal dependencies. Given that different models may possess varying abilities to learn from data, it is imperative to conduct comprehensive comparisons and analyses tailored to specific cases [31].

Ensemble learning and deep learning represent prominent methodologies for time series forecasting, particularly in the context of intricate and long-term predictions. Nevertheless, their opaque nature poses a challenge to discerning the factors that impact the accuracy of these forecasts [38]. Conversely, the identification of such factors assumes great importance when optimizing solar energy systems. Factors such as weather conditions [39], shading [40] and equipment characteristics [41] assume a pivotal role in system performance, cost reduction and site selection. Scrutinizing these factors can mitigate risks, optimize energy generation and facilitate the selection of the most suitable location for a solar power plant. Consequently, successful solar power generation necessitates the critical analysis of influential factors alongside effective forecasting techniques. XAI presents an opportunity to analyze the influential factors associated with black-box models. Developed to augment transparency, accountability and trust in AI systems, XAI empowers users by providing insights into the decision-making processes underlying these models. This is achieved through the provision of explanations or justifications for their outputs or decisions. By adopting XAI methods, AI systems become more comprehensible and accessible to users. An exemplary technique in the field of XAI is SHAP [42,43], which exhibits consistency, model-agnosticism and local interpretability. SHAP operates by utilizing SHAP values, derived from Shapley values [44] within cooperative game theory, to elucidate the output of a machine learning model. These values quantify the contribution of each variable towards a prediction, accounting for the interactions between variables. Consequently, SHAP offers a reliable and transparent approach to expounding machine learning models, thereby enabling users to enhance their understanding and trust in the decision-making processes of AI systems.

Several studies have been conducted to forecast solar power generation using various methods. However, such studies require attention to some issues that can affect their consistency and reliability. Firstly, there is a lack of standard process in applying these methods, which leads to inconsistencies in results [15]. Secondly, there is an insufficient analysis of factors affecting solar power generation [15,19,30,32,33,34,35,36,45]. Future research should address these issues to enhance the quality and reliability of such studies. This is crucial for developing accurate and effective methods for forecasting solar power generation, which supports the growth and sustainability of renewable energy.

This research makes a significant contribution to the field by focusing on the long-term forecasting of solar power generation and analyzing various environmental factors that affect it. To conduct this study, we obtained data from the Berkeley Power Plant. In Section 2, we present the methodology employed in our investigation. Section 2.1 outlines the standardized procedures for utilizing machine learning techniques. We then introduce ensemble learning in Section 2.2, followed by an exploration of deep learning methods in Section 2.3. Moving on to Section 2.4, we discuss the objective function utilized in our research. Additionally, we provide an overview of explainable AI and SHAP techniques in Section 2.5. In Section 3, we delve into the visualization of data and analyze the variables involved. Finally, in Section 4, we present our results and engage in a comprehensive discussion. We initiate this section by describing the process of data preprocessing in Section 4.1, followed by an evaluation of the forecasting performance in Section 4.2. Furthermore, we conduct an analysis of the factors that influence solar power generation in Section 4.3. In Section 6, we further elaborate on and discuss the outcomes derived from our study.

We emphasize two main contributions made by this research: (1) the establishment of a standardized usage process for machine learning techniques to ensure fair performance comparisons; and (2) an exploration of real-world applications of XAI technology for solar power generation.

2. Methodology

2.1. Standardized Usage Procedures of Machine Learning

In our study, we have placed a strong emphasis on the adoption of standardized procedures for using machine learning techniques. This decision stems from our observation of inconsistent and irregular model comparison practices in real-world applications [46]. Such inconsistencies can lead to unreliable results and hinder meaningful comparisons between different models. Standardization in machine learning refers to implementing a uniform and methodical approach when applying algorithms. Given that machine learning models rely heavily on data, a standardized process is essential to ensure the accuracy and validity of results obtained from comparing various algorithms. Consistent methodologies not only enhance the reliability of performance evaluations but also aid in determining the most effective algorithm suited for a specific task. This standardized approach involves several key components. First, it necessitates the use of consistent training data across experiments. By ensuring that all models are trained and tested on the same dataset, we can provide an equitable basis for comparison. Additionally, the way datasets are split into training and testing subsets must be consistent to prevent biases that could skew results. Overall, adopting these measures promotes fair evaluation and improves the overall robustness of model assessments in the context of solar power generation forecasting in smart cities.

2.1.1. Consistent Training Data

Machine learning methods differ from statistical methods in that they depend on data, rather than patterns. Therefore, consistent training data are essential to ensure fair and reliable comparisons between different machine learning methods. Comparing models trained on different datasets renders any such comparisons meaningless. Regrettably, some studies in the field of solar energy prediction have violated this principle. For instance, in a particular study [47], the authors used 24 months of data for a CNN model but only 12 months of data for other models, allowing the CNN model to outperform the others. However, such results are unrealistic and unreliable because they were obtained through an unfair comparison. It is crucial to emphasize the need for consistent training data when conducting machine learning experiments to ensure fair and accurate results.

2.1.2. Splitting of the Dataset

In the field of machine learning, it is crucial to partition the dataset into three subsets: the training set, validation set, and test set. Statistical methods typically divide the data into two parts: the fitting set and the testing set (Figure 2). The fitting set is used to fit mathematical formulae with the expertise of the human brain, while the testing set is used to evaluate the performance of the fit. In contrast, machine learning methods use a process of learning to construct a simulated fit, where the machine learns a large number of parameters with nonlinear properties to achieve a near-perfect fit to the training set. However, this perfect fit is often an uninterpretable black-box model and can lead to overfitting problems [48]. To obtain a generalized model, we need to constantly and artificially adjust hyperparameters, such as the number of layers, the number of neurons and the batch size of the network. This is where the validation set comes in. By verifying the suitability of these hyperparameters, we can keep the hyperparameters that make the model perform optimally in the validation set and fix the model. Finally, the performance of this fixed model on the test set is considered as its predictive performance, ensuring the reliability of the results to a large extent. Therefore, it is crucial to include the validation set in the dataset division process to produce more accurate and reliable prediction results.

Dividing a dataset into only two parts poses the risk of authors intentionally or unintentionally reporting results in their favor, compromising the fairness of comparisons. To ensure fairness, data should be split into three parts: a training set, a validation set, and a test set. The training set allows the machine to learn, while the validation set helps find the optimal hyperparameters of the model and fixes it. Finally, all fixed models are fairly compared on the test set.

2.2. Ensemble Learning

The Boosting algorithm is the most typical ensemble learning, and its principle is very simple, i.e., numerous weak learners are connected in series to build a strong learner. This study focuses on decision tree-based Boosting algorithms, which have demonstrated excellent performance in recent competitions. The three algorithms explored are XGBoost, LightGBM and CatBoost. XGBoost [49] is an upgraded version of Gradient Boosting, which enhances generalization ability by including a regular term as the sum of k trees’ complexity and second-order derivative information utilizing Taylor’s expansion formula. This results in learning richer information compared to traditional Gradient Boosting algorithms. LightGBM is designed for large-scale datasets and high-dimensional feature spaces, using a unique technique called GOSS [50] to select informative data points for training, hence reducing memory consumption and the required computation time. CatBoost is optimized for categorical features and employs a method called Ordered Boosting that encodes categorical features preserving their ordering, allowing them to be treated as numerical features. Additionally, it uses STBS [51] to handle missing values, making it more efficient than other Boosting algorithms.

Figure 3 visualizes the difference among the three of them. XGBoost uses depth-wise leaf growth to create a tree structure that is symmetric through balancing the instances in each node at every level of the tree. In contrast, LightGBM uses leaf-wise leaf growth, meaning it grows trees by selecting the leaf with the most significant gain, leading to deep but sparse trees. It also utilizes histogram-based Gradient Boosting to efficiently train unbalanced trees. CatBoost combines both approaches by using a predefined split threshold to determine whether to grow the tree depth-wise or leaf-wise and balances the number of instances in each leaf node to prevent overfitting.

2.3. Deep Learning

2.3.1. Artificial Neural Networks

ANNs (or MLP) are effective models for learning intricate nonlinear connections between the input and output variables [52]. ANNs comprise interconnected artificial neurons that transmit data from the input layer to the output layer in a feed-forward fashion. The input layer accepts input data, while the output layer produces the model’s output. Hidden layers, situated between the input and output layers, are referred to as intermediate layers. Each neuron in an ANN is characterized by a unique set of weights and biases that regulate its output (Figure 4). The equation for the output of an MLP with L layers can be written as

$$y=f_L\left(f_{L-1}(\cdots f_2\left(f_1\left(h_1\right)\right)\cdots )\right)$$

(1)

where y is the output of the MLP; $f_L$ is the activation function of the output layer; $f_1$, $f_2$, …, $f_{L-1}$ are the activation functions of the hidden layers; $h_i=w_i\times x+b_i$; $w_1$, $w_2$, …, $w_L$ are the weight matrices that determine the strength of the connections between layers; and $b_1$, $b_2$, …, $b_L$ are the bias vectors for each layer.

The activation function f can be any nonlinear function, such as the $sigmoid$ [53], $ReLU$ [54], or $tanh$ [55]. The weights and biases of the network are updated during training through an iterative learning algorithm known as back-propagation [56]. This algorithm employs gradient descent to minimize the discrepancy between predicted output and actual output by modifying the weights and biases accordingly.

2.3.2. RNN-Based Models

RNNs [57] are a specific type of neural network architecture designed to process sequential data by integrating an internal state that captures the temporal dependencies between inputs. Unlike traditional MLP models, which process input data in a single pass and do not maintain any memory of previous inputs, RNNs can process input sequences of variable lengths. At each time step, the internal state of the RNN is updated and acts as a form of memory [58], enabling the network to retain and integrate past inputs into its current processing. This characteristic makes RNNs highly adaptable for applications that require the handling of sequential data, such as speech recognition [59], natural language processing and time series forecasting. An illustration of the RNN’s capacity to incorporate past inputs into its current processing is provided in Figure 5. The equation for the output of an RNN can be written as

$$\begin{array}{c}\hfill h_t=f(U{x}^t+W{h}_{t-1}+b_h)\end{array}$$

(2)

$$\begin{array}{c}\hfill O^t=Softmax(V{h}_t+b_o)\end{array}$$

(3)

where $O^t$ is the output at time step t; $h_t$ is the hidden state at time step t; f is the activation function; W is the weight matrix for the previous hidden state; $b_i$ is the bias term of each layer; U is the weight matrix for the input at time step t; and $x^t$ is the input at time step t.

However, gradients can vanish or explode during training in RNNs [60]. This happens because gradients propagate back in time and their magnitude can increase or decrease exponentially, making it difficult for the RNN to retain information from earlier inputs, which ultimately impacts network accuracy. To address this problem, LSTM was developed [61]. LSTM is a type of RNN that uses a memory cell capable of maintaining its state over extended periods. The architecture consists of three gates, which regulate the flow of information into and out of the memory cell (Figure 6, left): input gate, output gate, and forget gate. During training, the network learns how to use these gates to selectively preserve or discard information. Specifically, the input gate controls what information from the input sequence should be stored in the memory cell, the output gate decides which information from the memory cell should be outputted, and the forget gate manages what information should be discarded. This selective retention and discarding of information enable the LSTM to capture long-term dependencies in the data and provide superior performance compared to traditional RNNs.

GRU [62], another type of RNN architecture, shares similarities with LSTM but does not have a separate memory cell. Instead, it uses a single hidden-state vector that integrates the memory cell and output gate of LSTM into a single component called the update gate. Additionally, GRU also has a reset gate that controls information to be discarded from the previous hidden state. Overall, GRUs are a simpler alternative to LSTM that can achieve comparable performance on many sequence modeling tasks [63,64], while requiring fewer parameters and less computation.

2.3.3. Bi-RNN-Based Models

The Bi-RNN [65] is a neural network architecture that consists of two RNNs operating on the input sequence in opposing directions. This design aims to capture both forward and backward dependencies critical for many tasks involving sequence modeling. The final output of the Bi-RNN is produced by concatenating or combining the outputs of both RNNs. Bi-RNN models enhance the RNN framework by incorporating a second layer to capture additional temporal information more effectively. The formulations for Bi-RNN, Bi-LSTM and Bi-GRU are fundamentally similar to their unidirectional counterparts (Equations (2) and (3)) [66,67]; therefore, this work does not provide reproducible results. Figure 7 shows the structure of a Bi-RNN model.

2.4. Objective Function

In order to evaluate the performance of machine learning models, various criteria have been established. Three commonly used criteria are the Mean-Squared Error ($MSE$), Mean Absolute Error ($MAE$) and the coefficient of determination ($R^2$). They can be calculated as follows:

$$\begin{array}{c}\hfill MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2\end{array}$$

(4)

$$\begin{array}{c}\hfill MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|y_i-{\widehat{y}}_i\right|\end{array}$$

(5)

$$\begin{array}{c}\hfill R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2}{{\sum }_{i=1}^n{\left({\widehat{y}}_i\right)}^2}}\end{array}$$

(6)

where $y_i$ is the true value; ${\widehat{y}}_i$ is the predicted value; and n is the number of observations.

2.5. Explainable AI

The prevalence of black-box models in modern AI systems is a concern for users as they only offer predicted outcomes without providing information on how model variables influence these predictions. On the other hand, interpretable models like linear regression provide clear insight into variable contributions to the model output. To address this issue, XAI [11,68] has been developed to enable humans to understand AI systems’ decision-making processes. XAI methods focus on exploring feature importance and interaction within black-box models during the training process. This approach offers benefits for domain experts, especially in fields such as photovoltaic power generation, as it provides interpretative results that aid with model rationality assessment and allows users to discover interactive relationships between variables.

SHAP is a widely used method for explaining the predictions of complex machine learning models. This method is model-agnostic, making it applicable to a wide range of models, including tree-based models, neural networks and linear models. The SHAP method is rooted in Shapley values, which measure the contribution of each player in a game. Shapley value is a well-established theory in cooperative games that seeks to allocate benefits equitably among players in a coalition. In recent years, there has been growing interest in applying this theory to model explanation in machine learning [69,70]. This is because there is a correspondence between the Shapley value and model explanation: the variables used for training can be thought of as “players” and the model’s predictions as “revenues”. Shapley value is defined by the following formula:

$$k_i=\sum _{S\subseteq N\setminus \left\{i\right\}}{\displaystyle \frac{\left|S\right|!(n-\left|S\right|-1)!}{n!}}\times (v(S\cup \left\{i\right\})-v\left(S\right)).$$

(7)

where $k_i$ is the contribution of the i-$th$ variable; N is the set of all players (features) (1, 2, 3 … i…n), which is the complete set; S is a subset of N, in which the explained feature i is removed, with a total of $2^N$; v is the gain function ($v\left(S\right)=E_{\widehat{D}}\left[f\left(x\right)\mid x_S\right]$, where $\widehat{D}$ is the empirical distribution of the training data; and f is the black-model.

The SHAP method offers both a global and local perspective on feature (variable) importance, indicating how each feature influences the overall predictions as well as how it affects a particular instance’s outcome. In recent years, the SHAP method has become increasingly popular across various domains, such as healthcare, finance and image recognition. By providing intuitive and insightful explanations for complicated machine learning models, the SHAP method is known to increase the transparency and interpretability of these models, thus building trust in their predictions.

3. Data Visualization and Variable Analysis

The dataset consists of nine environmental variables and a solar power generation forecasting target. Two of the variables in the dataset contain temporal information. The first variable records the time at which data were collected, with recordings taken every three hours, eight times per day (at hours 1, 4, 7, …, and 22). The second variable indicates whether it is daylight or night, with the period from 7:00 to 22:00 recorded as daylight and the period from 22:00 to 07:00 recorded as night. The remaining seven variables, excluding time-point variables, are presented in the subsequent Data Visualization section. The original dataset from the Berkeley power plant can be accessed directly on Kaggle https://www.kaggle.com/datasets/vipulgote4/solar-power-generation (accessed on 10 January 2024).

This study used a dataset with 2920 observations, which was split into three subsets: a training set (70%), a validation set (15%), and a test set (15%). The dataset contained nine variables, with six being numeric and three being categorical. The categorical variables are “Is Daylight”, “Hour” and “Sky Cover” (the classification description is depicted in Figure 8). The distribution of the Sky Cover classes is shown in Figure 8b. Numeric variables include “Distance to Solar Noon (km)”, “Average Temperature (°F)”, “Average Wind Direction (°)”, “Average Wind Speed (m/s)”, “Relative Humidity (%)” and “Average Barometric Pressure (inHg.)”. The data distribution is shown in Figure 8c–h.

As shown in Figure 9, most variables remain independent of each other, except for a strong negative correlation between “Distance” and “Is Daylight” (−0.84), while “Humidity” and “Sky Cover” had some positive correlation (0.42).

4. Results and Discussion

4.1. Data Preprocessing

In this study, our tasks involve comparing different long-term forecasting models and analyzing factors that impact solar power generation. The data processing approaches differ for these two tasks. For the forecasting task, we set the horizons h to 56, 120 and 240, representing the forecasted solar power generation for the next 7, 15 and 240 days. The dataset is segmented into 3-h intervals, resulting in eight data points per day (at hours 1, 4, 7, 10, 13, 16, 19 and 22). Therefore, a forecast horizon of 56 data points equates to 7 days (7 days × 8 data points), a horizon of 120 data points corresponds to 15 days (15 × 8), and a horizon of 240 data points represents 30 days (30 × 8), respectively. To accomplish this task, we construct lag features for each variable, resulting in a dataset following the following form: ${\widehat{y}}_{t+h}=f(X_t,y_t)$, where $X_t=(x_t^1,\cdots ,x_t^n)$.

In contrast, after determining the optimal forecasting model, we employ this model to re-model the original data to explore environmental factors influencing solar power generation. In the analysis phase, the dataset follows the form: ${\widehat{y}}_t=f(X_t,y_t)$. Additional details can be obtained by accessing our experimental code: https://github.com/Zhangyuyi-0825/Solar_power (accessed on 10 January 2024).

In our research, we analyzed data from a specific location in Berkeley and introduced the concept of “Distance to the Solar Noon”. This term describes the direct distance from the recording point to the sun when it is solar noon at that location. Typically, terms such as azimuth and zenith angles are used in geography to describe the sun’s position. “Distance to the Solar Noon” also involves spatial elements since it deals with measuring distance. However, its determination is inherently linked to the timing of solar noon, making it a time-dependent measure.

4.2. Forecasting Performance

The results of the performance comparison of the 10 forecasting models are demonstrated in

Table 2. Comparison of forecasting performance. The optimal model is filtered through the validation set and compared on the test set. The performance metric values reported here are taken from the test set.

Metrics		${\mathit{R}}^2$			MSE			MAE
Horizon	56	120	240	56	120	240	56	120	240
XGB	0.8950	0.8803	0.9365	0.1186	0.1488	0.0895	0.2436	0.2653	0.1429
LGB	0.8856	0.9180	0.9407	0.1126	0.1038	0.0799	0.2199	0.1738	0.1498
Cat	0.9117	0.9153	0.9405	0.0982	0.1041	0.0832	0.1912	0.1771	0.1433
ANN	0.8851	0.8635	0.8152	0.1262	0.1812	0.1989	0.1979	0.2212	0.2595
RNN	0.9196	0.8486	0.8544	0.0971	0.1820	0.1742	0.1854	0.2440	0.2530
LSTM	0.9064	0.8649	0.8393	0.1042	0.1433	0.1611	0.1772	0.2204	0.2452
GRU	0.9116	0.8785	0.8535	0.1049	0.1519	0.1662	0.1873	0.2290	0.2442
Bi-R	0.9095	0.8439	0.8723	0.1023	0.1988	0.1597	0.1934	0.2562	0.2439
Bi-L	0.9051	0.8635	0.8426	0.1050	0.1463	0.1585	0.1762	0.2285	0.2425
Bi-G	0.9131	0.8704	0.8434	0.1050	0.1586	0.1744	0.1877	0.2319	0.2565

. The hyperparameter settings and the configuration information for these models are represented in detail in Appendix A. Overall, LightGBM and RNNs demonstrate relatively superior performance compared to other models. Evaluation results for this study’s dataset indicate that RNN performs better at a forecasting horizon of 56, while LightGBM excels at horizons 120 and 240.

Figure 10 illustrates a visual comparison between their predicted and actual results. Notably, as the forecasting horizon increases, LightGBM consistently outperforms the RNN significantly. Conversely, irrespective of the forecasting horizon, the RNN’s predictions during nighttime periods (red rectangles) exhibit noticeable noise. Specifically, during these nighttime intervals, the solar power generation values in the original data become zero, whereas the RNN generates fluctuating non-zero values at those instances. In contrast, under such circumstances, LightGBM demonstrates more stable performance during nighttime, leading to more reliable and plausible forecasts pertaining to solar power generation.

Although more suitable forecasting models were found for the solar power generation dataset through comparative experiments, these models are different from traditional statistical models in that their output logic is not transparent to us. Rather, we only know that nonlinear relationships exist within them. This lack of transparency prevents us from fully relying on their forecast results and limits our ability to analyze the factors that influence these forecasts. Influencing factor analysis is an effective tool for forecasting and managing the capacity of solar power systems. The identification of key factors that impact system performance, such as weather patterns, geographical location and shading, can enable a more precise forecast of energy generation and an optimization of system performance through improved management strategies. Furthermore, historical data and trends derived from influencing factor analysis may support informed decisions regarding investments in solar technology and infrastructure.

4.3. Influencing Factors Analysis

The study employs the SHAP algorithm as a general method for XAI to perform an influence factor analysis. The SHAP algorithm, which calculates Shapley values, is described in detail in Section 2.5. We utilized LightGBM as an exemplar to showcase the implementation of SHAP. This choice is informed by SHAP’s source code library support for ensemble learning and superior visualization capabilities, which surpass those of deep learning. It is important to acknowledge that this disparity in technical capabilities is confined to the code domains only, and SHAP retains its role as an explanation scheme for all black-box models within the theoretical framework.

It is important to emphasize that the SHAP value of a variable represents the degree of its contribution to the forecasting results, and a larger value indicates that its corresponding variable is more important. We will use Figure 11 as an example to show in detail the analysis of influencing factors using SHAP as the basis of the technique.

Figure 11 shows the SHAP explanations for two sample points with timestamps of “1 September 2008, 7:00” and “1 September 2008, 13:00”. The baseline, denoted as $E\left[f\left(x\right)\right]$, represents unconsidered variables and is substituted with the average value of all forecasting values. In this study, $E\left[f\left(x\right)\right]=0.579$. In Figure 11a, starting from the bottom, the model’s baseline is 0.579. The inclusion of average temperature and average wind speed does not cause a change in the forecasting value $f\left(x\right)$. However, once relative humidity is included, $f\left(x\right)$ begins to vary, and each subsequent variable contributes to the forecasting. The calculation process is as follows: $f\left(X\right)=0.579-0-0+0.01+0.03-0.03+0.05-0.05+0.06-0.28=0.351$.

Thus, SHAP assigns the forecasting value $f\left(x\right)=0.351$ to each variable. The same process applies to Figure 11b. By comparing Figure 11a and Figure 11b, it can be observed that when the time changes from 7:00 to 13:00, most variables’ contributions to the predicted value turn positive. Particularly, as the distance to solar noon decreases, its contribution significantly rises to +1.85, exerting a decisive influence. However, these are only the explanations for two sample points. In practice, it is impossible to analyze each data point individually. Therefore, SHAP also provides global explanations for variables by taking the absolute values of local explanation results and averaging them. This average serves as the global explanation, presented in Figure 12.

Global explanation can comprehensively assess the importance of these variables. Figure 12 illustrates the ranking of variable importance for forecasting outcomes, with variables arranged from top to bottom in descending order based on their importance. For the entire dataset, the “distance to the solar noon” is the most important, with its significance surpassing that of the remaining variables by a large margin. Subsequently, sky cover and relative humidity take precedence, while other variables such as wind direction, wind speed and average temperature do not stand out in this comprehensive assessment. Up to this point, the aforementioned analysis is solely based on the static explanation of SHAP values, without incorporating variable values. Next, we will explore the SHAP values under variable value changes to achieve dynamic analysis. The global dynamic explanation is presented in Figure 13.

The analysis of Figure 13 helps us to examine the presence of a strong monotonic relationship between SHAP values and variable values. The key aspect of analyzing Figure 13 lies in observing the clear distinction between the blue and red regions. For instance, considering the most important variable, i.e., distance to the solar noon, when this variable has lower values, it is represented by blue dots located on the right side. This indicates that lower values of this variable have a positive impact on the forecasting of solar power generation. Conversely, the red dots representing higher variable values are concentrated on the left side, suggesting that larger values of this variable have a negative effect on the forecasting. The more distinct the separation between the red and blue regions, the stronger the monotonic relationship between variable values and their corresponding SHAP values. In addition, SHAP provides an interactive explanation plot (Figure 14 and Figure 15), which not only presents such monotonic relationships in detail, but also illustrates the interaction effects among variables.

We begin by conducting an analysis on continuous variables. As shown in Figure 14, the trend exhibited by the data points represents the relationship between variable values and SHAP values, while the color indicates the variable value that has the most significant interaction effect with the given variable. The influential factors based on interactive plots are analyzed as follows.

Distance to Solar Noon (Figure 14a). The trend displayed by the data points indicates a strong monotonic relationship, wherein the SHAP value decreases significantly as the variable value increases. This implies a considerable reduction in its contribution to the predicted value. Additionally, when the variable value increases to approximately 0.3, its SHAP value remains at a relatively low level (around 0.5); however, when the value reaches around 0.4, its contribution becomes negative. On the other hand, the right side of the plot demonstrates the variable that exhibits a notable interaction effect, which in this case is sky coverage. It can be observed that even when the distance is within 0.3 km, the red data points representing high sky cover can still lower the SHAP value to a lower level. However, once the distance surpasses 0.3 km, due to a significant decrease in power generation, such interaction effects lose their analytical value. Overall, it can be inferred that when the distance does not exceed 0.3 km and the sky cover remains below 2, the model can generate higher forecasting values. Finally, the results of the SHAP analysis indicate that the distance to the solar noon within 0.3 km, combined with a sky cover rate not exceeding 60%, is the most crucial environmental factor for high-quality solar power generation.

Relative Humidity (Figure 14b). As the relative humidity increases, its corresponding SHAP value gradually decreases, indicating a diminishing contribution to the forecasting values. As observed from the figure, when the distance to the solar noon is within 0.3 km, relative humidity below 60% exhibits a relatively positive impact on solar power generation.

Average Wind Direction (Figure 14c). In regard to the dataset, it can be observed that when the distance to the solar noon is within 0.3 km, the average wind direction falls between 25° and 30°, thus favoring solar power generation. However, it should be emphasized that this conclusion is only applicable to the given dataset.

Average Wind Speed (Figure 14d). In reference to this dataset, when the relative humidity remains below 60% and the average wind speed ranges from 15 m/s to 25 m/s, a significant increase in corresponding SHAP values is observed. Consequently, this exerts a positive influence on solar power generation.

Average Barometric Pressure (Figure 14e). Concerning this dataset, when the distance to the solar noon is within 0.3 km, an average barometric pressure ranging from 30 inHg to 30.2 inHg can generate positive SHAP values, indicating a positive effect on the forecasting value. However, the magnitude of this influence is significantly lower than that of the aforementioned four environmental factors.

Average Temperature (Figure 14f). Overall, in comparison with other environmental factors, the average temperature has a relatively minor impact on solar power generation. Based on the results, most data points fluctuate around SHAP = 0, indicating zero contribution to solar power generation. The only notable observation is that when the average temperature drops below 45 °F, it does exert a negative influence on solar power generation. However, above this temperature threshold, its impact remains limited.

Figure 15 illustrates the analysis of discrete variables, with a particular focus on the presentation of “Is Daylight” for the sake of analytical completeness. In practice, when this value is 0, indicating nighttime, the solar power generation decreases to zero. Conversely, when the value is 1, representing daytime, the solar power generation starts to increase.

The analysis of sky cover reveals that when the distance to the solar noon is within 0.3 km, the sky cover does not exceed 2, corresponding to 60%. This situation positively affects the solar power generation capacity by enhancing its efficiency. However, conversely, it leads to a significant negative impact on solar energy generation.

4.4. Application Example

The following conclusions can be derived from the feature importance output obtained through modeling using LightGBM and XAI techniques represented by SHAP. It is evident that the distance from the sun at noon is the most significant environmental factor, forming the foundation for the positive influences of other environmental factors. Specifically, it is crucial to maintain a distance within 0.3 km from the sun at noon in order to maximize solar power generation. Additionally, the following conditions further contribute to an increase in solar energy production: relative humidity below 60%, average wind direction between 25° and 30°, average wind speed ranging from 15 m/s to 25 m/s, average barometric pressure between 30 inHg and 30.2 inHg, average temperature below 45 °F, and cloud cover not exceeding 60%.

Taking these factors into consideration, we propose two potential sites for the power plant based on the topographic map of Berkeley (Figure 16). We have identified Location 1 with an average elevation of 340 m and Location 2 with an average elevation of 370 m. Both locations are recommended as suitable options for the solar power plant site selection.

5. Discussion

5.1. Analysis of the Reasons for the Optimal Overall Performance of Long-Term Forecasting in LightGBM

Currently, AI models used for time series forecasting in smart cities commonly include ensemble learning models and neural network models. Among the ensemble learning models are XGBoost, LightGBM and CatBoost, while the neural network models consist of ANN, RNN, LSTM, GRU, Bi-RNN, Bi-LSTM and Bi-GRU.

To begin with, ensemble learning models generally exhibit superior performance compared to neural network models in this context. Neural networks were originally designed to address complex problems in natural language processing, such as understanding that the sentences “I like to play football on my days off” and “On my days off, I like to play football” convey the same meaning despite their different structures. These tasks require recognizing intricate relationships within unstructured data, a capability at which neural networks excel. However, time series forecasting involves a unidirectional relationship between data points, where earlier data points influence later ones, but not vice versa. This makes neural networks prone to overfitting, especially as the prediction horizon extends. In contrast, ensemble learning models, being less complex, are better suited for handling structured data inherent in time series forecasting.

LightGBM, in particular, stands out among ensemble learning models for several reasons. It employs an efficient histogram-based splitting algorithm, is memory-efficient, and can natively handle categorical features without needing one-hot encoding. Furthermore, LightGBM’s unique leaf-wise growth strategy differentiates it from other ensemble learning models. This approach not only accelerates training but also scales effectively with large datasets, making it particularly advantageous for longer forecasting horizons.

5.2. Limitations of SHAP Applications to Deep Neural Networks

SHAP values, which are used to interpret predictions from deep learning models, have notable limitations. One significant limitation is their computational intensity, particularly with large and complex models. Calculating the contribution of each feature requires evaluating numerous permutations, making this process especially taxing for sequential data models like RNNs and LSTMs. While ensemble learning methods may offer faster computations than neural networks, overcoming these challenges often involves developing more efficient algorithms for estimating Shapley values or employing parallel computing techniques to reduce computation time. Additionally, while SHAP can help automate the identification of interaction effects within models, its effectiveness is limited by its reliance on the dataset. This reliance means that SHAP might not capture all possible interaction combinations, leading to potential gaps in understanding model behaviors.

5.3. Scalability and Infrastructure in Smart Cities

In evaluating the scalability of our forecasting models across multiple smart cities, we leverage transfer learning and adaptive techniques. This approach allows us to fine-tune the model for each city’s unique climate and environment without starting anew for every location. The use of XAI further aids in understanding local variations, ensuring the model remains reliable and accurate across various urban settings. Regarding infrastructure requirements, robust computing resources are essential. Platforms that support distributed computing are crucial for processing extensive datasets from multiple cities efficiently. Additionally, a comprehensive data infrastructure is necessary. This includes the collection of real-time data from IoT devices, weather stations and other relevant sources, along with efficient systems for data storage, processing and transmission. Finally, to maximize efficiency, the forecasting model should seamlessly integrate with existing smart grid and energy management systems within each city.

6. Conclusions

In this study, we assessed the performance of ten ensemble and deep learning models for long-term solar power generation forecasting. We employed the SHAP method, a technique from explainable AI, to examine the influence of various environmental factors on model predictions. Our results indicate that the RNN model is best suited for a 7-day forecast horizon. However, for 15-day and 30-day forecasts, LightGBM emerges as the superior choice due to its enhanced robustness, particularly during nighttime when RNN models tend to exhibit noise. This distinction highlights the importance of selecting appropriate models based on specific forecast horizons and conditions. Through our SHAP analysis, we identified “Distance to Solar Noon”, which reflects altitude, as the most significant environmental factor affecting solar power generation. Other influential factors include maintaining relative humidity between 40% and 60%, average wind direction between 25° and 30°, wind speeds exceeding 15 m/s, sky cover below 30%, and barometric pressure ranging from 30.0 inHg to 30.2 inHg. These findings underscore the role of higher altitudes as beneficial for optimizing solar power output under these conditions. Despite these insights, challenges remain, particularly concerning the application of SHAP to deep neural networks, which can be limited in capturing complex interactions within the data. Addressing these limitations represents an important avenue for future research, alongside exploring additional environmental variables and refining model accuracy across diverse geographical regions and temporal scales. This study contributes to the ongoing discourse on renewable energy forecasting by revealing key insights into model selection and environmental impact factors. The integration of explainable AI techniques further enhances our understanding and offers guidance for policymakers and stakeholders aiming to optimize solar power generation in smart cities. Future research should build on these findings, exploring innovative approaches to overcome existing limitations and extend the applicability of these methodologies in varied contexts.