Performance Evaluation of Two Machine Learning Techniques in Heating and Cooling Loads Forecasting of Residential Buildings

Abstract

Nowadays, since energy management of buildings contributes to the operation cost, many efforts are made to optimize the energy consumption of buildings. In addition, the most consumed energy in the buildings is assigned to the indoor heating and cooling comforts. In this regard, this paper proposes a heating and cooling load forecasting methodology, which by taking this methodology into the account energy consumption of the buildings can be optimized. Multilayer perceptron (MLP) and support vector regression (SVR) for the heating and cooling load forecasting of residential buildings are employed. MLP and SVR are the applications of artificial neural networks and machine learning, respectively. These methods commonly are used for modeling and regression and produce a linear mapping between input and output variables. Proposed methods are taught using training data pertaining to the characteristics of each sample in the dataset. To apply the proposed methods, a simulated dataset will be used, in which the technical parameters of the building are used as input variables and heating and cooling loads are selected as output variables for each network. Finally, the simulation and numerical results illustrates the effectiveness of the proposed methodologies.

1. Introduction

Increasing the number of cities and their populations throughout the world requires a great deal of energy to meet the needs of citizens. Recent studies have predicted that the population of cities will increase up to five billion by 2030 [1]. Nearly 40% of total energy consumption is related to the dwellings, and other building types constitute just a fraction of the buildings [2]. Supplying energy to the citizens requires associated resources in which limited sources are available. As the consumption of residential buildings forms a great amount of demand, regarding social welfare, residential consumption should be monitored and controlled [3]. On the other hand, heating and cooling are the most crucial energy sources among citizens, so in this regard usage of these energies should be managed.

Managing and optimizing the energy consumption of buildings requires having complete information about the performance of the building and environmental factors. Electricity, gas and heating supply are the most important resources of energy in a building, but the important final use applications are elevators, heating ventilation and air conditioning (HVAC), domestic hot water, and so on. Among the aforementioned energy sources, optimal operation of HVAC and indoor condition supply are two important factors in evaluating building energy performance [4,5]. HVAC, as a basic infrastructure in the building, plays an important role by changing the amount of internal cooling and heating loads of residential buildings. Despite the need for this system in buildings, there is a major concern that about 40% of all energy, especially in office buildings, is consumed by this system [6,7]. Forecasting the thermal loads plays an important role in optimizing the cooling and heating cost of the buildings, as the deviation from the optimally scheduled values will increase the total cost considerably [8].

Energy forecasting is a way to reach optimal operation of HVAC and energy management of residential buildings. In this regard, residential buildings’ consumption patterns could be predicted [9]. Nowadays, by developing technology many small-scale smart devices and building management systems (BMSs) can be installed on the residential buildings sites in order to monitor and record the load patterns of the buildings, and also environmental characteristics that have a high impact on the energy forecasting. Using such data, building consumption patterns could be predicted and controlled hourly. In addition, the necessity of energy forecasting has been expressed by some researches e.g., a review study of thermal energy consumption in the buildings pertinent to the comfort was introduced in [10] and the purpose of the study was showing how social-economic, fuel mix and climate change are affected by thermal energy comfort. Moreover, by load forecasting, buildings can schedule for the next day, not only to participate in the demand–response programs [11,12,13] but also to participate in energy trading [14,15].

So far, many studies have been done in order to evaluate load forecasting of the buildings. In a valuable study [16], an integrated design approach has been utilized to estimate life-cycle energy savings, cost-effectiveness of energy efficiency measures in new buildings, and carbon emission reduction. In [17], a multi-objective optimization for energy refurbishments of existing buildings through energy efficiency measures and HVAC systems have been carried out using a genetic algorithm coupled with a dynamic simulation tool. Predictor methods for heating load based on artificial neural networks (ANN) have been evaluated in [18] for office buildings where the impact of data size and dimensionality in ANN was inspected. In order for heating, ventilation, and air-conditioning (HVAC) system optimization in [19], electricity load forecasting based on ANN has been studied. Among three utilized algorithms such as Levenberg-Marquardt, Scaled Conjugate gradient back-propagation, and Bayesian Regularization (BR), the BR-based ANN showed the best performance. Another study proposed the energy forecasting method using statistical analysis for heating and cooling of an office building [20]. In [21], four hybrid techniques based on artificial neural network (ANN) and meta-heuristic algorithms such as artificial bee colony (ABC) optimization, particle swarm optimization (PSO), imperialist competitive algorithm (ICA), and genetic algorithm (GA) have been suggested for forecasting the heating load of buildings’ energy efficiency. Forecasting the cooling load has been done in [22] using a probabilistic entropy-based neural (PENN) method. Short-term cooling load prediction in order to optimize the operation of HVAC systems and energy efficiency measures in buildings has been done in [23] using multiple nonlinear regression (MNR), auto regressive (AR), and autoregressive with exogenous (ARX) models. In [24], the thermal comfort reduction of the energy consumption in the building by 36.5% was performed via a feedforward neural network (FFNN). A decision tree method has been suggested in [25] for energy demand forecasting and energy efficiency measures of a residential building. A comparative study of forecasting methods for heating and cooling load was done in [26], where machine learning techniques such as a deep neural network (DNN), gradient boosted machine (GBM), Gaussian process regression (GPR) and minimax probability machine regression (MPMR) were compared with each other. In [27], prediction of the cooling and heating loads of the building were done using ANN, classification and regression tree (CART), general linear regression (GLR), and chi-squared automatic interaction detector (CHAID). In the same work, the technical characteristics of the building were considered as input to the networks. In [28], sixteen residential buildings were evaluated in terms of heating and cooling energy consumption forecasting via adaptive linear time-series models. Likewise, cooling load forecasting based on data mining techniques was proposed in [29] to help design a more efficient building management system (BMS). In [30], the BMS based on electrical, economic, and ecological optimization using a genetic algorithm was introduced to improve energy efficiency of the buildings. General regression neural network (GRNN) has also been employed in [31] for cooling energy forecasting to optimize HVAC heat storage of public buildings.

In most of the aforementioned research works, meteorological data was used as an indicator and input for forecasting the cooling and heating loads of residential buildings. It is undeniable that environmental and meteorological factors do not affect the cooling and heating loads of residential buildings, but sometimes abrupt changes in weather could disrupt energy forecasting equations, reducing the accuracy coefficient and increasing the error in the energy forecasting operation. In this paper, high-precision prediction of cooling and heating loads of a building was done by using multilayer perceptron (MLP) and support vector regression (SVR) methods. A set of data on structural characteristics of the building was considered as an input variable, while the amount of cooling and heating load was considered as an output variable. Using this data and creating a linear mapping between input and output variables via the proposed methods, it is possible to make a more accurate prediction of cooling and heating loads.

The rest of the paper is organized as follows: Section 2 describes the case study and dataset. Section 3 represents the employed methodologies and techniques. Section 4 includes the simulation and numerical results and finally, Section 5 concludes the paper.

2. Case Study

The dataset used in this work was created by Tsanas and Xifara [32]. Twelve different buildings were simulated in Ecotect software to generate the dataset. The glazing area, distribution of the glazing area and the orientation are the parameters that make the buildings different from each other. Each building was simulated using eighteen preliminary cubes (3.5 × 3.5 × 3.5 m³) with the same materials for all buildings. The newest and most common materials in the building construction industry were selected for each of the eighteen elements so that the materials used for each of these elements were the same for all forms of construction. In the design process, three types of glazing areas such as 10%, 25%, and 40% were used as percentages of the floor area. In addition, it was assumed that buildings were in Greece, Athens. Sixty percent humidity, 0.3 m/s wind speed, lightning level of 300 $1x$ and 0.6 clo of clothing were considered as internal design conditions during simulation, while the infiltration rate was set to 0.5 for the air change rate with a wind sensitivity of 0.25 air changer per hour. The dataset includes 768 samples with eight features for each sample, namely $x_1,x_2,\dots ,x_8$ and $y_1,y_2$ as decision variables, which are listed in

Table 1. Details of input and output data.

Mathematical Symbol	Variables
$x_1$	Relative compactness
$x_2$	Surface area
$x_3$	Wall area
$x_4$	Roof area
$x_5$	Overall height
$x_6$	Orientation
$x_7$	Glazing area
$x_8$	Glazing area distribution
$y_1$	Heating load
$y_2$	Cooling load

[21,32]. This work aims to forecast $y_1$ as the heating load and $y_2$ as the cooling load using the aforementioned features as decision variables. Although the dataset was generated via simulation, it is notable that the proposed methods are applicable to the real-world dataset.

3. Methods

Artificial neural network (ANN) and machine learning algorithms as powerful tools in data mining were employed to do the modelling and forecasting tasks [33,34]. In this work, MLP and SVR were used as two application models of these algorithms to create a linear mapping between the technical parameters of building and the cooling and heating loads of the building in order to forecast the load/energy. In the following, each of the proposed methods are briefly introduced.

3.1. Multilayer Perceptron (MLP)

MLP has a fully connected layer structure, i.e., each neuron in a layer is connected to all neurons in the next and previous layers. The schematic of the MLP structure is shown in Figure 1, which illustrates a nonlinear mapping between the input vector and the output vector [35]. The neurons are connected through weights, and output signals are generated by a nonlinear transfer function [36].

$$Y=f(b+{\displaystyle \sum _{l=1}^N{w}_l{x}_l})$$

(1)

In Equation (1) $Y$ and $x$ are the output and input signals, respectively, f is the nonlinear transfer function, b and $w$ are the bias and weight vectors, respectively, and $N$ is the total number of the inputs. Since MLP has the ability to learn through training, a dataset with known input vector and output vector is required in which the weight vector is adjusted according to the output signals through training [37].

3.2. Support Vector Regression (SVR)

SVR is one of the training tools which was developed from the support vector machine (SVM). The principle of SVR is depicted in Figure 2. In this work, ε-SVR is employed for the training of data. ε-SVR is a classic model of SVR with the aim of finding a flat function, which has a small (ε) error from the obtained target [38]. In the case of SVR, the following function is trained using given training data such as $\{(x_1,y_1),(x_2,y_2),\dots ,(x_l,y_l),\dots (x_N,y_N)\}\subset \chi \times \mathbb{R},l=\{1,2,\dots ,N\}$, where $\chi$ illustrates the space of the input patterns:

$$f(x)=\langle w,x\rangle +b;w\in \chi ,b\in \mathbb{R}$$

(2)

where b is bias, and $\langle w,x\rangle$ represents the linear function of fitting input space to the feature space. Equation (3) is using to minimizing the risk function as follows:

$$R=\frac{1}{2}{\Vert w\Vert }^2+c{\displaystyle \sum _{i=1}^N(y_1,\langle w,x\rangle )}$$

(3)

In Equation (3), the selected loss function and $c>0$ specifies the tradeoff between the smoothness of f and allowed deviation larger than ε. In order to deal with the problem, the minimization problem of (4) must be solved.

$$\min \frac{1}{2}{\Vert w\Vert }^2+c{\displaystyle \sum _{l=1}^N({\xi }_l+{\xi }_l^{*})}$$

(4)

$$s.t.\{\begin{array}{l}{y}_l-\langle w,x_l\rangle -b\le \epsilon +{\xi }_l\\ \langle w,x_l\rangle +b-y_l\le \epsilon +{\xi }_l^{*}\\ {\xi }_l+{\xi }_l^{*}\ge 0\end{array}$$

where, ${\xi }_l,{\xi }_l^{*}$ are the slack variables which tackle the infeasible constraints. In order to solve the optimization problem, the dual problem of the (4) can be derived using the Lagrange function. In addition, $w$ can be defined as an integration of training patterns of $x$ linearly. Therefore, Equation (2) can be reformulated as [39]:

$$f(x)=b+{\displaystyle \sum _{l=1}^N({\alpha }_l-{\alpha }_l^{*})}\langle x_l,x\rangle$$

(5)

where, ${\alpha }_l,{\alpha }_l^{*}$ are the Lagrangian multipliers. Then, in order to put the nonlinearity in the algorithm, the training patterns $x_l$ can be modified by a map $\mathsf{\Phi }:\chi \to F$. In addition, Kernel function can be defined as:

$$k(x,x^{\prime }):=\langle \mathsf{\Phi }(x),\mathsf{\Phi }(x^{\prime })\rangle$$

(6)

According to the above-mentioned equations, the optimization problem of (4) can be modified, and finally, the function $f$ derived as follows:

$$f(x)=b+{\displaystyle \sum _{l=1}^N({\alpha }_l-{\alpha }_l^{*})k(x_l,x)}$$

(7)

It is notable that in the nonlinear optimization problem, the flatness function is searched among the feature space, not input space [40,41].

4. Simulation and Results

The MLP and SVR networks are designed to predict the cooling and heating load. Each of these networks was trained using a dataset as input. In this paper, 85% (658 samples) of the data were used to train and validate the proposed methods and the remaining 15% (110 samples) were used for testing. In the first stage, each network required a preliminary design to determine the number of neurons in the hidden layer and the coefficients of the network. After designing each network, the amount of training and testing data for the network was determined. In this work, 70% of the samples were considered as training data and 30% as test data to validate the training phase of each network. After training and testing each neural network or regression algorithm, the results need to be evaluated. To do this, correlation coefficient (R), mean squared error (MSE), root mean squared error (RMSE), and mean absolute error (MAE) can be used as statistical performance metrics. Each of the mentioned indices are calculated according to the following equations [42].

$$R=\frac{{\displaystyle \sum _{l=1}^N(x_l-\overline{x})(y_l-\overline{y})}}{\sqrt{{\displaystyle \sum _{l=1}^N{(x_l-\overline{x})}^2}{\displaystyle \sum _{l=1}^N{(y_l-\overline{y})}^2}}}$$

(8)

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum _{l=1}^N{(x_l-y_l)}^2}}$$

(9)

$$MSE=\frac{1}{N}{\displaystyle \sum _{l=1}^N{(x_l-y_l)}^2}$$

(10)

$$MAE=\frac{1}{N}{\displaystyle \sum _{l=1}^N\left|x_l-y_l\right|}$$

(11)

where $x_l$ and $y_l$ illustrate the actual value and predicted value. $\overline{x}$ and $\overline{y}$ $\overline{y}$ depict the mean of actual values and forecasted values, respectively. Figure 3 shows the good correlation coefficient between the real values and the predicted value by the network in the training, testing and validation steps for the MLP network. Figure 4 indicates an excellent correlation coefficient between the real values and the predicted value for the SVR network during the training phase.

Given the excellent correlation between the target data and the output of each network (as shown in Figure 3 and Figure 4), it can be clearly understood that each of these networks have passed the training phase well. Good training means that the network is able to identify inherent patterns in the nature of data and to predict the unknown data by using the learned patterns, so that each network learns how much of a cooling and heating load is required for each building with specific characteristics. With this training, each network can predict the amount of cooling and heating loads related to the input data of the test phase. After training, each network is validated using initial test data (30% of 85%). This is kind of a test for the training phase, which is done by the network itself. The prediction error in the test or validation, which is one of the most important values in evaluating the results, is presented in Figure 5 and Figure 6 for each of the MLP and SVR networks in the histogram form, respectively.

The error obtained in the error histogram model indicates the minimum and maximum prediction error. This means that in predicting the cooling and heating loads for the test data, the amount of error that each of the trained networks can have is equal to the amount provided in the above figures.

In evaluating and analyzing each of the above figures that somehow represent the performance of each network in the initial training and testing stages, it can be concluded that the training of proposed methods has been well validated using the desired data. It should be noted that when the network is trained with high accuracy, it is well designed and the amount of error in the validation and initial testing process depends more on the type of data. It also implies that the network will be able to accurately assess and predict new and unknown data. Each network is saved as a black box after training. This black box contains patterns that the network was able to identify during the training phase. Now, the new and unknown data must be used to test these networks and predict the cooling and heating load of buildings. To do this, 15% (110 samples) of the data, kept as unknown and new data, were used. Figure 7 and Figure 8 show the results of forecasting heating and cooling loads for new data using the trained MLP and SVR networks, respectively.

Performance evaluation of the proposed methods is presented in

Table 2. Results of accuracy and error for proposed methods in heating and cooling load prediction.

	Heating Load				Cooling Load
	R	MSE	RMSE	MAE	R	MSE	RMSE	MAE
MLP	0.9993	0.2335	0.4832	0.4118	0.9824	6.896	2.626	2.0973
SVR	0.9979	0.7838	0.8853	0.7780	0.9878	3.024	1.7389	1.4762

in terms of R, MSE, RMSE, and MAE.

Based on the results presented in the Table 2, it can be seen that the best prediction was related to the prediction of the heating load by the MLP method with the highest value of R (0.9993) and minimal errors in the form of MSE (0.2335), RMSE (0.4832), and MAE (0.4118). However, in predicting the cooling load, the SVR method with a large amount of R (0.9878) and lowest errors in the terms of MSE (3.024), RMSE (1.7389), and MAE (1.4762) made a good prediction. Highest values of MSE and RMSE errors of prediction were also related to the MLP method in the prediction of cooling load. The use of machine learning methods and their results are highly dependent on the type of input data. It is observed that there is a difference between the results of predicting the cooling load and heating load by each of the networks and the heating load is predicted with high accuracy. This difference is due to the poor correlation between the input data and the amount of cooling load relative to the heating load. To evaluate the effectiveness of the proposed methods in this paper, it is necessary to compare the results obtained with the results of other studies. Comparisons should be made with caution using similar datasets. To this end, a number of studies were selected for comparison in which similar data was used for predicting the cooling and heating loads. To express the effectiveness of the data type in the accuracy of the results, the results of several studies conducted to predict cooling and heating loads using different data were compared with the results obtained in this paper.

Table 3. Comparison of cooling and heating loads prediction results with other works.

Data Type	References	Heating Load (R)	Cooling Load (R)
Used data in this paper	MLP in this paper	0.9993	0.9824
	SVR in this paper	0.9979	0.9878
	DNN [14]	0.9805	0.9976
	GBM [14]	0.9853	0.9853
	GPR [14]	0.9984	0.9913
	MPMR [14]	0.8802	0.8955
	ANN [15]	0.9980	0.9840
	CART [15]	0.9960	0.9810
	GLR [15]	0.9950	0.9830
	CHAID [15]	0.9950	0.9810
	GA-ANN [18]	0.9800	-
	PSO-ANN [18]	0.9720	-
	ICA-ANN [18]	0.9700	-
	ABC-ANN [18]	0.9730	-
Different data	GRNN [28]	-	0.9640
	PENN [20]	-	0.9500
	MLR [20]	-	0.7510
	AR [20]	-	0.8370
	ARX [20]	-	08640
	MNR (initial prediction) [20]	-	0.8990
	MNR (final calibration) [20]	-	0.9580
	ANN [21]	0.9900	-
	Decision tree [22]	0.92	-

makes this comparison.

The comparison made in Table 3, shows the accuracy and efficiency of the proposed methods in this paper for forecasting the cooling and heating loads of the building. The use of machine learning applications and the selection of the applicable method for energy predicting and energy efficiency measures in residential buildings are significantly effective in saving energy consumption. The selected methods were able to realize the purpose of the paper with their high accuracy and achieve this important goal. Finally, it should be noted that the proposed solutions can also be used for real-world data.

5. Conclusions

Nowadays, the importance of energy saving and its management has raised many challenges in forecasting the heating and cooling loads of buildings. Most researchers in this field offer many methods and models for predicting heating and cooling loads to somehow increase the prediction accuracy. In this paper, based on machine learning models two MLP and SVR methods were proposed to predict the cooling and heating load of a residential building. The main idea of these methods was to create a linear mapping between the input and output variables to increase the prediction accuracy. After designing each of the proposed models, the technical parameters of a home building were used as inputs and the heating and cooling loads were used as the output variables of each network during the training phase. New and anonymous data were used to test the trained networks and for forecasting the heating and cooling loads. Finally, each trained network was able to reliably provide the heating and cooling load forecasts. Meanwhile, the MLP method with the maximal of R i.e., 0.9993 and the SVR method with the highest value of R i.e., 0.9878 predicted the heating and cooling loads, respectively.