Multi-Objective Plum Tree Algorithm and Machine Learning for Heating and Cooling Load Prediction

Abstract

The prediction of heating and cooling loads using machine learning algorithms has been considered frequently in the research literature. However, many of the studies considered the default values of the hyperparameters. This manuscript addresses both the selection of the best regressor and the tuning of the hyperparameter values using a novel nature-inspired algorithm, namely, the Multi-Objective Plum Tree Algorithm. The two objectives that were optimized were the averages of the heating and cooling predictions. The three algorithms that were compared were the Extra Trees Regressor, the Gradient Boosting Regressor, and the Random Forest Regressor of the sklearn machine learning Python library. We considered five hyperparameters which were configurable for each of the three regressors. The solutions were ranked using the MOORA method. The Multi-Objective Plum Tree Algorithm returned a root mean square error value for heating equal to 0.035719 and a root mean square error for cooling equal to 0.076197. The results are comparable to the ones obtained using standard multi-objective algorithms such as the Multi-Objective Grey Wolf Optimizer, Multi-Objective Particle Swarm Optimization, and NSGA-II. The results are also performant concerning the previous studies, which considered the same experimental dataset.

1. Introduction

Energy conservation and emission reduction have recently received a lot of attention in the context of the increase in energy consumption [1]. The prediction of heating and cooling loads could lead to a rational use of renewable energy to replace thermal energy systems which are based on fossil fuel. Moreover, reducing heating and cooling loads can help to decarbonize the building sector [2]. Therefore, the application of machine learning techniques has become important in the context of heating and cooling load prediction. Even though many living buildings are usually equipped with electricity metrics which provide almost complete statistics, the data are often not enough to develop complex machine learning models for the prediction of heating and cooling loads. Information about characteristics which are particular to each building, such as the wall area, the surface area, and the roof area, can complement the metrics data to develop algorithms which are more accurate and which better address the particularities of buildings.

Buildings are responsible for approximately 40% of total global energy consumption [3]. The improvement of energy efficiency and the conservation of more energy has become essential in recent years [4,5] due to the adverse effects of high-energy consumption on the environment. The estimation of heating and cooling loads depends on the characteristics of the structure. To construct energy-efficient buildings, it is helpful to develop conceptual systems that anticipate the cooling load in the residential building sector [6].

Since energy resources present limitations and have an important role in the economic development of countries, reduction in energy consumption represents a necessity [7,8]. The modeling of heating and cooling loads represents the cornerstone of energy-efficient building design. Furthermore, as stated in [9], energy efficiency leads to both environmental and financial benefits [10]. Moreover, energy efficiency directly impacts economic competitiveness and sustainable development [11]. These facts underscore the significance of the work presented in the current manuscript.

The accurate prediction of energy consumption and the determination of the factors that influence heating energy consumption are important [12] in the context of the substantial increase in energy consumption in the case of residential buildings [13]. Therefore, the use of advanced machine learning algorithms for energy consumption prediction presents great interest for researchers. As can be seen in [14], methods based on machine learning for the prediction of energy consumption have advanced significantly in recent years. The development and examination of machine learning algorithms which can learn from patterns in the data and make predictions presented a lot of interest for many scholars and scientists [15].

Several statistics, such as the ones presented by the authors of [16], estimate that by 2040, there will be an increase of nearly 25% in global energy demand. Also, to create a sustainable and healthy economy, it is important to measure the economic and environmental effects of energy production [17].

The optimization of building energy prediction represents an important research area because of its potential to improve the efficiency of energy management systems [18]. Many studies have shown that the air conditioning system consumes up to 38% of the total energy in the building sector [19,20].

As can be seen, the energy efficiency research domain will present great interest to the research community in the years to come, and the application of novel machine learning and artificial intelligence techniques can lead to a significant improvement in the existing techniques used for energy consumption prediction.

The solution presented in the manuscript is specific to the heating and cooling load prediction optimization problem, as it aims to improve the prediction results for two objectives, namely the heating load and the cooling load. Two particularities of the proposed solution were the consideration of three machine learning algorithms which had common hyperparameters and returned good predictions for energy data characterized by a small number of features and the development of an objective function which considered a 10-fold cross-validation and averaged the heating and cooling prediction results.

The main contributions of the work presented in this paper are as follows:

(1)

A critical review of the application of machine learning methods for the prediction of heating and cooling loads;

(2)

The introduction of a novel algorithm called the Multi-Objective Plum Tree Algorithm (MOPTA) which adapts the original Plum Tree Algorithm [21] to multi-objective optimization problems;

(3)

The ranking of the solutions using the MOORA method [22];

(4)

The adaptation of the MOPTA to the hyperparameter optimization and the optimal regressor selection for a machine learning methodology used to predict heating and cooling loads, using the Energy Efficiency Dataset of the UCI Machine Learning Repository as experimental support [23,24];

(5)

The development of an objective function that considers the averages of the heating and cooling RMSE results;

(6)

The comparison and validation of the obtained results with the ones obtained by the Multi-Objective Grey Wolf Optimizer (MOGWO) [25], Multi-Objective Particle Swarm Optimization (MOPSO) [26], and NSGA-II [27].

The manuscript is structured as follows: Section 2 presents the research background, Section 3 presents the MOPTA-based machine learning methodology for the optimization of heating and cooling load prediction, Section 4 presents the results, Section 5 compares the obtained results with the ones from previous studies, and Section 6 shows the conclusions.

2. Research Background

This section reviews representative recent studies that considered the application of machine learning for the prediction of heating and cooling loads.

In [28], the authors approached the prediction of the energy efficiency of buildings using machine learning techniques considering a small data approach. The method proposed by them considered the Support Vector Regression and the K-means algorithms. The prediction of heating and cooling parameters was considered as two separate tasks. The dataset was split at 75%:25% and the metrics used to evaluate the performance were mean square error and mean absolute error. Their proposed method was better in terms of mean square error and mean absolute error than other methods, such as the classical Support Vector Regression with rbf kernel.

The review from [29] presented a selection of representative studies that used data-driven techniques, such as machine learning and artificial intelligence, for the prediction of the cooling and heating loads of residential buildings. The review considered various techniques, such as ensemble learning, Artificial Neural Network, Support Vector Machines, probabilistic models, and statistical models. As support for their experiments, the review also considered recent studies that used the same experimental dataset as the one used in our manuscript. For example, the approach presented in [30] used an ensemble machine learning model based on three Random Forest models that achieved 0.999 $R^2$ for the heating load prediction and 0.997 $R^2$ for the cooling load prediction using a 10-fold cross-validation approach. On the other hand, the authors of [31] used an approach based on the Tri-Layered Neural Network and Maximum Relevance Minimum Redundancy that led to 0.289 mean absolute error for heating load and 0.535 mean absolute error for cooling load, respectively.

The approach presented in [32] considered a novel method for energy consumption estimation using Support Vector Machine and Random Forest. The Owl Search Algorithm [33] was used to improve the performance of these two algorithms. The root mean square error values returned by the approaches based on Support Vector Machine and Random Forest were 0.85 and 1.29 for heating and 1.02 and 1.65 for cooling, respectively.

The approach presented in [34] compared four algorithms, namely, the Linear Regression, the Decision Tree, the Random Forest, and the XGBoost. Like our approach, the data were split randomly into 80% training data and 20% testing data. The hyperparameters were optimized using Bayesian Optimization [35]. The best results in terms of root mean square error for the testing data were obtained by the XGBoost algorithm, as follows: 0.3797 for the heating load and 0.7578 for the cooling load.

The mean square error results obtained by the authors of [36] were 0.201 for the heating load prediction and 2.56 for the cooling load forecast. Another approach—for example, the one presented in [37], which used the Multilayer Perceptron and Support Vector Regression algorithms—returned 0.4832 and 0.8853 root mean square error values for the heating load prediction and 2.626 and 1.7389 root mean square error values for the cooling load prediction. On the other hand, the approach presented in [38] based on the Gated Recurrent Unit returned the 0.0166 and 0.0247 root mean square error values for the heating and cooling load predictions, respectively, when hold-out was used, and 0.01 root mean square error values for both heating and cooling load predictions when 10-fold validation was used.

The authors of [39] considered an approach based on a Multi-Objective Optimization method for the tuning of the hyperparameters of a Random Forest model used for the prediction of heating and the cooling loads. The two objectives that were optimized were the averages of the heating and cooling load prediction. Compared to their approach, our method also predicts which regressor to use as part of the multi-objective optimization process.

3. Multi-Objective Plum Tree Algorithm (MOPTA) Machine Learning Methodology for Heating and Cooling Load Prediction

The original version of the Plum Tree Algorithm was introduced in [21] with the following sources of inspiration:

• The plum trees flowering at the beginning of spring;
• The transformation of the flowers, which are pollinated into plums;
• The dropping of a percentage of the plums before maturity due to various reasons;
• The continuity of the lives of the plums after the harvest for a couple of weeks.

The PTA presents similarities with other bio-inspired algorithms, such as Chicken Swarm Optimization [40], Particle Swarm Optimization, Grey Wolf Optimizer [41], and Crow Search Algorithm [42], which influenced how particular mathematical parts of the algorithm were modeled.

Table 1. PTA configurable parameters.

Parameter	Description
$I$	the total number of iterations
$D$	the number of dimensions of the search space
$N$	the number of plums
$FT$	the fruitiness threshold
$RT$	the ripeness threshold
$\left[{FR}_{min},{FR}_{max}\right]$	the minimum and the maximum values of the fruitiness rate
$\epsilon$	a constant used for avoiding division by 0
$\left[X_{min},X_{max}\right]$	the minimum and the maximum possible values of the positions
$OF$	the objective function used to evaluate the plums

summarizes the PTA’s configurable parameters.

The PTA starts with the initialization of $N$ flowers in a search space with $D$ dimensions, such that the values are selected randomly from the range $\left[X_{min},X_{max}\right]$:

$$flowers=\left[\begin{array}{ccc}{flower}_{\mathrm{1,1}}^0& \dots & {flower}_{1,D}^0\\ \dots & \dots & \dots \\ {flower}_{N,1}^0& \dots & {flower}_{N,D}^0\end{array}\right].$$

(1)

Then, $N$ plums are initialized with the value of the flowers:

$$plums=\left[\begin{array}{ccc}{plum}_{\mathrm{1,1}}^0& \dots & {plum}_{1,D}^0\\ \dots & \dots & \dots \\ {plum}_{N,1}^0& \dots & {plum}_{N,D}^0\end{array}\right]=flowers.$$

(2)

The $OF$ is used to calculate the fitness values of the flowers and of the plums. The ${plum}_{gbest}$ is set to the position of the plum that has the best fitness value.

Then, the PTA runs the following instructions $I$ times.

At the beginning of each iteration, the positions of the following two plums are computed:

• ${plum}_{ripe}$—the plum with the best fitness value
• ${plum}_{unripe}$—the plum with the second-best fitness value

For each ${flower}_i^k$, where $k$ is the iteration number and $i=\overline{1,N}$, a random number $r$ from the range $[0, 1]$ is selected. Three cases, one for each phase, are considered further:

• Fruitiness Phase (r ≥ FT):

In this case, the positions of the flowers are updated using the following formula:

$${flower}_i^k={flower}_i^{k-1}+rand\left({FR}_{min},{FR}_{max}\right)\times \left({plum}_i^{k-1}-{flower}_i^{k-1}\right)$$

(3)

where $rand\left({FR}_{min},{FR}_{max}\right)$ is a random number from $\left[{FR}_{min},{FR}_{max}\right]$.

• Ripeness Phase (FT > r ≥ RT):

The following formula is used to update the positions of the flowers in this case:

$${flower}_i^k={flower}_i^{k-1}+2\times r_1\times \left({plum}_{ripe}-{flower}_i^{k-1}\right)+2\times r_2\times \left({plum}_{unripe}-{flower}_i^{k-1}\right)$$

(4)

where $r_1$ and $r_2$ are random numbers from $\left[0, 1\right]$.

• Storeness Phase (RT > r):

The positions of the flowers are updated as follows:

$${flower}_i^k={plum}_i^{k-1}\times \left(1+\mathcal{N}\left(0,{\sigma }^2\right)\right)$$

(5)

where $\mathcal{N}\left(0,{\sigma }^2\right)$ is a Gaussian distribution with mean $0$ and standard deviation ${\sigma }^2,$ defined as follows:

$${\sigma }^2=\left\{\begin{array}{c}\begin{array}{cc}1,& \mathrm{i}\mathrm{f} OF\left({plum}_i^{k-1}\right)<OF\left({plum}_{ripe}\right)\end{array}\\ \begin{array}{cc}{e}^{\frac{OF\left({plum}_{ripe}\right)-OF\left({plum}_i^{k-1}\right)}{\left|OF\left({plum}_i^{k-1}\right)\right|+\epsilon }},& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\end{array}\right.$$

(6)

Then, the positions of the flowers are updated to be in $\left[X_{min},X_{max}\right]$. For each $j=\overline{1,D}:$

• If ${flower}_{i,j}^k<X_{min}$ then ${flower}_{i,j}^k=X_{min}$;
• If ${flower}_{i,j}^k>X_{max}$ then ${flower}_{i,j}^k=X_{max}$.

After the positions of the flowers are updated, for each ${plum}_i^k$, where $k$ is the iteration number and $i=\overline{1,N}$, the following formula is used:

$${plum}_i^k=\left\{\begin{array}{c}\begin{array}{cc}{flower}_i^k,& \mathrm{i}\mathrm{f} OF\left({flower}_i^k\right)<OF\left({plum}_i^{k-1}\right)\end{array}\\ \begin{array}{cc}{plum}_i^{k-1},& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\end{array}\right.$$

(7)

At the end of each iteration, the position of ${plum}_{gbest}$ is updated to the position of the plum with the best fitness.

Finally, when all iterations are completed, the PTA returns the value of the ${plum}_{gbest}$.

3.1. Heating and Cooling Load Prediction

The Energy Efficiency Dataset was split randomly using a 5-fold cross-validation. For each split out of the five splits, the testing data were represented by one different fold, while the training data were represented by the remaining folds. The training data were standardized using the Z-score for the values of each column, while the testing data were standardized using the mean and the standard deviation values, which were computed for the training data.

The algorithms that were used were the Extra Trees Regressor (ETR), the Gradient Boosting Regressor (GBR), and the Random Forest Regressor (RFR). The regressors were configured with the hyperparameter values described by the plums. The eight metrics which were used to evaluate the results were the averages of the $RMSE$, $R^2$, $MAE$, and $MAPE$ across the 5 folds for heating and cooling load prediction.

3.2. MOPTA Multi-Objective Fitness Function

Each position of a plum corresponds to an algorithm and its hyperparameter configuration. To apply the multi-objective fitness function to a plum, it is necessary to convert the values that describe the position of the plum to integers first. This is done using the floor function, which takes a real value as input and returns the greatest integer value that is less than or equal to it.

Figure 1 illustrates the high-level view of the multi-objective fitness function.

Compared to the approach presented in [39], we did not consider the bootstrap parameter, as we aimed to use a set of hyperparameters that can be configured for all algorithms. However, compared to that approach, we added a new dimension that describes the algorithm, namely GBR, RFR, or ETR, such that 0 corresponds to GBR, 1 to RFR, and 2 to ETR, respectively.

The inputs of the fitness function are the converted position of the plum and the train data. The first dimension describes the algorithm, while the other five dimensions describe the values of the hyperparameters.

We performed a 10-fold cross-validation on the train data, and we computed the average RMSE values. In each partition, the test data are represented by one fold, and the train data are represented by the other nine folds. The configured selected algorithm was applied twice in each partition, depending on the prediction type. The first time, it was used to predict the heating load, while the second time, it was used to predict the cooling load.

The output of the fitness function is represented by the following two values:

• The average Heating RMSE, denoted as ${RMSE}_H$;
• The average Cooling RMSE, denoted as ${RMSE}_C$.

The MOPTA aims to obtain minimal values for both values.

3.3. Multi-Objective Adaptations of PTA

The multi-objective adaptations of the PTA considered in this manuscript are similar to the ones we used in [25], and they are based on the method presented in [43]. The major adaptations introduced by the MOPTA are the application of an external archive for the saving and retrieval of the solutions that are pareto-optimal and the use of this archive for obtaining the values for the ripe and the unripe plums.

The dominance relations between ${Plum}_1=\left({RMSE}_{H_1},{RMSE}_{C_1}\right)$ and ${Plum}_2=\left({RMSE}_{H_2},{RMSE}_{C_2}\right)$ are defined as follows:

(1)

If ${RMSE}_{H_1}\le {RMSE}_{H_2}$ and ${RMSE}_{C_1}\le {RMSE}_{C_2},$ and at least one of the relations ${RMSE}_{H_1}<{RMSE}_{H_2}$ and ${RMSE}_{C_1}<{RMSE}_{C_2}$ is true, then ${Plum}_1$ dominates ${Plum}_2$;

(2)

If ${RMSE}_{H_1}\ge {RMSE}_{H_2}$ and ${RMSE}_{C_1}\ge {RMSE}_{C_2},$ and at least one of the relations ${RMSE}_{H_1}>{RMSE}_{H_2}$ and ${RMSE}_{C_1}>{RMSE}_{C_2}$ is true, then ${Plum}_2$ dominates ${Plum}_1$;

(3)

If neither (1) nor (2) is true, then ${Plum}_1$ and ${Plum}_2$ are non-dominated.

A set that contains two solutions is non-dominated if neither solution dominates the other one.

3.3.1. Plum Matrix Grid Computation

Figure 2 presents the methodology for the computation of the plum matrix grid.

The input is represented by a set of plums $\left\{{plum}_1,\dots ,{plum}_{N_{plums}}\right\}$, where $N_{plums}$ is the total number of plums.

• step 1: The costs of the plums are computed using the multi-objective fitness function. The result is the following matrix:

  $$Cost=\left[\begin{array}{cc}{RMSE}_{H_1}& {RMSE}_{C_1}\\ \dots & \dots \\ {RMSE}_{H_{N_{plums}}}& {RMSE}_{C_{N_{plums}}}\end{array}\right]$$

  (8)
• step 2: The minimum and the maximum cost values of ${RMSE}_H$ are computed using the $Cost$ matrix and the grid’s inflation parameter ($ϵ$):

  $$\left[\begin{array}{c}{min}_{{RMSE}_H}\\ {max}_{{RMSE}_H}\end{array}\right]=\left[\begin{array}{c}min\left({RMSE}_{H_1},\dots ,{RMSE}_{H_{N_{plums}}}\right)\\ max\left({RMSE}_{H_1},\dots ,{RMSE}_{H_{N_{plums}}}\right)\end{array}\right]$$

  (9)

  $$∆{RMSE}_H=ϵ\times \left({max}_{{RMSE}_H}-{min}_{{RMSE}_H}\right)$$

  (10)

  $$\left[\begin{array}{c}{RMSE}_{H_{min}}\\ {RMSE}_{H_{max}}\end{array}\right]=\left[\begin{array}{c}{min}_{{RMSE}_H}-∆{RMSE}_H\\ {max}_{{RMSE}_H}+∆{RMSE}_H\end{array}\right]$$

  (11)
• step 3: Similarly, the minimum and the maximum cost values of ${RMSE}_C$ are computed as follows:

  $$\left[\begin{array}{c}{min}_{{RMSE}_C}\\ {max}_{{RMSE}_C}\end{array}\right]=\left[\begin{array}{c}min\left({RMSE}_{C_1},\dots ,{RMSE}_{C_{N_{plums}}}\right)\\ max\left({RMSE}_{C_1},\dots ,{RMSE}_{C_{N_{plums}}}\right)\end{array}\right]$$

  (12)

  $$∆{RMSE}_C=ϵ\times \left({max}_{{RMSE}_C}-{min}_{{RMSE}_C}\right)$$

  (13)

  $$\left[\begin{array}{c}{RMSE}_{C_{min}}\\ {RMSE}_{C_{max}}\end{array}\right]=\left[\begin{array}{c}{min}_{{RMSE}_C}-∆{RMSE}_C\\ {max}_{{RMSE}_C}+∆{RMSE}_C\end{array}\right]$$

  (14)
• step 4: The plum matrix $Grid$ was defined using the number of grids $ng,$ such that the $x$-axis presents the endpoints of the ${RMSE}_H$ minimization objective, and the $y$-axis presents the endpoints of the ${RMSE}_C$ minimization objective.
• step 5: The formulas that are used for the computation of the index of ${plum}_i,$ with the cost $\left({RMSE}_{H_i},{RMSE}_{C_i}\right)$ such that $i=\overline{1,N_{plums}},$ are as follows:

  $${index}_H=\min \left(i|{RMSE}_{H_i}<{Grid}_{i,1}\right)$$

  (15)

  $${index}_C=\mathit{min}\left(i|{RMSE}_{C_i}<{Grid}_{i,1}\right)$$

  (16)

  $$index\left({plum}_i\right)={index}_H\left({plum}_i\right)+ng\times \left({index}_C\left({plum}_i\right)-1\right)$$

  (17)

The output is represented by the plum indices set $\left\{index\left({plum}_1\right),\dots ,index\left({plum}_{N_{plums}}\right)\right\}$.

3.3.2. Plum Selection Methodology

The ripe and unripe plums were selected using the archive of plums ${archive}_{plums}$. The selection of these two plums was performed at each iteration of the MOPTA for each plum.

Figure 3 presents a high-level view of the methodology for the plum selection.

The set $OccIndex$ of occupied indices for ${archive}_{plums}$ is calculated as follows:

$$OccIndex\left({archive}_{plums}\right)=Set\left(index\left({plum}_1\right),\dots ,index\left({plum}_{as}\right)\right)$$

(18)

such that the function $Set$ converts the numbers received as input into a list of unique numbers sorted in increasing order. Suppose that there are $M$ cells that are occupied, and each one of them is defined by the cell index $cindex$. Then:

$$OccIndex\left({archive}_{plums}\right)=\left\{{cindex}_1,\dots ,{cindex}_M\right\}$$

(19)

The following vector that stores the cell count of each plum is defined:

$$OccCnt\left({archive}_{plums}\right)=\left\{{ccnt}_1,\dots ,{ccnt}_M\right\}$$

(20)

such that for each $i=\overline{1,M}$, the value ${ccnt}_i$ represents how many plums are present at the location ${cindex}_i$.

Then, a random number $r$ is selected from $\left\{1,\dots ,M\right\}$ using a roulette wheel selection mechanism defined by the following:

$$P_{plum}=\left[\begin{array}{c}\frac{{ccnt}_1^{-\pi }}{{\sum }_{i=1}^M{ccnt}_i^{-\pi }}\\ \dots \\ \frac{{ccnt}_M^{-\pi }}{{\sum }_{i=1}^M{ccnt}_i^{-\pi }}\end{array}\right]$$

(21)

The set $Selected\left({archive}_{plums}\right)$ was defined by the formula:

$$Selected\left({archive}_{plums}\right)=\left\{{plum}_i|index\left({plum}_i\right)={cindex}_r\right\}$$

(22)

such that $i=\overline{1,as}$, where $as$ is the archive size, was used to select the ripe or the unripe plum, randomly considering the uniform probability.

3.3.3. Plum Removal Methodology

The size of ${archive}_{plums}$ was adjusted during each iteration of the algorithm if it was greater than the maximum archive size ($mas$). Figure 4 presents a high-level view of the methodology for the removal of the plums.

Therefore, a number of $\left(as-mas\right)$ plums were removed from the archive using the steps similar to the ones presented in the plum selection methodology, with the following two adaptations:

• Adaptation 1: the plums removal probability, which was used instead of the $P_{plum}$ probability, was defined using the formula:

  $$P_{removal}=\left[\begin{array}{c}\frac{{ccnt}_1^{-\zeta }}{{\sum }_{i=1}^M{ccnt}_i^{-\zeta }}\\ \dots \\ \frac{{ccnt}_M^{-\zeta }}{{\sum }_{i=1}^M{ccnt}_i^{-\zeta }}\end{array}\right]$$

  (23)

  such that the parameter $\zeta$ describes the plum selection pressure.
• Adaptation 2: the removal of the selected plum from ${archive}_{plums}$, as the last step of the methodology.

3.3.4. MOPTA for Heating and Cooling Prediction

Algorithm 1 presents the MOPTA for heating and cooling prediction.

Algorithm 1 MOPTA for Heating and Cooling Prediction
1:	$\mathbf{Input} I, D, N, FT, RT, {FR}_{min}, {FR}_{max}, \epsilon , X_{min}, X_{max},OF,mas,ng,ϵ,\pi ,\zeta$
2:	$\mathbf{Output} {archive}_{plums}$
3:	$\mathrm{initialize} N$$\mathrm{flowers} \mathrm{in} \mathrm{the} D$$‐\mathrm{dimensional} \mathrm{space} \mathrm{with} \mathrm{values} \mathrm{from} \left[X_{min}, X_{max}\right]$;
4:	$\mathrm{initialize} N$$\mathrm{plums} \mathrm{to} \mathrm{the} \mathrm{positions} \mathrm{of} \mathrm{the} N$ flowers;
5:	$\mathrm{adapt} \mathrm{the} \mathrm{positions} {plum}_i$$\mathrm{and} {flower}_i \left(i=1,\dots ,N\right)$ to arrays of integers;
6:	$\left({RMSE}_{H_i},{RMSE}_{C_i}\right)=OF\left({plum}_i\right) \left(i=1,\dots ,N\right)$;
7:	$plums=GetDomination\left(plums\right);$
8:	${archive}_{plums}=GetArchive\left(plums\right);$
9:	$grids=GetGrids\left({archive}_{plums},ng,ϵ\right);$
10:	$\mathbf{for} iter=1$$\mathrm{to} I$ do
11:	$\mathbf{for} i=1$$\mathrm{to} N$ do
12:	$as=SizeOf\left({archive}_{plums}\right)$;
13:	$\mathrm{determine} {plum}_{ripe}$$\mathrm{and} {plum}_{unripe}$$\mathrm{from} {archive}_{plums}$$\mathrm{using} as$$, \pi$$, grids$;
14:	$\mathrm{update} r$$\mathrm{to} \mathrm{a} \mathrm{random} \mathrm{number} \mathrm{from} [0, 1]$;
15:	$\mathrm{update} {flower}_i$$\mathrm{according} \mathrm{to} FT$$, RT$$, {FR}_{min}, {FR}_{max}, r$;
16:	$\mathrm{adjust} \mathrm{the} \mathrm{flowers} \mathrm{to} \mathrm{be} \mathrm{in} \mathrm{the} \mathrm{range} \left[X_{min}, X_{max}\right]$;
17:	end for
18:	$\mathbf{for} i=1$$\mathrm{to} N$ do
19:	$\mathrm{update} {plum}_i$$\mathrm{and} {flower}_i$ to arrays of integers;
20:	$\mathrm{use} OF$$\mathrm{to} \mathrm{get} \mathrm{the} \mathrm{fitness} \mathrm{values} {Plum}_i$$\mathrm{and} {Flower}_i$$\mathrm{of} {plum}_i$$\mathrm{and} {flower}_i$;
21:	$\mathbf{if} dominates({Flower}_i,{Plum}_i)$ then
22:	${plum}_i={flower}_i$;
23:	${Plum}_i={Flower}_i$;
24:	end if
25:	end for
26:	$plums=GetDomination\left(plums\right);$
27:	$nplums=NonDominated\left(plums\right);$
28:	$nplums={archive}_{plums}\cup nplums$;
29:	$nplums=GetDomination\left(nplums\right);$
30:	${archive}_{plums}=GetArchive\left(nplums\right);$
31:	$grids=GetGrids\left({archive}_{plums},ng,ϵ\right);$
32:	$as=SizeOf\left({archive}_{plums}\right);$
33:	$ResizeArchive\left(as>mas,as,mas,{archive}_{plums},\zeta \right);$
34:	end for
35:	$\mathbf{return} {archive}_{plums}$;

The input parameters of the MOPTA consist of the standard input parameters of the PTA, which are presented in Table 1, and the following additional parameters:

• $mas$—the maximum archive size of the repository that contains the nondominated solutions;
• $ng$—the number of grids per objective;
• $ϵ$—the grid’s inflation parameter;
• $\pi$—the pressure parameter used during the plum selection;
• $\zeta$—the plum selection pressure parameter used during the plum removal.

The $OF$ is a multi-objective function that returns the average values of the ${RMSE}_H$ and ${RMSE}_C$ of the ML algorithm trained and validated according to the position of the plum. The range $\left[X_{min},X_{max}\right]$ was adapted such that the first dimension describes the selected regressor, while the other dimensions describe the limits of the hyperparameters considered in the training of the ML algorithm.

The output of the MOPTA is ${archive}_{plums}$, which consists of the non-dominant plums after $I$ iterations.

The $N$ flowers were initialized with random values from $\left[X_{min},X_{max}\right]$ in the $D$-dimensional search space (line 3), while the $N$ plums were initialized to the positions of the $N$ flowers in line 4. Then, both the positions of the plums and of the flowers were adapted to arrays of integers in line 5 using the floor function. The multi-objective fitness function OF presented in Section 3.2 was applied to each ${plum}_i (i=1,\dots ,N)$ (line 6).

Using the conditions presented at the beginning of Section 3.3, the dominance relation was determined in line 7 of the algorithm. The ${archive}_{plums}$ of non-dominated plums was created in line 8. Then, using the steps presented in Section 3.3.1, the grid matrix and the indices of the plums were computed.

The instructions from lines 11–33 were repeated for $I$ iterations. For each ${flower}_i$ $\left(i=1,\dots ,N\right)$, the instructions from lines 12–16 were performed. The current size of the archive $as$ was updated in line 12 to the total number of plums from ${archive}_{plums}$. The values of ${plum}_{ripe}$ and ${plum}_{unripe}$ were computed in line 13 using the values of $as$, $\pi$, $and grids$ and the plum selection methodology presented in Section 3.3.2.

Initially, the ripe and the unripe plums were selected randomly from ${archive}_{plums}$. If $as>1$, then ${plum}_{ripe}$ was selected from ${archive}_{plums}-\left\{{plum}_{unripe}\right\}$ following the steps from Section 3.3.2.

The positions ${flower}_i$ $\left(i=1,\dots ,N\right)$ were updated in line 15 using Equations (3)–(6) for the three phases: fruitiness phase, ripeness phase, and storeness phase. The equations for the storeness phase were adapted for the multi-objective optimization using a procedure adapted after the one from [44]. Equation (6) was adapted to the equation:

$${\sigma }^2=\left\{\begin{array}{c}\begin{array}{cc}1,& \mathrm{i}\mathrm{f} OF\left({plum}_i^{k-1}\right)<OF\left({plum}_{ripe}\right)\end{array}\\ \begin{array}{cc}{e}^{\frac{F\left({plum}_{ripe}\right)-F\left({plum}_i^{k-1}\right)}{\left|F\left({plum}_i^{k-1}\right)\right|+\epsilon }},& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\end{array}\right.$$

(24)

such that the function $F$ was defined as follows:

$$F\left(plum\right)=\frac{{RMSE}_H+{RMSE}_C}{2}$$

(25)

where $\left({RMSE}_H,{RMSE}_C\right)=OF\left(plum\right)$.

Then, the positions of the flowers were updated to be in $\left[X_{min}, X_{max}\right]$ (line 16).

The instructions from (lines 19–24) were performed for each ${plum}_i$ $\left(i=1,\dots ,N\right)$. First, the ${plum}_i$ and the corresponding ${flower}_i$ were updated to arrays of integers using the floor function (line 19). Then, the $OF$ was used to compute the fitness values ${Plum}_i$ and ${Flower}_i$ of ${plum}_i$ and ${flower}_i$, respectively (line 20). If ${Flower}_i$ dominated ${Plum}_i,$ then the position ${plum}_i$ and the fitness value ${Plum}_i$ were updated to ${flower}_i$ and ${Flower}_i$, respectively.

The plum dominance relation was determined again in line 26. The non-dominated plums $nplums$ were computed in line 27. Then, in line 28, the plums from ${archive}_{plums}$ were appended to $nplums$ (line 29) and the $nplums$ were updated in line 30.

The matrix $grids$ was computed in line 31, while the value $as$ was computed in line 32. If the value $as$ was greater than $mas$, then $(as-mas)$ plums were removed from the archive, according to the methodology presented in Section 3.3.3.

Finally, the MOPTA returned the ${archive}_{plums}$ as output in line 35.

3.3.5. Solution Ranking Using MOORA

The solutions which were returned by the MOPTA were ranked using an adaptation of MOORA [22,45].

The matrix $D$ was defined as follows:

$$D=\left[\begin{array}{cc}{RMSE}_{H_1}& {RMSE}_{C_1}\\ \dots & \dots \\ {RMSE}_{H_m}& {RMSE}_{C_m}\end{array}\right]$$

(26)

where $m$ is the size of the plums archive and $\left({RMSE}_{H_i},{RMSE}_{C_i}\right)$ represent the $RMSE$ values predicted by the model trained according to the position of the $i$-th plum, where $i=1,\dots ,m$.

The values of $D$ were normalized as follows:

$$\left[\begin{array}{c}{RMSE}_{H_j}^{*}\\ {RMSE}_{C_j}^{*}\end{array}\right]=\left[\begin{array}{c}\frac{{RMSE}_{H_j}}{\sqrt{\sum _{i=1}^m{RMSE}_{H_i}^2}}\\ \frac{{RMSE}_{C_j}}{\sqrt{\sum _{i=1}^m{RMSE}_{C_i}^2}}\end{array}\right] (j=1,\dots ,m)$$

(27)

The MOORA scores of the plums from the archive of size $m$ were finally computed as follows:

$$S_{MOORA}=\left[\begin{array}{c}{RMSE}_{H_1}^{*}+{RMSE}_{C_1}^{*}\\ \dots \\ {RMSE}_{H_m}^{*}+{RMSE}_{C_m}^{*}\end{array}\right]$$

(28)

The most dominant plum was the one with the lowest MOORA score.

3.3.6. MOPTA Methodology for Heating and Cooling Prediction

Figure 5 presents the high-level view of the MOPTA methodology which was used for the prediction of the heating and cooling loads.

The input of the methodology was represented by the Energy Efficiency Dataset. The data were split into Training Data and Testing Data considering a 5-Fold Cross-Validation approach, such that five splits were performed. Each time, one fold was used for testing and the remaining ones for training. Then, the Standardized Training Data and the Standardized Testing Data were obtained. The MOPTA was run using the Standardized Training Data as input, with a 10-fold cross-validation to evaluate the plums. The archive returned by the algorithm was evaluated using MOORA, and the plum with the best MOORA score was further considered to evaluate the predictions. The predictions were evaluated using the $MAPE$, $RMSE$, $R^2$, and $MAE$ metrics.

4. Results

The experiments were performed in Python version 3.12.3 using the sklearn library on a machine with the following properties:

• Processor: Intel Core i9-13900 K 3.00 GHz;
• RAM: 64 GB;
• Operating System: Windows 11 Pro;
• SDD: 1 TB.

All the computations were CPU-based.

4.1. Energy Efficiency Dataset

The Energy Efficiency Dataset used in the experiments was characterized by 768 samples, eight attributes, and two responses. The dataset was obtained considering 12 building shapes, simulated in Ecotect.

Table 2. Energy Efficiency Dataset features summary.

Feature	Variable	Minimum Value	Maximum Value	Type
$X1$	Relative Compactness	0.62	0.98	attribute
$X2$	Surface Area	514.5	808.5
$X3$	Wall Area	245	416.5
$X4$	Roof Area	110.25	220.5
$X5$	Overall Height	3.5	7
$X6$	Orientation	2	5
$X7$	Glazing Area	0	0.4
$X8$	Glazing Area Distribution	0	5
$Y1$	Heating Load	6.01	43.1	response
$Y2$	Cooling Load	10.9	48.3

presents the summary of the features.

The dataset was split randomly into five folds of an approximately equal size, such that the Testing Data were represented by one of the folds while the Training Data were represented by the other four folds.

4.2. Hyperparameters Configuration

Table 3. Hyperparameter ranges.

Hyperparameter	Minimum Value	Maximum Value
n_estimators	200	600
max_depth	10	100
min_samples_leaf	1	10
min_samples_split	2	10
max_features	1	8

presents the ranges of the hyperparameters used in the experiments. The values were inspired by the ones used by the authors of [39].

4.3. MOPTA Configuration Parameters

Table 4. MOPTA configuration parameters summary.

Configuration Parameter	Value
$I$ (iterations)	50
$D$ (number of dimensions)	6
$N$ (number of plums)	30
$FT$ (fruitiness threshold)	0.8
$RT$ (ripeness threshold)	0.2
${FR}_{min}$ (minimum fruitiness rate)	0.5
${FR}_{max}$ (maximum fruitiness rate)	1
$\epsilon$ (avoiding division by 0 constant)	${10}^{-300}$
$X_{min}$ (minimum position values)	[0, 200, 10, 1, 2, 1]
$X_{max}$ (maximum position values)	[3, 601, 101, 11, 11, 9]
$OF$ (objective function)	the multi-objective function presented in Section 3.2
$mas$ (maximum archive size)	30
$ng$ (number of grids)	10
$ϵ$ (grid’s inflation parameter)	0.1
$\pi$ (pressure parameter)	4
$\zeta$ (plum selection pressure parameter)	2

presents the MOPTA configuration parameters used in our experiments.

As a remark, in the case of $X_{max},$ the table also adds the value 1 to the upper limit since the search space is represented by continuous values. However, if the upper limit is obtained, then the value 1 is subtracted from that value. Figure 6 presents this adjusting transformation more clearly for the first dimension.

As can be seen in the figure, the values from $[0, 1)$ are adjusted to $0$, the values from $[1, 2)$ are adjusted to $1$, and the values from $[2, 3]$ are adjusted to $2$, respectively.

4.4. MOPTA Prediction Results

Table 5. MOPTA prediction results summary.

Metric		RMSE	R2	MAPE	MAE
Fold 1	Heating	0.043057	0.99797	0.012089	0.03
	Cooling	0.091637	0.991753	0.047623	0.06
Fold 2	Heating	0.032117	0.999037	0.006547	0.02
	Cooling	0.071462	0.994942	0.058136	0.05
Fold 3	Heating	0.038437	0.998473	0.029722	0.03
	Cooling	0.075794	0.993825	0.023327	0.05
Fold 4	Heating	0.029834	0.999056	−0.001601	0.02
	Cooling	0.058583	0.99617	0.044078	0.04
Fold 5	Heating	0.035152	0.998872	0.007234	0.03
	Cooling	0.083508	0.99387	0.165621	0.05
Mean	Heating	0.035719	0.998682	0.010798	0.026
	Cooling	0.076197	0.994112	0.067757	0.05

presents the results obtained by the MOPTA for each of the five folds and the mean results.

As can be seen in the table, in the case of the MAPE metric, the result was negative for Fold 4. The negative value is justified by the fact that after the standardization operation, the labels had both positive and negative values. In all five cases, the selected algorithm was the GBR.

4.5. Comparison to the Prediction Results Obtained Using the Default Parameters

Table 6. Comparison of the MOPTA results to the ones obtained by each algorithm when the default parameters were used.

	Metric		RMSE	R2	MAPE	MAE
Fold 1	GBR	Heating	0.058349	0.996273	0.044175	0.04
		Cooling	0.183215	0.967036	0.119231	0.12
	RFR	Heating	0.059156	0.996169	0.046786	0.04
		Cooling	0.18708	0.965631	0.124183	0.12
	ETR	Heating	0.054964	0.996693	0.036841	0.04
		Cooling	0.197788	0.961584	0.14311	0.12
	MOPTA	Heating	0.043057	0.99797	0.012089	0.03
		Cooling	0.091637	0.991753	0.047623	0.06
Fold 2	GBR	Heating	0.042297	0.99833	0.018927	0.03
		Cooling	0.161946	0.974028	0.27774	0.11
	RFR	Heating	0.045063	0.998104	0.020777	0.03
		Cooling	0.172825	0.970421	0.269599	0.11
	ETR	Heating	0.043534	0.998231	0.018459	0.03
		Cooling	0.185254	0.966014	0.208484	0.11
	MOPTA	Heating	0.032117	0.999037	0.006547	0.02
		Cooling	0.071462	0.994942	0.058136	0.05
Fold 3	GBR	Heating	0.052812	0.997118	0.244372	0.04
		Cooling	0.128911	0.982138	0.053806	0.09
	RFR	Heating	0.052765	0.997123	0.151096	0.04
		Cooling	0.156191	0.973779	0.07873	0.1
	ETR	Heating	0.0502	0.997396	0.11682	0.03
		Cooling	0.172112	0.968161	0.069121	0.09
	MOPTA	Heating	0.038437	0.998473	0.029722	0.03
		Cooling	0.075794	0.993825	0.023327	0.05
Fold 4	GBR	Heating	0.041434	0.99818	0.006423	0.03
		Cooling	0.138274	0.978668	0.275395	0.1
	RFR	Heating	0.042305	0.998103	0.003031	0.03
		Cooling	0.152909	0.973914	0.37535	0.09
	ETR	Heating	0.039679	0.998331	0.013102	0.03
		Cooling	0.163703	0.970101	0.426203	0.09
	MOPTA	Heating	0.029834	0.999056	−0.001601	0.02
		Cooling	0.058583	0.99617	0.044078	0.04
Fold 5	GBR	Heating	0.039741	0.998559	0.011487	0.03
		Cooling	0.167864	0.975231	0.469739	0.11
	RFR	Heating	0.042071	0.998385	0.015281	0.03
		Cooling	0.1788	0.971899	0.501039	0.11
	ETR	Heating	0.04125	0.998447	0.018558	0.03
		Cooling	0.197247	0.965802	0.511074	0.12
	MOPTA	Heating	0.035152	0.998872	0.007234	0.03
		Cooling	0.083508	0.99387	0.165621	0.05
Mean	GBR	Heating	0.046927	0.997692	0.065077	0.034
		Cooling	0.156042	0.97542	0.239182	0.105999
	RFR	Heating	0.048272	0.997577	0.047394	0.034
		Cooling	0.169561	0.971129	0.26978	0.105999
	ETR	Heating	0.045925	0.997819	0.040756	0.032
		Cooling	0.183221	0.966332	0.271598	0.105999
	MOPTA	Heating	0.035719	0.998682	0.010798	0.026
		Cooling	0.076197	0.994112	0.067757	0.05

compares the results obtained by the MOPTA approach to the ones obtained by each of the algorithms GBR, RFR, and ETR when the default values were used. We considered these three algorithms in the comparison because they are used by the MOPTA as part of the optimization process. Moreover, the best solution returned by the MOPTA describes which of the three algorithms is applied and the optimal values of the hyperparameters. Basically, in this table, each MOPTA result corresponds to one of the three algorithms, depending on the value of the first dimension of the best plum, tuned according to the values of the remaining dimensions of the best plum. To get reproducible results, each of the algorithms was initialized with a random_state equal to 42.

The obtained results show that the MOPTA RMSE results were better than the ones returned by the GBR, the RFR, and the ETR, both for the heating predictions and the cooling predictions in all cases.

4.6. Comparison to Other Multi-Objective Optimization Approaches

The results obtained using the MOPTA were compared to the ones obtained by the MOGWO, the MOPSO, and the NSGA-II. We considered these algorithm methods in the comparison because a part of the mathematical equations of the Plum Tree Algorithm were inspired by the Grey Wolf Optimizer and the Particle Swarm Optimization, while the Genetic Algorithms, which are at the base of the NSGA-II, are one of the most popular evolutionary algorithms. Moreover, the Particle Swarm Optimization and the Grey Wolf Optimizer are some of the most popular swarm intelligence algorithms. Therefore, we considered the multi-objective implementations of these three benchmark algorithms to validate our results. In the case of the MOGWO algorithm, we used the implementation from our previous work [25], as it was used for this type of problem. The MOPSO and NSGA-II were also used in [25] to validate our results. For NSGA-II, we considered the implementation from the DEAP (Distributed Evolutionary Algorithms in Python) framework [46].

Some of the configuration parameters have the same values as in the case of the MOPTA, while other parameters were specific to each algorithm. The common configuration parameters for all four algorithms were the number of iterations $I$, the number of dimensions $D$, the population size $N$, the minimum and the maximum position values $X_{min}$ and $X_{max}$, and the objective function $OF$.

The configuration parameters that were common to the MOPTA, the MOGWO, and the MOPSO were the maximum archive size $mas$, the number of grids $ng$, the grid’s inflation parameter $ϵ$, the pressure parameter $\pi$, and the selection pressure parameter $\zeta$.

Table 7. Specific configuration parameters values.

Algorithm	Configuration Parameter	Value
MOGWO	$a$ (exploration parameter)	$2$
MOPSO	$V_{min}$ (minimum velocity value)	$-6$
	$V_{max}$ (maximum velocity value)	$6$
	$w_{min}$ (minimum inertia weight)	$0.2$
	$w_{max}$ (maximum inertia weight)	$0.9$
	$c_1$ (cognitive component)	$2$
	$c_2$ (social component)	$2$
NSGA-II	$CP$ (crossover probability)	$0.8$

presents the specific configuration parameter values for each algorithm.

Table 8. Comparison of the MOPTA results to the ones obtained by the MOGWO, the MOPTA, and the NSGA-II.

	Metric		RMSE	R2	MAPE	MAE
Fold 1	MOGWO	Heating	0.043057	0.99797	0.012089	0.03
		Cooling	0.091637	0.991753	0.047623	0.06
	MOPSO	Heating	0.042969	0.997979	0.012419	0.03
		Cooling	0.092244	0.991644	0.044089	0.06
	NSGA-II	Heating	0.043012	0.997975	0.010445	0.03
		Cooling	0.091747	0.991734	0.047607	0.06
	MOPTA	Heating	0.043057	0.99797	0.012089	0.03
		Cooling	0.091637	0.991753	0.047623	0.06
Fold 2	MOGWO	Heating	0.032117	0.999037	0.006547	0.02
		Cooling	0.071462	0.994942	0.058136	0.05
	MOPSO	Heating	0.034733	0.998874	0.00745	0.02
		Cooling	0.074935	0.994439	0.078264	0.05
	NSGA-II	Heating	0.03464	0.99888	0.007623	0.02
		Cooling	0.075657	0.994331	0.078111	0.05
	MOPTA	Heating	0.032117	0.999037	0.006547	0.02
		Cooling	0.071462	0.994942	0.058136	0.05
Fold 3	MOGWO	Heating	0.038437	0.998473	0.029722	0.03
		Cooling	0.075794	0.993825	0.023327	0.05
	MOPSO	Heating	0.038646	0.998456	0.052916	0.03
		Cooling	0.076207	0.993758	0.020738	0.05
	NSGA-II	Heating	0.041798	0.998194	0.050196	0.03
		Cooling	0.072592	0.994336	0.016488	0.05
	MOPTA	Heating	0.038437	0.998473	0.029722	0.03
		Cooling	0.075794	0.993825	0.023327	0.05
Fold 4	MOGWO	Heating	0.03216	0.998903	0.001763	0.02
		Cooling	0.060117	0.995967	0.055273	0.04
	MOPSO	Heating	0.032466	0.998882	−0.000460	0.02
		Cooling	0.056647	0.996419	0.049985	0.04
	NSGA-II	Heating	0.031941	0.998918	−0.001230	0.02
		Cooling	0.057837	0.996267	0.05597	0.04
	MOPTA	Heating	0.029834	0.999056	−0.001601	0.02
		Cooling	0.058583	0.99617	0.044078	0.04
Fold 5	MOGWO	Heating	0.035853	0.998827	−0.002201	0.02
		Cooling	0.114958	0.988383	0.12386	0.06
	MOPSO	Heating	0.035628	0.998841	0.007137	0.03
		Cooling	0.084536	0.993718	0.171809	0.05
	NSGA-II	Heating	0.036782	0.998765	0.008075	0.03
		Cooling	0.089095	0.993022	0.250595	0.06
	MOPTA	Heating	0.035152	0.998872	0.007234	0.03
		Cooling	0.083508	0.99387	0.165621	0.05
Mean	MOGWO	Heating	0.036325	0.998642	0.009584	0.024
		Cooling	0.082794	0.992974	0.061644	0.052
	MOPSO	Heating	0.036888	0.998606	0.015892	0.026
		Cooling	0.076914	0.993995	0.072977	0.05
	NSGA-II	Heating	0.037635	0.998546	0.015022	0.026
		Cooling	0.077385	0.993938	0.089754	0.052
	MOPTA	Heating	0.035719	0.998682	0.010798	0.026
		Cooling	0.076197	0.994112	0.067757	0.05

compares the MOPTA results to the ones obtained by the other three multi-objective optimization algorithms.

The MOGWO returned the best RMSE for cooling for Fold 1 and Fold 2 and the best RMSE for heating for Fold 2 and Fold 3. The MOPSO returned the best RMSE for heating for Fold 1 and the best RMSE for cooling for Fold 4. The NSGA-II returned the best RMSE for cooling for Fold 4. The MOPTA returned the best RMSE for heating for all folds except for Fold 1, and the best RMSE for cooling for Fold 1, Fold 2, and Fold 5. Also, the MOPTA obtained the best mean RMSE values both for the heating predictions and for the cooling predictions. Another remark is that all of the multi-objective algorithms selected the GBR, even though, as can be seen in Table 6, the GBR does not always return the best RMSE results when compared to RFR and ETR when the default parameter values are used.

Table 9. Summary of the algorithm ranking.

Algorithm	Heating	Cooling	Summary
MOGWO	2	2	4
MOPSO	1	1	2
NSGA-II	0	1	1
MOPTA	4	3	7

describes how many times each algorithm was the best with respect to the five folds.

We can see that in seven cases, the MOPTA was the best. The second algorithm, the MOGWO, was the best in only four cases.

4.7. Computational Load Analysis

This section presents a computational load analysis of the algorithms used in our experiments from the perspective of the running time.

Table 10. Running time comparison.

Algorithm	Running Time (ms)
MOPTA	70,475,987
MOGWO	34,962,471
MOPSO	31,266,781
NSGA-II	5,457,951
GBR	297
RFR	797
ETR	657

summarizes the total running time expressed in milliseconds for each algorithm across all five folds.

As can be seen in the table, the GBR, the RFR, and the ETR had the best running time. The running time of the MOPTA, which was approximately 19.5 h, was almost double compared to the running time of the MOGWO and MOPSO algorithms. The running time of the NSGA-II was almost 12 times better than the one of the MOPTA.

However, we also want to point out that a grid search which searches through all combinations of hyperparameters, namely $3\times 401\times 91\times 10\times 9\times 8=\mathrm{78,820,560}$, and which would need an average of around 200 ms per experiment, a value which is slightly less than that of the GBR algorithm when it is tuned with the default parameters (e.g., 297 ms), would need around 15,764,112,000 ms to complete, or around 182 days. With respect to these remarks, we can conclude that the MOPTA has a much better running time compared to the running time of the standard grid search.

4.8. Robustness and Convergence Analysis

In this section, we discuss the robustness and the convergence of the MOPTA, and we compare the results to the ones corresponding to the MOGWO, the MOPSO, and the NSGA-II.

To compute the robustness values, we consider the heating and cooling RMSE results obtained for each fold and calculate the standard deviation.

Table 11. Standard deviation results comparison summary.

Algorithm	Heating (std)	Cooling (std)
MOPTA	0.005224	0.012496
MOGWO	0.004611	0.021239
MOPSO	0.004057	0.013306
NSGA-II	0.004700	0.013706

summarizes the comparison of the standard deviation (std) values obtained for each multi-objective optimization algorithm.

If we are to consider that a lower variability means a better robustness value, then the MOPSO returned the best result for the heating standard deviation, while the MOPTA returned the best result for the cooling standard deviation.

For the analysis of the convergence of the MOPTA, we identified the first iteration, which returned the best value for each fold. We performed similar calculations for the other multi-objective algorithms.

Table 12. Convergence analysis results summary.

Algorithm	Fold	Best Position	Iteration
MOPTA	Fold 1	[0, 600, 10, 10, 6, 8]	3
	Fold 2	[0, 600, 91, 10, 9, 4]	2
	Fold 3	[0, 600, 10, 10, 5, 6]	5
	Fold 4	[0, 600, 10, 7, 8, 5]	1
	Fold 5	[0, 600, 11, 10, 2, 8]	20
MOGWO	Fold 1	[0, 600, 10, 10, 7, 8]	27
	Fold 2	[0, 600, 68, 10, 2, 4]	11
	Fold 3	[0, 600, 10, 10, 4, 6]	17
	Fold 4	[0, 600, 62, 10, 9, 6]	47
	Fold 5	[0, 600, 35, 5, 8, 8]	22
MOPSO	Fold 1	[0, 567, 42, 10, 10, 8]	33
	Fold 2	[0, 589, 73, 6, 2, 5]	43
	Fold 3	[0, 576, 35, 10, 2, 6]	23
	Fold 4	[0, 584, 67, 9, 2, 6]	43
	Fold 5	[0, 548, 48, 10, 4, 8]	2
NSGA-II	Fold 1	[0, 599, 56, 11, 3, 7]	29
	Fold 2	[0, 537, 17, 6, 3, 5]	32
	Fold 3	[0, 587, 22, 9, 7, 5]	23
	Fold 4	[0, 589, 86, 9, 2, 8]	46
	Fold 5	[0, 567, 13, 11, 8, 4]	36

summarizes the convergence analysis results.

The MOPTA converged relatively fast compared to the other algorithms except for Fold 5, where it obtained the best result in Iteration 20. In the case of the other algorithms, the best results were obtained after more iterations, except for the PSO and Fold 5, when the best result was obtained in Iteration 2.

5. Discussions

This section compares our results to the ones obtained by recent studies in the literature.

Table 13. Comparison to the literature results.

Year	Approach	Method	Heating (RMSE)	Cooling (RMSE)
2024	this	A method based on Multi-Objective Plum Tree Algorithm which optimized both the heating and the cooling root mean square error.	0.035719	0.076197
2024	Guo et al. [47]	A method based on consolidated artificial neural network paradigms and metaheuristic algorithms.	2.4417	2.8031
2023	Abdelkader et al. [48]	A hybrid parameterized deep learning model based on the Bayesian Optimization.	1.69	2.79
2023	Moldovan [21]	A method based on an ensemble of four regressors which was optimized using the Plum Tree Algorithm.	0.043124	0.159903
2023	Salami et al. [34]	An XGBoost hybrid machine learning methodology for the prediction of the energy loads.	0.3797	0.7578
2022	Ghasemkani et al. [31]	A Tri-Layer Neural Network and Maximum Relevance Minimum Redundancy-based method.	0.45689	0.81391
2019	Prasetiyo et al. [49]	A Naïve Bayes Classifier-based method.	0.1491	0.1412
2019	Bui et al. [50]	A method that uses the Imperialist Competition Algorithm to optimize the weights and the biases of an Artificial Neural Network.	2.7819	2.7795

presents a summary of the result comparison building upon the comparison results presented in [21]. The articles presented in the table were selected so that a 5-fold cross-validation was used, and the models performed two predictions, one for the heating load and the other one for the cooling load.

The results presented in [47,48] are better than the ones presented in [50], but they are not directly comparable because no standardization was performed. For articles [31,49,50], we presented only the best values. In the case of article [34], which used a 5-fold cross-validation like our approach, we presented only the best results. Even though the results from [34] are not directly comparable, as we also used standardization, they were better than the ones from [31]. However, the root mean square error results of the current approach based on the Multi-Objective Plum Tree Algorithm were significantly better than the results obtained by the Plum Tree Algorithm-based ensemble.

Even though they are not directly comparable to our results because of different pre-processing configurations or cross-validation settings, the recent studies based on the latest deep learning methods returned promising results. For example, the approach presented in [51] based on deep neural networks returned a root mean square error equal to 0.0137. However, compared to our approach, the problem was converted into an image processing problem by transforming the data into image datasets. Rounding the heating and cooling load values to the closest integer, the issue was converted into a multi-class classification issue.

6. Conclusions

The manuscript presented a novel approach based on the Multi-Objective Plum Tree Algorithm for the prediction of the heating and cooling loads. The dataset that was used for the testing and validation of the approach was the Energy Efficiency Dataset from the UCI Machine Learning Repository. The solutions were ranked using the MOORA method. The results are better than the ones returned by the individual predictors Gradient Boosting Regressor, Random Forest Regressor, and Extra Trees Regressor, respectively. The results were also comparable to the ones returned by other multi-objective optimization approaches such as the Multi-Objective Grey Wolf Optimizer, the Multi-Objective Particle Swarm Optimization, and the NSGA-II, respectively. Compared to the results obtained in one of the previous research studies, where the Plum Tree Algorithm was used to tune an ensemble of predictors, the results were better. Also, the Multi-Objective Plum Tree Algorithm results were compared to the ones from the literature. The following directions are proposed for future research work:

• The improvement of the performance of the proposed algorithm through hybridization or the use of concepts such as Levy flights;
• The comparison of the results obtained by the Multi-Objective Plum Tree Algorithm to the ones obtained using other multi-objective optimization approaches that were not considered in the current paper;
• The application of the Multi-Objective Plum Tree Algorithm to more engineering problems from the field of energy efficiency prediction;
• The adaptation of the prediction methodology presented in the manuscript to a larger class of energy-consuming buildings, the ones which belong to the heavy industry, considering more data about the characteristics of the buildings, which complement characteristics such as the surface area, the overall height, and the orientation.