Comparison of Factorial and Latin Hypercube Sampling Designs for Meta-Models of Building Heating and Cooling Loads

Abstract

Interest in research analyzing and predicting energy loads and consumption in the early stages of building design using meta-models has constantly increased in recent years. Generally, it requires many simulated or measured results to build meta-models, which significantly affects their accuracy. In this study, Latin Hypercube Sampling (LHS) is proposed as an alternative to Fractional Factor Design (FFD), since it can improve the accuracy while including the nonlinear effect of design parameters with a smaller size of data. Building energy loads of an office floor with ten design parameters were selected as the meta-models’ objectives, and were developed using the two sampling methods. The accuracy of predicting the heating/cooling loads of the meta-models for alternative floor designs was compared. For the considered ranges of design parameters, window insulation (WDI) and Solar Heat Gain Coefficient (SHGC) were found to have nonlinear characteristics on cooling and heating loads. LHS showed better prediction accuracy compared to FFD, since LHS considers the nonlinear impacts for a given number of treatments. It is always a good idea to use LHS over FFD for a given number of treatments, since the existence of nonlinearity in the relation is not pre-existing information.

1. Introduction

The abrupt increase in global energy use is a significant factor that augments global warming. As urbanization continues, the total energy used in the building sector has increased to approximately 40% of the global energy use and contributes to as much as 30% of the global CO₂ emission [1]. In Korea, energy use in the building sector constitutes approximately 18% of the total energy consumption. This number increases to 55% in Seoul, the largest cosmopolitan city in South Korea [2]. Many researchers have conducted studies on topics focused on the reduction of the global energy consumption and CO₂ emission. Such investigations have expanded to the building sector and are focused primarily on: (1) reducing the heating and cooling loads of buildings, (2) decreasing energy consumption by optimally controlling highly efficient installations, and (3) using as much renewable energy as possible.

A building’s heating and cooling load, and hence its energy consumption, can be reduced by designing or renovating the building in a way that minimizes these loads. In many cases, a dynamic building energy simulation is used to estimate building cooling and heating loads since no actual building exists during the building design phase [3,4,5,6,7,8,9,10,11,12]. By analyzing the simulation results, it is possible to evaluate the impact and sensitivity of the design factors on the loads. Once sufficient data is available, meta-models representing the functional relationship between the design factors and heating and cooling loads can be derived. In addition to building design factors, this study has been expanded to predicting the heating and cooling load when constructing a given shape of a building at an arbitrary site by developing a meta-model that considers varying climate factors [12]. With the recent developments in machine learning and artificial intelligence-related research, along with the availability of building-information databases, the use of meta-models in building energy management is advancing [13,14,15,16,17].

To understand the functional relationship between building design factors and heating and cooling loads, building energy simulation is required for several design conditions consisting of a combination of building design factors. These design conditions are called “treatments”. Depending on how these treatments are prepared, they are classified into various experimental design methods such as one factor at a time (OFAT), factorial, and Latin hypercube sampling (LHS) methods [18]. OFAT is the most classic design of experiment (DOE) for investigating changes in responses (in this study, the heating and cooling loads), and varies one factor at a time. OFAT is an excellent method for understanding the functional relationship between the design factors and responses if there is no interaction between the design factors. However, it has been reported that interactions of the building envelope design factors affect the loads significantly [5,9,19]. Factorial design is an experimental design that considers the interactions between the design factors. A factorial design can be either a full or fractional factorial design [20]. When the number of design factors is large, as it is in cases where the design factors affect building heating and cooling loads, only two levels are considered for each factor in order to reduce the number of treatments for dynamic building simulations. Considering ten design factors, as in Xu et al. [9], the full factorial design method, shown in Figure 1a, requires that the response of $2^{10}=1024$ treatments be obtained from experiments or simulations. In cases where higher-order interactions are neglected, fractional factorial design (FFD) can obtain the functional relationships using half, a quarter, or one eighth subsets of the full factorial design treatments. These two-level factorial designs consider each treatment condition only at both endpoints of the factor range (Figure 1a) and assume that the response changes linearly with the factor level.

These factorial designs are the most frequently used experimental designs, since they enable an experimenter to study the joint effect of the factors (or process/design parameters) on a response [20,21]. There have been many studies investigating the functional relationship between heating and cooling loads and building envelope design factors using the fractional factorial design method [4,9,12,19,22]. The regression analysis of the collected experimental and simulation results makes a sensitivity and influence analysis of the design factors during the design phase of a low-energy building feasible.

Delgarm et al. [23] observed a nonlinear relationship between building design factors and heating and cooling loads. To incorporate nonlinear functional relationships, three or more levels of each factor must be considered. For ten factors at three levels, a total number of treatments of 59,049 ($3^{10}=$59,049) should be estimated for heating and cooling loads, which is impractical. Instead, the response surface design (RSM) can be used to consider nonlinearity for a few (3–5) design factors. However, this requires prior information on which factors have nonlinear relationships to the response values in order to consider them as nonlinear terms. Otherwise, 3ⁿ treatments are necessary in order to consider all possible nonlinearity, as shown in Figure 1b.

According to the design selection guideline presented in the handbook from NIST [18], the central composite design and Box-Behnken design are adequate for cases comprising two to four design factors. The guideline also recommended to perform screening to reduce the number of design factors if the cases consider four or more factors prior to using the aforementioned methods. Therefore, they might not be suitable for building envelope optimization problems where the number of factors presumably exceeds ten. The D-optimal design, on the other hand, could decrease the number of trials or simulations; however, pre-existing or expected information of functional relation should be given to generate the design of test conditions such that the number of design cases will vary according to the provided functional relation. In stark contrast to this study, pre-existing knowledge of the relationship between the design factors and the building and heating cooling loads (BHCL) is not required.

In essence, if the experimental design only involves a small number of factors (<5), conventional design methods such as factorial design, composite central design (CCD) and Box-Behnken design can be adopted. If the cost of experiments and simulations is regarded as crucial, the screening process of these designs can sieve less significant factors and direct the focus toward the more critical ones in order to save financial and time investment. If the experimental design consists of a large number of factors (≥10), the space filling design, such as LHS, should be employed. LHS is the representative method and the surrogate models are derived from the experimental results to analyze and represent the functional relationship between the response and design factors [24]. LHS is discussed more in the next paragraph.

Recent advances in computing power and simulation methodologies have introduced the space-filling Latin hypercube sampling (LHS) method for generating meta-models using experimental design and statistical analysis. These meta-models can be used as surrogate models to find optimal values instead of directly finding optimal values for optimization problems. Figure 1c shows the treatments projected onto a plane of two factors. LHS is a “random” sampling method for selecting treatments evenly over the entire sampling space and investigate the functional relationships between the considered parameters and the responses when the relations are highly nonlinear. LHS has been widely used in building performance analysis. In particular, LHS is often used for uncertainty and sensitivity analysis [25,26,27,28,29,30,31,32,33].

This study compares the accuracy and efficiency of meta-models developed using LHS and FFD and provides guidelines for the selection of experimental designs for researchers developing meta-models using dynamic building simulations. To the best of the authors’ knowledge, there has been no explicit comparative study on the effectiveness, efficiency, and accuracy of the Latin hypercube sampling and factorial design methods for the optimization of building envelope design. The improvement in the model accuracy due to the ability of LHS to consider nonlinearity is investigated quantitatively in order to provide guidelines for researchers on the selection of an appropriate experimental design.

2. Methods

2.1. Baseline Model Selection and Main Assumptions

Simulation conditions such as weather conditions, building load, internal gains and operation schedules are briefly illustrated in

Table 1. Simulation conditions.

Weather data	Seoul.TMY2 (Climate zone 4)
Set temperature	26 °C in Cooling season and 20 °C in Heating season [35]
Internal heat gain [36,37]	People: 0.1 people/m2 Lighting: 12 W/m2 Equipment: 16 W/m2 Schedule: Taken from [35,36]
Building load [36,37]	U-value of exterior wall: 0.365 W/m2 K U-value of window: 2.84 W/m2 K SHGC of window: 0.4
Infiltration [38]	0.3 ACH

, while the thickness and thermophysical properties of the wall are described in

Table 2. Thickness and thermophysical properties of the wall.

	Thickness (m)	Conductivity (W/m K)	Capacity (J/kg K)	Density (kg/m3)
Inside gypsum plastering	0.025	0.209	840	800
Insulation material	0.07	0.047	1190	30
Outside wall panel	0.024–0.151 *	0.039	840	110

* varied to simulate the intended insulation performance.

. More details on the baseline building were described in Xu et al. [9]. They performed a parametric study of the impacts of the building envelope design factors on the building heating and cooling loads using the automated TRNSYS building energy simulation data [34]. For brevity, the details and the floor area schematics are not presented in this study.

Since our objective is to compare the factorial design and Latin hypercube sampling design methods for optimal building envelope design, the climate condition was fixed at one building location (Seoul, Korea) to focus on the comparison of the two design methods. Refer to Yong et al. [19] for further information on the climate condition effect on BHCL.

Table 3. List of factors and their ranges.

Factor	Abbreviation	Level
		Low (0)	High (1)
Floor area (m2)	FA	1000	2000
Aspect ratio	AR	1	2
Orientation (degrees)	OR	South (0)	West (90)
Window-to-wall ratio (%)	WWR	25	52
Ceiling height (m)	CH	2.4	2.9
Plenum height (m)	PH	0.8	1.2
Wall insulation (W/m2 K)	WI	0.15	0.36
Window insulation (W/m2 K)	WDI	0.75	2.84
Solar heat gain coefficient (%)	SHGC	20	70
Air leakage (ACH)	ACR	0.1	0.3

lists the design factors considered and their ranges. Heat transfer rate through building envelope depends on the indoor and outdoor temperature difference, effective area of heat transfer, and internal and external heat gain. The indoor temperature, which is the target temperature for the sake of residence comfort, is assumed to be constant for building heating and cooling load estimation, while the outdoor temperature varies in time along with the weather condition where the building is located. The same amount of energy as the heat transfer rate should be supplied to, or removed from, the building to keep the indoor temperature consistent at the target value of thermal comfort. This translates to the required building heating and cooling load (BHCL). Hence, the building heating load increases under colder climate conditions while the cooling load decreases and vice versa under hotter climate conditions. The effective heat transfer area and building thermal capacity are functions of the building envelope design factors such as floor area (FA), ceiling height (CH), plenum height (PH), and aspect ratio (AR). BHCL increases with the effective heat transfer area, while it decreases with the building thermal capacity. Therefore, the influence of the factor changes might not be explicit due to the competition of the impacts on both the effective heat transfer area and the thermal capacity. Improvement of wall insulation (WI) and window insulation (WDI) increases the heat transfer resistance. The window-to-wall ratio (WWR) is another factor that affects the effective heat transfer resistance of the building envelope, since WI is generally better than WDI. The increase in WWR could increase BHCL when it is desirable to reduce the heat transfer rate, while it could reduce BHCL for cases where it is beneficial to expedite the heat transfer. Solar heat gain coefficient (SHGC) and infiltration air change rate (ACR) control other forms of heat exchange from the sun and outdoor air, which could either increase or decrease BHCL depending on the time of day and weather conditions.

As mentioned above, any change on the building design factor could either increase or decrease BHCL depending on the time, location, and magnitude, and the impact could alter depending on the levels of other factors. Therefore, it is a very difficult and complicated task to make decisions for optimal building design considering the main effects and interactions of these design factors. There have been studies conducted to develop methodologies that handle the tasks systematically [9,19,33]; these prepared the data in order to investigate the functional relationship between BHCL and building envelope design factors and derived the surrogate meta model by applying regression analysis and machine learning techniques.

Since the 2-level FFD considers the end point of each factors, there is no difficulty in implementing the designed treatments. In contrast, LHS is relatively hard to implement on every designed condition for every factor. It is especially difficult to consider many types of windows with many levels of window insulation (WDI) and solar heat gain coefficient (SHGC); hence, three levels for each factor (0.75, 1.7 and 2.84 W/m²K for WDI and 20, 40, and 70% for SHGC) were considered, since it was challenging to prepare the windows to consider more levels for the two design factors in TRNSYS due to the interdependence between them. All the design factors other than WDI and SHGC were regarded as continuously varying factors.

2.2. Design of Experiments

Xu et al. [9] and Yong et al. [12,19] used fractional factorial design (FFD) to prepare treatments consisting of alternative building designs with varying design factors. They considered the main effects and all two-factor interactions while neglecting the nonlinear effects. Figure 1a shows the design points of the full factorial design projected on a surface spanned by the composition of two design factors located at the endpoints. The full factorial design consists of 1024 treatments for the ten design factors considered.

Figure 1c shows the sampling points for the Latin hypercube sampling. The design points were widespread over the surface of two design factors generated by the Latin-Hypercube function of the pyDOE package for Python [39]. LHS tends to equalize and maximize the distance between design points to provide uniform random sampling. It treats every design variable with equal importance and ensures uniformly distributed sampling in a given design space. A total of 2048 treatments or sampling points were generated, for which specific treatments were used to develop the meta-models for building heating and cooling loads. Figure 2 illustrates the property of LHS by sampling sizes. Of the full treatment set, 1024 treatments were randomly sampled from the main set and the rest of the treatments were used as a validation set. Sub-sample sets have 128, 256 and 512 sample points that are extracted from the main set.

Figure 3 shows the schematics of the comparison process. The treatments or the design points were generated according to the experimental designs, wherein each treatment represented an alternative building envelope design. The heating and cooling loads for all treatments were obtained by using the energy simulation automation process according to Kang et al. [34]. The meta-models were derived using stepwise regression analysis for the cases in

Table 4. List of cases considered to compare the accuracy of FFD and LHS.

Case	Meta-Model				Validation Data
	Meta-Model	Sampling Method	Regression Method	Sampling Number
Case 1-1	Meta-model a-1	FFD	Linear	128/256/512	FFD
Case 1-2	Meta-model a-2		Square
Case 1-3	Meta-model b-1	LHS	Linear	128/256/512/1024
Case 1-4	Meta-model b-2		Square
Case 2-1	Meta-model a-1	FFD	Linear	128/256/512	LHS
Case 2-2	Meta-model a-2		Square
Case 2-3	Meta-model b-1	LHS	Linear	128/256/512/1024
Case 2-4	Meta-model b-2		Square

, and their accuracy and validity were analyzed. Equation (1) shows the general form of the meta-model.

$$Y_k=f\left(X_1, X_2,X_3,X_4, \cdots X_n\right)=c_{0k}+\sum c_{ik}·X_i+\sum c_{ijk}·X_i·X_j+ϵ$$

(1)

here, X and Y represent the design and response variables, respectively. The subscripts i, j, and k are used to identify different variables and coefficients, c. The first summation term in the right-hand side stands for the main effects of the design variable, and the second summation term is the interaction terms. The residual of the regression is shown as $ϵ$.

Stepwise regression was performed to avoid overfitting when using the open-source statistical software, R [40]. Then, for the last process, the accuracy of the generated models is validated and compared. Therefore, the cases were classified according to the validation test data, the design point sampling method, and the regression method used.

Table 4 shows the list of cases used to compare the accuracy of the meta-models. The cases were mainly divided into two subgroups according to the “design” point sampling method used. The design points, or treatments, were determined by using either FFD or LHS. The cases were further divided based on the type of regression model (i.e., a linear or a quadratic model) and illustrated as the meta-models. The sampling points of FFD concentrate at the endpoints, whereas they are distributed over the design space for LHS. This leads to the deviation of the model prediction accuracy for different validation sets. The veracity of the meta-models was tested against the two different validation data sets, FFD and LHS, and the cases were divided into cases 1 and 2 according to the “validation” data sets from the regression analysis excluded in the derivation of the metal-models. Each validation data set contained 512 and 1024 treatments for FFD and LHS, respectively. Therefore, the prediction accuracy comparison of meta-models is performed not only for the validation data of the same sampling method, but also for the ones from the other sampling method as shown in Figure 3.

3. Results and Discussion

3.1. Development of the Meta-Models

The coefficients of the meta-models for building heating and cooling loads are listed in

Table A1. List of the main effect and interaction terms and coefficients of Equation (1) for the heating load of the building.

Fractional Factorial Sampling (Meta-Model a-1, Sampling Number: 512)				Latin Hypercube Sampling (Meta-Model b-2, Sampling Number: 512)
Factor	Coefficient	Factor	Coefficient	Factor	Coefficient	Factor	Coefficient
(Intercept)	−19.74	FA:SHGC	5.09E−05	(Intercept)	−26.5	FA:WDI	−0.00316
AR	−0.4107	FA:WWR	−5.8E−05	AR	−1.009	FA:SHGC	0.000189
FA	0.004394	FA:WI	−0.00482	FA	0.004271	FA:WWR	−7.8E−05
CH	4.888	FA:ACR	−0.00116	CH	6.91	FA:WI	−0.00312
PH	2.921	CH:WDI	1.54	PH	−1.152	FA:OR	−5.8E−06
WDI	−2.606	CH:SHGC	−0.05731	WDI	−0.215	FA:I(SHGC^2)	−1.9E−06
SHGC	0.1946	CH:WWR	0.03245	SHGC	1.254	FA:I(WDI^2)	0.000439
WWR	0.0607	CH:WI	2.985	WWR	0.1274	CH:WDI	3.798
WI	11.65	CH:ACR	21.2	WI	1.113	CH:SHGC	−0.1626
ACR	−9.729	PH:WDI	1.462	ACR	−18.79	CH:WWR	0.03005
OR	−0.0115	PH:SHGC	−0.0389	OR	−0.05114	CH:WI	6.2
AR:FA	−0.00023	PH:WWR	0.04828	I(SHGC^2)	−0.01724	CH:ACR	20.66
AR:CH	0.2574	PH:WI	4.248	I(WDI^2)	−1.113	CH:OR	0.01076
AR:WDI	0.3186	PH:ACR	7.721	AR:PH	1.001	CH:I(SHGC^2)	0.001321
AR:SHGC	−0.0069	WDI:SHGC	−0.02956	AR:WDI	1.258	CH:I(WDI^2)	−0.4713
AR:WWR	0.01086	WDI:WWR	0.1064	AR:SHGC	−0.05625	PH:WDI	0.9934
AR:OR	0.002964	WDI:WI	0.6382	AR:OR	0.007653	PH:SHGC	−0.102
FA:CH	−0.00068	WDI:ACR	1.479	AR:I(SHGC^2)	0.000671	PH:WWR	0.04976
FA:PH	−0.00124	WDI:OR	0.001436	AR:I(WDI^2)	−0.2225	PH:WI	7.536
FA:WDI	−0.00171	SHGC:WWR	−0.00271	FA:CH	−0.00129	PH:ACR	10.43

and

Table A2. List of the main effect and interaction terms and coefficients of Equation (1) for the cooling load of the building.

Fractional Factorial Sampling (Meta-Model a-1, Sampling Number: 512)				Latin Hypercube Sampling (Meta-Model b-2, Sampling Number: 512)
Factor	Coefficient	Factor	Coefficient	Factor	Coefficient	Factor	Coefficient
(Intercept)	64.26	FA:WI	0.002763	(Intercept)	56.58	FA:WDI	0.00051
AR	−1.47	FA:ACR	−0.00197	AR	−2.57	FA:SHGC	−0.00017
FA	0.007343	CH:SHGC	0.07293	FA	0.009083	FA:WWR	−7.7E−05
CH	−2.811	CH:WWR	0.07476	CH	−1.513	FA:I(SHGC^2)	6.71E−07
PH	−2.543	PH:WDI	−0.5328	PH	−1.738	CH:SHGC	0.06898
WDI	−3.046	PH:SHGC	0.08419	WDI	−0.2447	CH:WWR	0.07604
SHGC	−0.1781	PH:WWR	0.05117	SHGC	−0.01485	PH:SHGC	0.07062
WWR	−0.2029	WDI:SHGC	0.05068	WWR	−0.313	PH:WWR	0.04219
WI	−11.52	WDI:WWR	−0.03779	WI	−9.937	PH:OR	0.01052
ACR	−13.18	WDI:ACR	2.941	ACR	−16.71	WDI:SHGC	0.2018
OR	−0.02924	SHGC:WWR	0.008425	OR	−0.04141	WDI:WWR	−0.1112
AR:FA	−0.0002	SHGC:WI	−0.02208	I(SHGC^2)	0.000367	WDI:ACR	6.495
AR:WDI	−0.1019	SHGC:ACR	−0.09084	I(WDI^2)	0.1003	WDI:I(SHGC^2)	−0.00488
AR:SHGC	0.02076	SHGC:OR	−0.00013	AR:PH	−0.7593	SHGC:WWR	0.01565
AR:WWR	0.01458	WWR:WI	0.06953	AR:SHGC	0.07467	SHGC:ACR	−0.3382
AR:OR	0.02109	WI:ACR	5.886	AR:WWR	0.03442	SHGC:I(WDI^2)	−0.09947
FA:CH	−0.00092			AR:OR	0.02044	WWR:WI	0.1422
FA:WDI	0.000441			AR:I(SHGC^2)	−0.00061	WWR:ACR	0.05938
FA:SHGC	−0.00011			AR:I(WDI^2)	−0.05166	WWR:I(SHGC^2)	−9E−05
FA:WWR	−8.7E−05			FA:CH	−0.0015	WWR:I(WDI^2)	0.01679

. Although the coefficients of the meta-models were determined for all four meta-models of the eight cases in Table 4, only the cases of the sample number 512 were shown for brevity of the presentation. There is no second-order term for the regression model of FFD, since the design points only have two levels at the endpoints for all design factors. Therefore, Meta-model a-1 and Meta-model a-2 yielded the same regression equation, and the same discussion applies for Meta-model a-1 and a-2 of the second case. Meta-model b-1 was derived from the regression analysis considering only the linear effects of design factors from the LHS sampling points. In contrast, it was found that the second-order terms of SHGC and WDI were influential and not negligible in meta-model b-2, while those of other design factors were removed during the stepwise regression analysis. They have a significant positive effect on the accuracy of the BHCL predictions. Plugging in the coefficients to Equation (1), the BHCL can be obtained for any combination of design factors in the range considered in Table 1. The main effect factors are the single factors listed in Table A1 and Table A2 while the two-factor interactions are indicated by two factors joined by a colon (e.g., AR: FA, FA: WI). If the coefficients for the main effects are positive, the increases in the factors increase the heating and cooling loads, and vice versa. For the interactions, the loads increase when the two factors change in the same direction, i.e., when the levels of both factors increase or decrease simultaneously, for the case of having positive coefficient.

3.2. Comparison of the Accuracy of FFD and LHS

The heating and cooling loads of the validation sets were predicted by the developed meta-models. Prediction accuracy of each case are compared using root mean square errors (RMSEs) and adjusted coefficient of determination (R²) value. The fitting accuracy of the meta-models are compared mainly on the basis of the R² value. The RMSEs of the predictions for all cases as defined in Equation (2) are compared in Figure 4a,b.

$$\mathrm{RMSE} = \frac{{{\displaystyle \sum }}_{i=1}^n\sqrt{{\left(y_i-x_i\right)}^2}}{n}$$

(2)

where y_i, x_i, and n represent the predicted loads, the TRNSYS estimated loads, and the number of validation points, respectively.

RMSEs generally decreased with an increase in sampling number and so as its decrease rate. The RMSEs of the cooling load were relatively lower than that of the heating load. However, the difference is negligible for LHS cases considering nonlinearity (Case 2-4). The more substantial errors of the linear models for heating loads are attributed to the neglect of the higher nonlinearity effects for the heating loads. Regarding the meta-models’ prediction accuracy with the FFD validation data, the meta-models using the FFD design points showed the best accuracy. The meta-models using the LHS design points showed much better performance for the LHS validation data. Further details are discussed in the following sections.

3.2.1. Comparison of the Meta-Model Accuracy for FFD Validation Set

Case 1-1 was chosen to develop the meta-model, with the design points at the two endpoints for all design factors. It shows the lowest RMSE when the models are validated with the FFD validation data sets.

Prior to the comparison of the prediction accuracy, the fitting accuracies of each meta-models are compared in

Table 5. Fitting and prediction accuracy for the FFD validation data (HL: Heating Load, CL: Cooling Load).

Sampling Number	Fitting Accuracy			Prediction Accuracy
	Meta-Model a-1 (HL,CL)	Meta-Model b-1 (HL,CL)	Meta-Model b-2 (HL,CL)	Case 1-1 (HL,CL)	Case 1-3 (HL,CL)	Case 1-4 (HL,CL)
128	0.9966, 0.9959	0.8585, 0.9517	0.9951, 0.9968	0.9940, 0.9935	0.6731, 0.8918	0.9806, 0.9909
256	0.9971, 0.9963	0.8023, 0.9423	0.9929, 0.9966	0.9956, 0.9943	0.7943, 0.9382	0.9917, 0.9960
512	0.9968, 0.9960	0.8126, 0.9398	0.9926, 0.9966	0.9964, 0.9950	0.8224, 0.9443	0.9929, 0.9966
1024		0.8263, 0.9446	0.9926, 0.9968		0.8312, 0.9469	0.9935, 0.9970

. In the case of the meta-model developed by FFD (Meta-model a-1), the sampling accuracy with the minimum sampling number showed the R² value closest to 1. However, meta-model b-1 showed lower accuracy of less than 0.9, but it gets better with the consideration of square term (meta-model b-2).

In Figure 5a,b, Case 1-1 shows a far smaller error than Case 1-4, which was implied by the higher determination coefficients, R², shown in Table 5. On closer inspection in Figure 4a,b, if the RMSE is evaluated with the FFD validation data sets, Case 1-1 is the best. Case 1-2 merges with Case 1-1, since the regression cannot consider the quadratic terms due to the lack of intermediate levels for design factors with two-level factorial designs. While FFD is not able to consider nonlinear responses (since it considers two levels for each design factor), the LHS method is able to account for them, since it samples the intermediate points between the limits. The inability to consider the nonlinear response explains the meta-model similarity between the meta-models for Cases 1-1 and 1-2, as well as for Cases 2-1 and 2-2. Case 1-3, the linear meta-model with the LHS design conditions, has a large number of intermediate levels for all factors except for all the three levels of WDI and SHGC. The meta-models for Case 1-3 (meta-model b-1) do not consider nonlinearity. The RMSE on predicting the heating and cooling loads with the 128 points of sampling data were 7.89 and 5.20, respectively, when validating the meta-models with the FFD validation data sets. The RMSEs of Case 1-3 for larger sampling numbers stay at the highest level. Case 1-4 is the case deriving the meta-models using the LHS design conditions while considering nonlinearity. Although the accuracies according to the sampling numbers on heating load and cooling load (1.42–2.38 and 0.94–1.56, respectively) were significantly improved compared to that of Case 1-3, they were still worse than that of Case 1-1 for the FFD validation data sets, and the difference is greater for the heating load than the result from the cooling load. Hence, the R² values for the cases were: 1-1, 1-4, and 1-3 (in descending order) for the FFD validation data sets (Table 5).

Since all FFD design points were located at the endpoints with only two levels (at the lowest and highest values for each design factor), the predictions that used the meta-models developed from the FFD design conditions had good accuracy at the terminal points, i.e., FFD sampling points, as shown in Figure 5a,b. The predictions of the meta-models using FFD design conditions were more accurate than the ones derived using LHS design points with the nonlinear effect consideration. The predictions of FFD meta-models are distributed closer to the line than those of the LHS meta-models.

3.2.2. Comparison of the Meta-Model Accuracy for LHS Validation Set

The LHS meta-models with second-order terms showed the best accuracy with the LHS validation data (Cases 2-1 to 2-4). The RMSE of the FFD meta-models increased for the LHS validation data set, where the sampling points were spread widely over the whole design space. The meta-models using the LHS design points showed better accuracy for the LHS validation data set. This can be explained by the prediction accuracy (R²) presented in

Table 6. Prediction accuracy for the LHS validation data.

Sampling Number	Prediction Accuracy
	Case 2-1 (HL,CL)	Case 2-3 (HL,CL)	Case 2-4 HL,CL)
128	0.6917, 0.9220	0.6497, 0.8733	0.9752, 0.9889
256	0.6959, 0.9235	0.7753, 0.9266	0.9911, 0.9955
512	0.6967, 0.9233	0.7940, 0.9340	0.9910, 0.9960
1024		0.8089, 0.9379	0.9911, 0.9964

showing Case 2-4, which had the highest R² value for heating and cooling loads projection. Although the meta-models developed with LHS design sampling conditions accounting for the second-order terms showed poorer accuracy for the FFD validation points as observed with Case 1, the better accuracy for the design conditions with intermediate factor levels lowered the average errors for LHS meta-models as shown in Figure 6a,b. Since the prediction error is of interest for the application of the meta-model, the fitting error information is omitted in Table 6 for brevity of the presentation.

Regression analysis was used to derive equations to estimate the heating and cooling loads at arbitrary points within the design factor variation range shown in Table 1. It is crucial to estimate the loads accurately for the points away from the endpoints in order to locate the optimum points with precision. Therefore, the accuracy assessments for the LHS validation sets are more important than those for FFD validation sets.

Comparing the two sampling methods for linear models, i.e., Cases 2-1 and 2-3 in Figure 4a,b, LHS is the better sampling method, since the accuracy improved for the cases in which the sampling number is greater than 128. The prediction accuracy improvement resulting from the consideration of the nonlinearity in Case 2-4 is significant for almost all sampling numbers. As shown in Figure 6a, the heating load prediction values using the FFD meta-models made an island of treatments away from the exact line, whereas those of the LHS meta-models stayed close to the exact line. Two strings of treatment deviating away from the exact line appeared for the cooling load prediction case, as shown in Figure 6b. Neglecting the nonlinear impact of SHGC and WDI on the building energy loads is the cause of these errors.

The errors of the linear model with LHS are approximately 4.4 and 3.9 times greater than those of the nonlinear model with LHS, which implies that it is always better to use LHS over FFD and to consider the nonlinearity. Although the meta-model derived from FFD showed good accuracy for the prediction of the sets at the endpoints, the prediction accuracy for the design sets away from the endpoints is better with the meta models using LHS.

Compared with the factorial design method, LHS was more accurate and efficient for the cases when the loads varied nonlinearly with the design factor changes. Fractional factorial design (FFD) could be a practical choice when there is no nonlinear effect. However, it is a better idea to use LHS over FFD for a given number of treatments since the existence of nonlinearity in the relation is not a pre-existing information. LHS could provide all the necessary information including the linearity test result for the same number of treatments with FFD, consideration of all two-factor interactions and better accuracy for intermediate factor level design points.

3.3. Analysis of Factor Influences

The impacts of the main effects and the interactions of the ten building envelope design factors were analyzed using the F-values obtained from the regression results for the meta-models. Equation (3) shows the definition of the F-value, which is the ratio of building energy load variation due to the main effects and interactions considered in the meta-model to the variation not explained by the meta-model.

$$\mathrm{F}-\mathrm{value}=\frac{explained variation/\left(k-1\right)}{unexplained variation/\left(n-k\right)}$$

(3)

where k and n are degree of freedom and the total number of data considered for the regression, respectively. The comparison of F-values could represent the relative importance of the main effect and interactions in building design.

Figure 7 and Figure 8 compare the relative importance of each effect in different meta-models. The relative importance was represented as the ratio of the load variation due to an effect to the total variation. The figures show the first twenty effects in descending order of importance. WDI, ACR, SHGC, CH, and WWR were the important design factors for the heating load of the building shown in Figure 7, while the main effects and interactions of SHGC, WDI, and WWR are the important design factors affecting the cooling load of the building shown in Figure 8. The main difference is the appearance of the main effects and interactions of the nonlinear terms in the LHS models, which improved the accuracy of the meta-models. There were notable variations in the sequence of importance between the FFD meta-models and the LHS models. The choice of proper experimental design method can affect not only the accuracy of the model prediction, but also the relative importance of the parameters considered.

Figure 9a,b show the building heating and cooling load variation with a design factor change and the other factors fixed at the middle levels. The hollow symbols in the figures imply that the levels are not explicitly considered in the experimental design and that they are estimated purely from the meta-models. The solid symbols represent the levels considered for the design points. The loads changed nonlinearly with the change in WDI and SHGC.

The slopes in Figure 9a,b compare the significance of the design factors. ACR is the most significant factor affecting the heating load while WDI shows comparable impact on the heating load with nonlinear behavior both on heating and cooling load. With the increase of WDI, which means the decrease of window performance on insulation, the heating load increases while the cooling load decreases. Although the impact of SHGC change on nonlinear behavior is relatively lowers than that of WDI, SHGC shows more nonlinear significance on the building load.

To gain a deeper understanding on the nonlinear effect of WDI and SHGC on heating and cooling load, three additional simulations were conducted. In the process of quantifying heating and cooling loads in TRNSYS software, WDI and SHGC were re-calculated based on the pane temperature for every time step. Re-calculated WDI and total heat flux according to the intended SHGC of each time step are obtained as the output and discussed as the mean values in Figure 10. All factors except SHGC were defined with the mid-value of the designed ranges. With the increase of SHGC, total heat fluxes both from inside and outside increases. Therefore, the inter-dependence between WDI and SHGC and its consideration in TRNSYS software, WDI also increased with the increase of the SHGC. In Figure 10, the total heat flux on the window increases with the increase in SHGC. The WDI value also doubled even when the input WDI was fixed at 1.7. This can result in the increase of the heating load presented in Figure 9a. The curve of WDI is similar to the nonlinearity behavior of SHGC during the heating season. The tendency where the glazing at 70% SHGC showed higher heating load than at 20% can be explained by the increase of the total heat flux. In contrast, the cooling load increases with the increased heat flux influenced by the decreased window performance (SHGC).

SHGC is the most significant design factor for cooling load and it shows severe nonlinear impact. Contrary to the heating load, the cooling load is at the lower minimum level of SHGC.

4. Conclusions

The Latin hypercube sampling (LHS) method was applied to investigate the effect of a building’s envelope design factors on the heating and cooling loads of the building. Compared with the factorial design method, LHS was more accurate and efficient for the cases wherein the loads changed nonlinearly with the design factor changes. The fractional factorial design (FFD) method could be a practical choice when it is certain that there is no nonlinearity effect.

The prediction accuracy generally improved with the number of sampling points. The accuracy improvement was significant when meta-models considered nonlinear relations between the loads and the design factors. The heating and cooling loads showed a strong nonlinearity with a change of WDI and SHGC. The FFD, while they showed good accuracy for the endpoints, had measurable errors on predictions for the treatments with intermediate values of WDI and SHGC since FFD could not capture the nonlinear behavior. For any given number of sampling points, the LHS meta-models accounting for the nonlinearity predicted the design conditions with intermediate values of WDI and SHGC accurately.

The determination coefficient, R², of regression analysis might not be the best index to evaluate the performance of meta-models. Assessment of the meta-model performance should be based on the prediction accuracy for the treatments with intermediate values of design factors since the meta-models are developed to predict loads of a building with arbitrary values within the range of design factors considered.

This study compared the accuracy on predicting the heating/cooling loads of an office building with the meta-models, FFD and LHS, and also identified the impacts of nonlinear characteristics on the building load. Although this study focused on the office building, readers who are trying to analyze building load using meta-models for all types of loads can refer to the methodology we have used. In addition, the methodology part can help wider audience get a deeper understanding on the principle of nonlinear behavior on building loads.

The ranges of design factor variation were relatively narrow, since the objective of this study was to optimize the building envelope design for a zero-energy building. More design factors are expected to show nonlinear relationships with the load in cases with more comprehensive range of design factors, where LHS is always the better choice over the factorial design method.