A Comparative Study of the Simulation Accuracy and Efficiency for the Urban Wind Environment Based on CFD Plug-Ins Integrated into Architectural Design Platforms

Abstract

The deterioration of the urban environment is a problem which has captured the attention of governmental departments and researchers, who are committed to improving the urban environment from the perspective of optimizing urban morphology. Although many researchers have applied computational fluid dynamics (CFD) plug-ins to study the problems of urban ventilation and pollutant accumulation, studies on the reliability and simulation accuracy verification of CFD plug-ins are currently scarce. Therefore, we used three CFD plug-ins based on different architectural design platforms to evaluate and compare their operation difficulty, simulation accuracy, and efficiency through the analysis of the simulation results of urban ventilation. This study complements the reliability validation of CFD plug-in simulations and guides urban planners and architects in the selection and application of CFD plug-ins. The results show that the CFD plug-in generally underestimates the wind speed at the pedestrian level and the prediction accuracy is poor in the wake area of obstacles, especially with the GH_Wind plug-in. Under the 0° inflow direction, the simulation results of the Butterfly plug-in were the most consistent with the experimental values. When the inflow direction increased to 22.5° and 45°, the Autodesk CFD showed the best simulation accuracy. Overall, Autodesk CFD achieves a balance between simulation accuracy and speed in urban airflow simulation.

1. Introduction

With the rapid development of urbanization in China, the urban population exceeds 60% of the total population, and is still increasing [1]. Urban scale, building density, building height, and other morphological parameters vary greatly, resulting in a series of urban environmental problems such as a decreased ventilation performance, an enhanced urban heat island intensity [2], and pollutant accumulation [3,4], which affect urban microclimate and human health [5]. To mitigate these urban problems, researchers have begun to improve urban ventilation by optimizing city blocks or building forms [6,7], and using green building technology to reduce carbon emissions [8,9,10,11]. In addition, many researchers are also working to innovate and optimize the methods and tools, such as computational fluid dynamics (CFD) [12,13,14], geographic information systems (GIS) [15,16], the weather research and forecasting models (WRF) [17,18], and urban dynamic analysis algorithms [19,20,21] for studying and improving the efficiency of solving urban problems. However, for urban planners and architects, it is still unclear which research methods and tools are best to use to study urban environmental problems, which need to be discussed, and suggestions are provided in the subsequent design work.

At present, the CFD method has become the most widely used for outdoor environment research other than wind tunnel experiments and field measurements [14,22,23,24]. Urban planners and architects can perform CFD simulations without the high time costs of experiments and measurements, which is important for design work where time is limited and the rapid feedback of results is required [25]. CFD software such as ANSYS Fluent [26], PHOENICS [27], and OpenFOAM [28] has been widely used by researchers to simulate outdoor ventilation conditions and pollutant dispersion [23,29,30]. However, most of them need to be operated by professional engineers and require users to reserve a large amount of corresponding expertise, which makes it difficult for urban planners and architects to learn and apply them [14,25]. Therefore, CFD simulation has not been widely used in urban environmental studies in actual urban and architectural design practice. Exploring simplified CFD methods and workflows and developing lightweight CFD tools are effective ways for urban planners and architects to widely apply CFD to help study and optimize urban environmental problems in design practice [31].

The integration of CFD methods into architectural design platforms to support numerical simulation through plug-ins is an important achievement that brings convenience to urban planners and architects [31,32,33]. The CFD plug-in allows geometric model generation, numerical simulations, and result visualization to be done on the same interface of the design platform, greatly simplifying the simulation workflow. Urban planners and architects can use the CFD plug-in to set up workflows and perform CFD simulations in their familiar design environments [32]. As lightweight tools, CFD plug-ins require much fewer computer resources than CFD software, typically occupying only 1% of the storage space of CFD software [31,34,35,36]. In addition, the high cost and time involved in learning a lot of CFD knowledge and becoming familiar with CFD simulations have limited and made significant barriers to acquiring CFD simulation skills for urban planners and architects in the past [25,36]. With this in mind, when integrating CFD into a design platform, developers simplify CFD methods and workflows in several aspects, such as the numerical schemes [37,38], turbulence models [39,40], and mesh generation methods [41,42,43], to make CFD simulations easy to understand and perform. A CFD plug-in usually develops a simple graphical user interface (GUI) on the interface of the architectural design platform to provide users with clear and explicit guidance, such as Swift, Autodesk CFD, and WS-Snake, and the automatic meshing function is developed to help users quickly get started with simulations [32]. Moreover, when the flow field is large, or the flow is complex, the CFD method usually needs a long computation time, which cannot meet the needs of urban and architectural design to quickly obtain feedback on performance simulation results [34,35]. In response, Waibel et al. [44] integrated a fast fluid dynamics (FFD) method into the plug-in, greatly reducing the time required to perform airflow simulations on an architectural design platform. However, a large number of studies have found that the simulation results obtained by CFD plug-ins are not as accurate as CFD software [38,44,45,46,47], and now more and more researchers apply CFD plug-ins to study urban environmental problems [31,32]. Therefore, to avoid misleading guidance for design and research, the reliability and simulation accuracy of the CFD plug-in simulations needs to be validated and studied specifically. For this, this paper aims to conduct a detailed study and comparison of the simulation accuracy of different CFD plug-ins.

At present, urban geometric models used in outdoor ventilation studies are mainly divided into three categories: an ideal cubic block [48,49,50], actual urban geometry [51,52,53], and a typical morphological model obtained by analyzing the real morphology [54,55,56]. The ideal cubic block model is widely used to analyze the effects of morphology indices on urban wind-thermal environments and to understand the correlation between form and environmental parameters [57,58]. In addition, the ideal cube model is widely used by researchers to validate the accuracy of the CFD simulations because reliable experimental data have been obtained from numerous wind tunnel experiments and field measurements on the ideal cubic block models [59]. Before using CFD software to study urban ventilation and pollution transportation, Xu et al. [60] and Shen et al. [53] verified the reliability of the simulation setup and results according to the benchmark urban cubic block model and its experimental data proposed by the Architectural Institute of Japan (AIJ) guidelines. In these studies, the simulated wind velocity values of 78 measuring points in the urban block are in good agreement with the experimental data and meet the requirement that the relative deviation of the highest wind speed point is less than 15% as recommended by the AIJ [59], indicating that the CFD simulation setup is reasonable and reliable. In addition, several studies compared the simulation results of CFD plug-ins and commercial CFD software to analyze the simulation accuracy of CFD plug-ins. Chronis et al. [45] found that the CFD plug-in RhinoCFD was less accurate than OpenFOAM in simulating indoor natural ventilation conditions. Several studies have shown that the simulation accuracy of Autodesk CFD and Autodesk Flow Design is still not as good as that of OpenFOAM [38,47,61]. However, few studies have compared the simulation results with wind tunnel experiments or field measurements when verifying the simulation accuracy of CFD plug-ins, so studies should be conducted to fill this gap. Therefore, in this study, we used the ideal urban cubic block model proposed by AIJ and its wind tunnel experimental data to evaluate and compare the accuracy of CFD plug-ins in urban airflow simulation, to improve the rationality and credibility of the study. Moreover, it is worth noting that Waibel et al. [44] and Li et al. [47] found that GH_Wind is several times faster than Autodesk CFD for urban airflow simulations, but it is far less accurate than Autodesk CFD in predicting the airflow field in the leeward of obstacles. So far, the relationship between CFD plug-in simulation speed and accuracy has not been fully explored. Therefore, this paper also makes a comprehensive comparison of the variations in the calculation speed of different CFD plug-ins in urban wind environment simulation, to explore the relationship between simulation speed and accuracy.

In short, the previous studies mainly focused on the application of CFD plug-ins to urban environmental problems and they paid little attention to the simulation accuracy and efficiency of CFD plug-ins. However, the simulation results obtained by CFD plug-ins will be used directly or indirectly to guide the modification and optimization of urban problems. To avoid misleading analysis results, it is necessary to pay attention to the reliability and accuracy of the plug-in simulation. In addition, the performance and characteristics of the CFD plug-in are the factors that urban planners and architects must consider when selecting it. A comprehensive evaluation of the simulation accuracy, efficiency, and operation convenience of CFD plug-ins will help to provide a clear reference for users to select a suitable CFD plug-in for research, but the research in this aspect is still lacking. Therefore, in this study, three CFD plug-ins based on two different architectural design platforms were selected to perform outdoor airflow conditions of the ideal urban cubic block model, as proposed by the AIJ, and their simulation accuracy and efficiency were compared and evaluated. By using the wind velocity ratio and other parameters, the variation trend of simulation accuracy under the condition of inflow direction change was explored. This paper is organized as follows. Section 2 introduces the study object, the CFD plug-ins under study, numerical methods, and setup of boundary conditions. Section 3 discusses the performance of CFD plug-ins in simulating urban natural ventilation conditions, including simulation accuracy and efficiency, and puts forward suggestions on the problems of the learning and application of CFD plug-ins. Finally, the conclusions and limitations are presented in Section 4.

2. Methodology

2.1. Study Object

In the AIJ guidelines, seven experimental cases are proposed [59]. Among them, a typical urban block model named “case D” was chosen by this study. The case D model consists of 82 low-rise buildings (40 × 40 × 10 m) and one high-rise building (25 × 25 × 100 m) (Figure 1). The Niigata Institute of Technology has conducted wind tunnel experiments on its scale model at a scale of 1:400, as shown in Figure 1A. According to weather data collected by the local weather station, the inflow velocity at 100 m reference height was set as 6.65 m/s [59,62]. Since the wind tunnel model was scaled down, the inflow velocity of the wind tunnel should also be corrected during the experiment. The experimental staff set 78 monitoring points around the central high-rise building to analyze the airflow and wind speed near the central high-rise building. Since AIJ has obtained reliable experimental data about wind velocity through wind tunnel experiments, case D was selected as the study object in this paper.

2.2. CFD Simulation Tools Integrated into Architectural Design Platforms

For studying the simulation accuracy and speed of CFD integrated into architectural design platforms, three commonly used CFD plug-ins, including Autodesk CFD, Butterfly, and GH_Wind, were investigated and studied.

2.2.1. Autodesk CFD

Autodesk CFD is a CFD software developed by Autodesk company, which integrates several basic functions into the Autodesk Revit platform in the form of plug-ins [63]. Autodesk CFD is often used in industrial fluid flow control design, electronic heat dissipation simulation, thermal analysis of construction equipment, and advanced physical motion simulation and provides multiple post-processing modes for fluid flow and thermal simulations. A significant advantage of Autodesk CFD is its adaptive and automatic meshing capabilities, which greatly simplify modeling efforts.

2.2.2. Butterfly

Butterfly is a CFD plug-in based on the Grasshopper platform [64]. It includes an object-oriented Python package, which allows Butterfly to call the solver OpenFOAM for CFD simulation on the Grasshopper platform [28,31]. As part of the Ladybug Tools suite, it can be coupled with other tools for performance simulations. In addition, Butterfly is open source, which offers a good base for secondary development.

2.2.3. GH_Wind

GH_Wind is a plug-in using the FFD method developed for airflow simulations based on the Grasshopper platform [44,65,66]. It is a toolkit that includes a wind tunnel module, automatic mesh generation, and solving and visualization modules for pressure and velocity fields. According to the summaries of Refs. [44,65], GH_Wind is much faster than the CFD tools but less accurate than most CFD software, especially in the wake area. Therefore, GH_Wind can quickly generate simulation results and feedback to users, which has great potential to reduce time and trial error costs. However, GH_Wind is not as widely used and validated as the other CFD tools due to its short history.

2.3. Computational Domain and Boundary Conditions

Butterfly, GH_Wind, and Autodesk CFD used hexahedral boxes to construct computational domains. According to the actual size of the AIJ wind tunnel experimental model, the wind tunnel experiment domain was set as 1.8 m × 1.8 m × 1.8 m. The high-rise building was located in the center of the domain plane so that the central high-rise building and the test point area meet the recommendations of the AIJ guidelines, namely, the distance from the target area to the inlet in the computational domain is 5 H. The distance from the edge of the target area to the outlet is 10 H, where H is the highest point of the building complex (100 m for this study) [59,67]. The setup of the computational domain meets the recommendation that the distance from the target area to the boundary of the computational domain is 2.5 H to avoid undue acceleration of the fluid caused by the boundary of the calculation domain.

The airflow from the inlet will be affected by terrain relief, obstacle friction, and other factors during the flow process. The wind velocity distribution and contour along the vertical direction will change. The airflow closer to the ground is more affected by friction, resulting in a lower wind speed. Therefore, the airflow simulation should consider the influence of surface roughness and the atmospheric boundary layer (ABL) on the inflow velocity profile [13,68]. Butterfly’s inflow profile adopts the logarithmic function form recommended by the European cooperation in the field of scientific and technical research (COST) in the vertical dimension using Equations (1) and (2) [68]:

$$U=\frac{U^{*}}{K}ln\left(\frac{z-z_g+z_0}{z_0}\right)$$

(1)

$$U^{*}=K\frac{U_{ref}}{ln\left(\frac{z_{ref}+z_0}{z_0}\right)}$$

(2)

where $U^{*}$ is the friction velocity; $U_{ref}$ is the reference wind speed at the reference height $z_{ref}$, and the value is 6.65 m/s; $K$ is the von Karman constant; $z$ is the vertical coordinate; $z_0$ is the roughness height of the surface; and $z_g$ is the $z$ coordinate of the lowest starting surface. To match the atmospheric conditions, the inflow speed at the 10 m reference height was set to 6.65 m/s for Butterfly and GH_Wind.

For GH_Wind, the “Terrain” option in the inlet boundary condition module was set to 4 to obtain the inflow profile consistent with Butterfly [44,64]. In addition, if the “Terrain” option is set to an integer within 0–3, the inflow profile in the power function recommended by ASHRAE will be obtained, as shown in Equation (3) [44].

$$u\left(z\right)=u_{met}{\left(\frac{{\delta }_{met}}{z_{met}}\right)}^{{\alpha }_{met}}{\left(\frac{z}{\delta }\right)}^{\alpha }$$

(3)

where $u_{met}$ is the reference wind speed value; ${\delta }_{met}$ is the reference boundary layer thickness, which is set as 210 m; ${\alpha }_{met}$ is the index of the nearby weather station with a value of 0.1; $z_{met}$ is the reference height of inflow, usually 10 m above the ground; $\delta$ and $\alpha$ are the indices of boundary layer thickness and local building topography, and their values are related to the selected topography.

In this study, the inflow wind profile consistent with those of Butterfly and GH_Wind was fitted by the piecewise linear function of “height-velocity” in Autodesk CFD so that the three CFD tools could produce the same inflow, as shown in Figure 2 [63].

In addition, outlet boundary conditions were set as pressure outlets for Butterfly and Autodesk CFD. The outlet boundary condition of GH_Wind defaults to the zero gradient condition. The ground, sides, and top of the computational domains and all building surfaces were set as non-slip wall boundaries during simulation, where the roughness height ($z_0$) of the surface is set to 0.002 m. In our previous studies [24,53,60,69], the AIJ benchmark model and its experimental data were used to verify the accuracy of the numerical simulation. In addition to the default settings of the CFD plug-in, the authors used the same setup method, such as the boundary conditions and grid size distribution, as our previous papers verified using AIJ data [24,60].

2.4. Grid Generation and Mesh Independence Tests

2.4.1. Butterfly

The regular hexahedral cells of Butterfly were used to generate a computational grid and the snappy hex mesh (SHM) component was used to identify and match the geometry of the buildings to generate good quality and high fitness meshes [64]. In the computational domain, three local encryption zones were set up near the central high-rise building (including the observation points), near the building group, and between the building group and the boundary of the domain so that the meshes of the area which were closed to the building were more refined. Figure 3 shows the detailed mesh for the building group generated by Butterfly.

In this paper, different sizes of grid schemes were tested for Butterfly. The size of the grid scheme takes one-tenth of the length of the shortest side of the central high-rise as the reference dimension (2.5 m), representing the scheme with the largest grid size [59,70]. The reference dimension was divided by √2 in turn to yield the grid size for the other schemes. In addition, on the premise of meeting the convergence criteria, the corresponding results of the same iterations were selected for comparative analysis.

Table 1. The computation time of different grid schemes for Butterfly.

Grid Schemes	Grid 1	Grid 2	Grid 3	Grid 4	Grid 5	Grid 6	Grid 7
Minimum mesh size (m)	2.5	1.875	1.25	0.83	0.625	0.5	0.44
Number of the total meshes (million)	0.073	0.13	0.4	0.86	1.3	2.8	8.4
Iterations (times)	5000	5000	5000	5000	5000	5000	5000
Computation time (h)	0.35	1.6	4.2	6.9	11.3	20.6	47.3

shows the mesh number of different grid schemes of Butterfly and the time required for airflow simulations.

For ease of distinction, points 26 to 29 are considered the windward side of the central building; points 40 to 43 are considered the leeward side; points 26, 32, 36, and 40 are considered the streets on the left side of the central building; and points 33, 37, and 43 are considered the streets on the right side. As shown in Figure 4, the prediction of Grid 1 for the central building windward area (points 34, 35, 38, and 39) is significantly low. The higher the mesh resolution, the closer the simulation results are to the most delicate mesh. The deviation between Grid 3 and 7 is close to that between Grid 4 and 7, and the maximum computation time of Grid 3 is 5.18 h, which has a significant advantage compared with the full simulation computation time of Grid 4 (12.75 h). Moreover, the simulation results obtained by Grid 3 are consistent with the experimental values, and the simulation accuracy is acceptable. Therefore, for the 0° wind direction simulation, the corresponding setting of Grid 3 is an excellent choice to balance the calculation accuracy and the calculation time cost.

2.4.2. Autodesk CFD

Autodesk CFD automatically recognizes geometry, adaptively adjusts, and generates tetrahedral unstructured meshes to improve the grid quality [63]. In this paper, when using Autodesk CFD, the maximum grid size was set as the length (25 m) of the shortest side of the central building. The minimum size setting meets the requirement that there are 5 meshes between pedestrian height and ground level, i.e., 0.4 m grid size (Figure 5A).

The wind velocity prediction of Grid 1 in the windward area of the central building is too low, as shown in Figure 5B.

Table 2. The computation time of different grid schemes for Autodesk CFD.

Grid Schemes	Grid 1	Grid 2	Grid 3	Grid 4	Grid 5
Number of the total meshes (million)	0.26	0.58	1	1.8	2.6
Iterations (times)	500	500	500	500	500
Computation time (h)	0.25	0.5	1.1	1.5	3.7

shows the mesh number of the different grid schemes of Autodesk CFD and the time required for airflow simulations. From the trend perspective, the more acceptable the grid resolution, the closer the result is to the experimental values. The deviation between Grid 2 and 5 is closer to that between Grid 3 and 5. The calculation time of Grid 2 is 0.95 h, which is shorter than that of Grid 3 (1.6 h). In addition, the simulation results of Grid 2 are consistent with the wind tunnel values. Therefore, for 0° wind direction airflow simulation, Grid 2 is an excellent choice to balance the calculation accuracy and time cost.

2.4.3. GH_Wind

GH_Wind produces a uniform voxelization domain. Hexahedral voxels are evenly distributed in the computational domain [44]. When obstacles occupy a certain proportion of the volume of grid cells, the grid cells are transformed into voxels representing obstacles for calculation. The computational grid generated using GH_Wind is shown in Figure 6. Since GH_Wind does not support changing the inflow direction, altering the inflow can only be achieved by rotating the building group. However, rotating the building group implies a change in the meshing of the computational domain, and matching the uniform grid generated by GH_Wind to the sloping edges of the building results in a zigzagged grid, which may affect the simulation accuracy. In addition, GH_Wind can only set global dimensions for the grid, so the sizes of each voxel are consistent. Due to the limitation of computer performance, in the mesh sensitivity test, the most refined mesh size was set as 2 m. Due to the lack of wind tunnel data on the vertical plane, the GH_Wind simulation results were compared with the verified OpenFOAM results.

As shown in Figure 7A, the simulation results of various grid schemes of GH_Wind on the 2 m height plane are generally too low, which is inconsistent with the trend of OpenFOAM results. GH_Wind has poor accuracy in predicting airflow conditions at low altitudes. Grid D has a better prediction ability on the windward and left sides of the central high-rise building. In addition, the wind velocity results of GH_Wind on the horizontal plane with a 90 m height are basically consistent with the trend of OpenFOAM results (Figure 7B). As shown in

Table 3. The computation time of different grid schemes for GH_Wind.

Grid Schemes	Grid A	Grid B	Grid C	Grid D	Grid E	Grid F
Global grid size (m)	24	18	12	8	4	2
Number of the total meshes (million)	0.0135	0.03.2	0.108	0.365	2.915	23.32
Iterations (times)	1800	1800	1800	1800	1800	1800
Computation time (h)	0.1	0.18	0.52	1.56	13.52	58.3

, with the decrease in the global grid size, both the number of the total meshes and the computation time increases sharply. It is worth noting that the results obtained by Grid A with the largest mesh size produced a minor error, which is consistent with the view proposed by Waibel et al. [44] that GH_Wind should be simulated with the large time step and coarse mesh (LTCM). It also means that the accuracy of GH_Wind in airflow simulation is constantly improved with the height increase.

This paper reduces the size of the computational grid to improve the reliability of the simulation under the premise of computer resources. Based on the 2 m (superfine mesh), 3 m (fine mesh), 4 m (standard mesh), and 5 m (coarse mesh) grid schemes, the wind velocity distribution on the vertical plane were compared and analyzed to verify that the simulation accuracy of GH_Wind changes with the height.

As shown in Figure 8, the wind velocity in the windward area of the buildings is more consistent with the change rule of the OpenFOAM results than that in the leeward area. The simulation results of the ultra-fine mesh scheme are more accurate than those of other schemes in general. In addition, the results of ultra-fine mesh are consistent with the reference value at x = 60 m behind the leeward area of the central building. The delicate and standard meshes are more consistent with the reference values at x = 15 m before the windward region of the central building. It can be seen that the smaller mesh size is beneficial to improving the simulation accuracy in the leeward area of obstacles. In addition, the trend of the coarse mesh at x = 15 m behind the leeward region of the central building is consistent with the reference value, and the coincidence degree becomes higher and higher as the height increases.

2.5. Numerical Method and Turbulence Models

Butterfly applied the smooth solver developed in OpenFOAM, which uses the Gaussian–Seidel algorithm as a smoother to make the solution converge to a certain tolerance. GH_Wind needs to solve three diffusion equations (one for each velocity component in the x-axis direction $u_x$, and the x-axis is parallel to the 0° inflow wind direction) and one pressure-related Poisson equation for each time step. Still, the solver only needs to solve one equation at a time, so the developer chooses a simple Jacobi solver [44] as the CFD solver. Autodesk CFD applied the default segregated solver for the urban wind environment. The convergent residuals for pressure ($p$), momentum ($v$), and turbulent kinetic energy ($k$) variables were set to 10⁻⁴ and the convergent residual for the energy variable was set to 10⁻⁶. In particular, since GH_Wind was developed to speed up the computation, it sacrifices some accuracy, resulting in residuals of most variables that generally do not meet the convergence criteria when GH_Wind is applied [34,35,44]. Therefore, the change in the residual (especially the velocity component $u_x$) is used to judge whether the convergence state is reached. According to the results in Figure 9A, the development of 300 iterations is closer to the wind tunnel data. However, it can be seen from the wind velocity contours that the airflow in the wake region has not been fully developed. Hence, it is not reliable to take the first minimal point of the residual curve as a result. The results of 1000, 1800, and 3000 iterations are very similar, the relative deviations are less than 5%, and the variation of residuals tends to be flat after 1000 iterations (Figure 9B). Therefore, considering the simulation accuracy and economy, the results corresponding to the starting point where the residual tends to be flat are selected for subsequent simulation analysis.

By observing the results in Figure 10A, the simulation results of the re-normalization group k-epsilon (RNGKE) turbulence model are closest to the experimental values. Within points 30 to 38, the relative deviation between the results of the RNGKE turbulence model and the observed values is less than 20% (Figure 10B). The results obtained by the RNGKE turbulence model are more consistent with the experimental values than those obtained by other turbulence models. In addition, the contours obtained by Butterfly and Autodesk CFD are shown in Figure 11, where RNGKE can simulate the more detailed and higher resolution of the eddies and turbulence, including areas on the windward–leeward and lateral sides of the central high-rise. Therefore, the RNGKE turbulence model was adopted by Butterfly and Autodesk CFD in the subsequent simulations. In addition, GH_Wind still lacks a turbulence model. This paper reduced the Reynold number $Re$ by increasing the numerical viscosity $\nu$ to 0.1 and hence compensating for the turbulent viscosity. The issue of numerical viscosity has been discussed and studied by Waibel et al. [44] and Zuo et al. [35,71]. Detailed settings for establishing the CFD simulation workflow are shown in

Table 4. The setup of CFD plug-in simulations for urban wind environment prediction.

Computational Conditions			Parameter Configuration
Computational domain size			1.8 m × 1.8 m × 1.8 m
Grid type	Butterfly		Structured grid
	GH_Wind		Structured grid (voxelization)
	Autodesk CFD		Unstructured grid
Reference wind velocity			6.65 m/s (at 10 m height)
Convection item			$\mathrm{Quick} \mathrm{scheme} \mathrm{for} U,V,W,k,\epsilon$
Boundary conditions	Inlet	Butterfly	Inflow profile in logarithmic function $U=\frac{U^{*}}{K}ln\left(\frac{z-z_g+z_0}{z_0}\right)$ $U^{*}=K\frac{U_{ref}}{ln\left(\frac{z_{ref}+z_0}{z_0}\right)}$
		GH_Wind
		Autodesk CFD	Inflow profile in “velocity-height” piecewise linear function
	Outlet	Butterfly	Pressure outlet condition
		Autodesk CFD
		GH_Wind	Zero gradient condition
	Top		No-slip wall condition
	Ground
	Sides
Turbulence models	Butterfly		RNG k-epsilon turbulence model
	Autodesk CFD
	GH_Wind		$\mathrm{Numerical} \mathrm{viscosity} \gamma =0.1$

.

3. Results and Discussion

3.1. Analysis of Simulation Accuracy

3.1.1. 0° Wind Direction

Figure 12A shows that all of the CFD plug-in simulations underestimated wind speed in general at the pedestrian level. In the area near the central high-rise (points 24 to 53), the simulation results of the three CFD plug-ins are in good agreement with the wind velocity variation trend of the wind tunnel experimental value, especially in the streets on both sides of the central building. At the junction between the wake area of the front buildings and the windward area of the central building (points 16 to 23), the wind velocity variation trend obtained by the CFD plug-in simulations is basically consistent with the wind tunnel experiment. However, in the wake area of the central building and the front building (points 5, 6, 8–11, 13, 14, 41, and 42), only the wind velocity values obtained by Butterfly were consistent with the experimental data, with an average relative deviation of 18.7%. Autodesk CFD and GH_Wind show a large prediction deviation, especially GH_Wind, whose average relative deviation is as high as 68.3%. This reflects that GH_Wind has the worst accuracy in predicting ventilation conditions in the wake area of obstacles.

At the street exits between the front buildings (points 1 to 4) and at the street entrances between the back buildings (points 75 to 78) at the pedestrian level, the average relative deviations between the simulated and experimental wind velocity values obtained by Butterfly and Autodesk CFD were 94.4% and 77.5%, respectively. This means that they show extremely poor prediction accuracy in transition areas with large changes in the area and cannot restore the real ventilation condition. In contrast, the relative deviations of the Butterfly and Autodesk CFD simulations from the experiment were 13.6% and 20.5%, respectively, in transition areas with little change in the area (such as streets next to the central building). Their simulation accuracy was improved and the accuracy of the Butterfly’s simulation was acceptable. In addition, at the measurement point with the highest wind velocity (point 35), the relative deviations between Butterfly, Autodesk CFD, and GH_Wind and the experimental values were 7.5%, 21.9%, and 38.4%, respectively. For the point with the lowest wind velocity (point 53), the relative deviations are 47.2%, 14.6%, and 66.2%, respectively. On this basis, Butterfly has acceptable simulation accuracy in strong wind regions according to the difference of less than the 15% as recommended by the AIJ [24,59], while Autodesk CFD’s simulation shows good accuracy in wake wind regions.

In this paper, when the difference between the simulation result and the experimental value is less than or equal to 25%, the corresponding simulation accuracy of the measurement point is considered to be acceptable [72]. For statistical purposes, a 75% prediction envelope allowing a maximum relative deviation of 25% was distinguished around the reference line $X=Y$, as shown in Figure 12B–D. The more points concentrated in this area obtained by correlation analysis, the higher the simulation accuracy of the corresponding CFD plug-in. For Butterfly’s simulation result, 54.5% of the measured points at the pedestrian level had a strong correlation with the experimental values. This means that the Butterfly simulations have the highest agreement with the experiment in the case of 0° wind direction. The structural hexahedral mesh of Butterfly is advantageous for the simulation of cases where there are many vertical and parallel relationships between the incoming flow and the geometric models. In addition, there are 50.0% and 27.3% of the measurement points with a strong correlation between the simulation results of Autodesk CFD and GH_Wind and the experimental values, respectively. Thus, in the case of 0° wind direction, GH_Wind’s simulation shows the worst accuracy.

3.1.2. 22.5° Wind Direction

Although the simulation results of CFD plug-ins are still lower than the experimental values on the whole at the pedestrian level, compared with the 0° inflow direction, the overall deviation between the simulation results of CFD plug-ins (except GH_Wind) and the experimental values is reduced (Figure 13A). The simulation results of CFD plug-ins at the junction of the wake area of the front building and the windward area of the central building still keep a basically consistent change trend with the experimental value, which is slightly affected by the change in the inflow direction. It is worth noting that the difference between the simulation results of Autodesk CFD and the experimental values in the wake region of the center and the front building is smaller than that of Butterfly and GH_Wind, with a deviation of only 16.6%. When the building group rotates, the geometries and inflow lose a lot of orthogonal relations, resulting in the weakening of the advantage of the structured hexahedral mesh of Butterfly. In contrast, the unstructured tetrahedral mesh generated by Autodesk CFD can better fit the geometries, guaranteeing a good mesh quality and simulation accuracy. However, the prediction accuracy of GH_Wind in the wake area of obstacles is still poor and does not improve with the change in the wind direction.

The wind speed results of the Butterfly and Autodesk CFD simulations, at the street exits between the front buildings and at the street entrances between the back buildings, were more consistent with the experimental values than the 0° wind direction. The differences between the simulated wind speed and the experimental value were reduced to 38.3% (Butterfly) and 48.5% (Autodesk CFD), respectively, indicating that the simulation accuracy of Butterfly was significantly improved compared with the case of the 0° wind direction. This means that their prediction accuracy in the transition region where the area changes greatly tends to improve with the increase in the incoming flow angle. Still, their simulations exceed the maximum allowable deviation (25%). In addition, the simulation results of Butterfly and Autodesk CFD are in a good agreement with the experimental values in transition areas with little change in the area (such as the streets next to the central building). However, in these areas, the average deviation between GH_Wind’s simulation results and the experimental values increased from 39.8% (0° wind direction) to 57.5%, indicating the simulation accuracy decreased. Moreover, at the measurement point with the highest wind speed (point 39), the relative deviations between Butterfly, Autodesk CFD, and GH_Wind and the experimental values were 11.8%, 14.7%, and 68.3%, respectively. For the point with the lowest wind speed (point 14), the relative deviation is 8.7%, 4.1%, and 43.5%, respectively. According to the 15% allowable error recommended by the AIJ, Autodesk CFD showed an acceptable simulation accuracy in both strong and weak wind regions, while Butterfly maintained a good simulation accuracy in strong wind regions. However, the difference between GH_Wind’s results and the experimental values becomes larger when predicting the wind speed at the measuring points located in the strong wind area. This may be because, after the incoming flow direction changes, part of the strong wind regions corresponding to the 0° wind direction becomes the wake area of the building. Plus, GH_Wind is unable to encrypt the region around the central high-rise locally. After the urban model rotates, the edge of the central building appears as an obvious zigzag grid, resulting in a decrease in grid quality in the strong wind region. Therefore, the simulation accuracy of GH_Wind tends to decrease when the incoming flow angle increases.

Moreover, 51.5% of the Autodesk CFD’s simulation results of the total measured points at the pedestrian level showed a strong correlation with the experimental values (within the 75% prediction envelope), indicating the best simulation accuracy (Figure 13B) and an improvement in the simulation accuracy of Autodesk CFD compared to the 0° wind direction. In addition, there were 43.9% and 22.7% measurement points of Butterfly and GH_Wind with a strong correlation between the simulation results and experimental values, respectively (Figure 13C,D), indicating that their simulation accuracy tended to decrease with the increase in the incoming flow angle.

3.1.3. 45° Wind Direction

Figure 14A shows that the wind velocity results of the Butterfly and Autodesk CFD simulations were still lower than the experimental values on the whole at the pedestrian level, but compared with the cases of the 0° and 22.5° inflow directions, they achieved the smallest overall deviation under the case of the 45° wind direction. However, the overall deviation between the simulation result obtained by GH_Wind and the experimental values increased continuously (from 0° to 45°). In addition, when the inflow angle is 45°, the two sides constituting the corner of the central building become the main windward faces. For the upwind area of the central high-rise building (points 11, 12, 21, 22, 29, and 30), the simulation results of Butterfly and Autodesk CFD basically agree with the experimental values, with an average relative deviation of 5.3%. Among the three plug-ins, the simulation results of Autodesk CFD in the wake areas of the central building (points 32, 36, and 40–42) and the front buildings (points 2, 4, and 12–14) maintain the best agreement with the experimental values, with an average deviation of 23.6%. It shows that Autodesk CFD has a good prediction ability in the wake region of obstacles, which may be because the unstructured tetrahedral mesh generated by Autodesk CFD can fit the geometries and their edges well. However, the poor prediction accuracy of GH_Wind for the wake area of obstacles becomes worse with the increase in the inflow angle, and the problem of underestimating wind speed becomes worse.

The simulation accuracy of Butterfly and Autodesk CFD was again improved in the street exits between the front buildings and the street entrances between the back buildings when compared with the case of the 22.5° wind direction. This may be because, when the urban model was rotated to 45°, the area changed from the street exits and entrances to the strong wind region before the central high-rise decreased, and the transition of the flow became more relaxed. The average differences between the simulated and experimental wind velocity values are reduced to 28.6% (Butterfly) and 32.5% (Autodesk CFD), respectively, but still do not meet the requirement of error within 25%. This means that their prediction accuracy in the transition regions where the area changes greatly tend to improve with the increase in the incoming flow angle. In addition, the Butterfly and Autodesk CFD simulations for the street areas adjacent to the central building (points 25, 30–39, and 44) were very close and in close agreement with the experimental values. However, in this area, the average deviation between GH_Wind simulation results and experimental values increased from 57.5% (22.5° wind direction) to 71.1%, and the simulation accuracy decreased. In addition, at the measurement point with the highest wind velocity (point 35), the relative deviations between Butterfly, Autodesk CFD, and GH_Wind and the experimental values were 5.3%, 13.8%, and 72.6%, respectively. For the lowest wind speed point (point 24), the relative deviations are 11.5%, 24.3%, and 76.1%, respectively. According to the 15% maximum error recommended by the AIJ, Butterfly showed acceptable simulation accuracy in strong and weak wind regions, while Autodesk CFD maintained good simulation accuracy in strong wind regions. However, the simulation results of GH_Wind greatly deviate from the experimental values in strong and weak wind regions, showing a trend of decreasing the prediction ability with the increase in the inflow angle (from 0° to 45°). This may be because when the inflow direction changed to 45°, most of the strong wind areas corresponding to the 0° and 22.5° wind direction became the wake areas of the front buildings. In addition, after the rotation angle of the urban model increases, the problem of the zigzag meshes on the edges of the central building is aggravated.

For Autodesk CFD’s simulation, 54.5% of the results of the measured points at the pedestrian level showed a strong correlation with the experimental values (within the 75% prediction interval) (Figure 14B). In the simulation results of Butterfly and GH_Wind, 48.5% and 15.2% of the measured points showed a strong correlation between the simulated and experimental values, respectively (Figure 14C,D). It indicated that Autodesk CFD has the best simulation accuracy among the three plug-ins for the cases of 22.5° and 45° wind direction. However, regardless of wind direction, GH_Wind always has the worst simulation accuracy.

3.1.4. Summary

In summary, although the CFD plug-in generally underestimates the wind speed at the pedestrian level under any inflow direction, the overall deviation between the simulation results and the experimental values decreases with the increase in the incoming flow angle. Butterfly’s simulation results have the strongest correlation with the wind tunnel experimental values at the pedestrian level under the wind direction of 0°, which means that good prediction accuracy can be achieved. Autodesk CFD has the best simulation accuracy for the wind direction conditions of 22.5° and 45°. However, the simulation accuracy of GH_Wind is the worst among the three CFD plug-ins, and the prediction accuracy decreases with the increase in the incoming flow angle. In addition, even when the wind direction changed, Butterfly could still maintain good prediction accuracy at the point with the highest wind speed, making the relative deviation between the simulated and the experimental values lower than the 15% maximum error recommended by the AIJ. In contrast, Autodesk CFD showed high accuracy at the point with the lowest wind speed. In addition, Butterfly had the best prediction accuracy for the wind velocity on the streets next to the central high-rise building, but poor prediction ability for wind velocity in the entrance and exit areas of the street. Autodesk CFD can obtain the most consistent result with the experimental value when simulating the airflow conditions in the leeward areas of the obstacles. However, GH_Wind will seriously underestimate the wind speed in this area. In short,

Table 5. Comparison of simulation accuracy of different CFD plug-ins.

CFD Plug-Ins	Simulation Accuracy					The Accuracy Trend with Increasing Angles (0°–22.5°–45°)
	Overall	Around the Central Building
		Windward	Leeward	Left	Right
Butterfly	★★	★★	★	★★	★★	↓ and then ↑
Autodesk CFD	★	★★	★★	★	★	↑
GH_Wind				★	★	↓

summarizes the comparison of the simulation accuracy of the three plug-ins for wind velocity in different areas. The more stars “★”, the greater the advantage of the plug-in in corresponding aspects.

3.2. Analysis of Computational Time Cost

According to Section 2.4, when the CFD plug-ins adopt a 2 m grid scheme for airflow simulations in the target area, the total number of Butterfly, Autodesk CFD, and GH_Wind meshes are 0.13, 0.26, and 23.32 million, respectively, taking 0.35, 0.25, and 58.3 h. It can be seen that under the premise that the mesh size of the target area is the same, the Autodesk CFD’s computational speed is the fastest, followed by Butterfly. However, even though GH_Wind uses the FFD method for solving, it takes hundreds of times longer to compute than the other plug-ins, which is related to the inability of GH_Wind to perform local encryption for the grid. If the grid resolution of GH_Wind is brought to the same level as the other CFD plug-ins, the total number of grids is hundreds of times larger. At this point, the total number of GH_Wind grids is far beyond the range of computer resources. In addition, according to the residual diagram, Autodesk CFD only needs to run hundreds of iterations to meet the convergence criteria. At the same time, Butterfly takes thousands of iterations, which means that Autodesk CFD’s solvers and algorithms are more efficient. By comparing Table 1 and Table 2, it can be found that the computational time of Autodesk CFD and Butterfly is 3.7 h and 20.6 h, respectively, when the mesh size of the target area is 0.5 m (there are more than three meshes in the region between the pedestrian level and the ground at this point). Therefore, it is speculated that the computational time of Autodesk CFD will be much lower than Butterfly when the mesh size is reduced. Among the three CFD plug-ins, Autodesk CFD can better meet the requirements of simulation accuracy and speed in the early stage of urban and architectural design.

3.3. Characteristics and Applicability of Different Plug-Ins

3.3.1. Advantages and Disadvantages

The Butterfly is part of the Ladybug Tools and can be downloaded free from the Ladybug website [64]. While GH_Wind does not yet have an official website, users can download the plug-in for free on the GitHub website. Unlike the other two plug-ins, Autodesk CFD is commercial CFD software developed by Autodesk, which requires a paid download [63]. Although there is little information available on Butterfly learning, it is simulated using the verified CFD solver OpenFOAM, which has a large user base and technical support community to help architects get started. The paid plug-in Autodesk CFD has a mature GUI and clean workflow, with regular updates and open access to companion tutorials for users to quickly learn and get started, which is user-friendly [32,36,63]. However, the tutorial and studies on GH_Wind have not been widely presented due to the short development time.

Since Butterfly is based on the Grasshopper platform, it is necessary to connect battery modules to create and set up workflows for CFD simulations, which makes it difficult for architects to get started to some extent [33]. GH_Wind, like Butterfly, needs to establish workflows on the Grasshopper platform through the connections of battery modules. Still, it contains fewer modules and parameters to input, making it more operationally friendly than Butterfly.

Autodesk CFD’s automated meshing technology recognizes geometry and quickly divides the mesh, generating good-quality grid schemes for non-expert users. GH_Wind has also developed an automatic meshing function. However, Butterfly requires manual meshing by the user. In addition, to reduce the time spent in the pre-processing stages checking for geometric problems, Autodesk CFD automatically performs model checking and modification during meshing. However, Autodesk CFD lacks 3D modeling capabilities and requires models generated by specialized modeling software to be imported. In particular, Autodesk CFD provides users with cloud computing services to help them save computer resources when solving large projects.

Table 6. Comparison of CFD plug-ins integrated into architectural design platforms.

Architectural Design Platforms	CFD Plug-Ins	Simulation Categories			GUIs (or Not)	Simulation Capabilities
		Airflow	Pollutant	Thermal		Accuracy	Speed	Ease of Operation
Autodesk Revit	Autodesk CFD	√	√	√	√	★★	★	★
Rhino Grasshopper	Butterfly	√				★
	GH_Wind	√					★★	★★

compares different CFD plug-ins’ advantages, disadvantages, and differences. The more “★,” the more advantages in this aspect.

3.3.2. Application Suggestions

Although the CFD plug-in simplifies the CFD method and workflow, it still requires users to have some basic knowledge of CFD to avoid misleading simulation results [25,31,73].

Different turbulence models have little influence on the simulation speed, but they have an essential effect on the simulation accuracy [31,47,74]. When Butterfly is used for urban wind environment prediction, the RNGKE turbulence model can achieve better simulation accuracy. In addition, GH_Wind lacks a turbulence model, and the turbulence effect should be compensated by increasing the numerical viscosity [44,47]. According to Section 3, Butterfly and Autodesk CFD can obtain relatively accurate simulation results in predicting wind velocity at measurement points inside the urban block. In addition, the simulation accuracy of Autodesk CFD for the wake area of obstacles is slightly better than Butterfly. It is worth noting that the simulation accuracy of the three CFD plug-ins decreases with the increase in the inflow angle. It is speculated that the rotation of the building group makes the edges of buildings appear as a zigzag grid, which leads to a reduction in the grid quality [44]. Therefore, it is suggested that users encrypt the grid again in the target area after changing the inflow direction to improve the simulation accuracy.

GH_Wind often obtains inaccurate simulation results at low heights or pedestrian levels, but the simulation accuracy is significantly improved with the elevation of the monitoring plane. In this paper, even if the coarse mesh is used for GH_Wind, the simulation results generally agree with the verified CFD results on the horizontal plane corresponding to a 90 m height. Therefore, this paper speculated that GH_Wind is more suitable for the airflow simulation of high-rise buildings. In this case, the size of the global grid can be appropriately increased to save computational time. Notably, to improve prediction accuracy, Waibel et al. [75] proposed a coupled approach that combined FFD simulations and machine learning (ML) algorithms to predict wind pressure on the high-rise façade. Rapid simulation results were used as input for the ML model to provide many training samples within a limited time to improve the computational accuracy.

According to Ref. [73], part of the model information or data may be lost in importing the model into Autodesk CFD, especially for complex geometric models. Therefore, it is recommended to simplify the model before importing it into Autodesk CFD to reduce the complexity of the model. Also, users should check necessary data promptly after the model is imported. In addition, Butterfly and GH_Wind are both based on the open-source platform Grasshopper and have more secondary development potential than Autodesk CFD. Finally, for architects who want to perform fast and timely wind environments inside the urban block in the early stages of urban building design, Autodesk CFD and GH_Wind are architect-friendly regarding simulation speed and operation difficulty.

4. Conclusions

Rapid urbanization has caused some urban environmental problems. Therefore, researchers have begun to apply simulation tools such as CFD plug-ins to study the optimization of urban form to improve urban ventilation. However, few previous studies have verified the reliability of the simulation results obtained by CFD plug-ins, and few studies have evaluated the simulation accuracy of CFD plug-ins using wind tunnel experiments or field measurement data. Therefore, to clarify the credibility of CFD plug-in simulations and avoid obtaining misleading simulation results, this study compared and analyzed the simulation accuracy and efficiency of three CFD plug-ins on different architectural design platforms, to provide guidance and reference for urban planners and architects when choosing CFD plug-ins. The main research results are as follows:

• While the CFD plug-in simplifies the CFD method and workflow, it still requires the users to have some basic CFD knowledge. Several CFD plug-ins have developed clear GUIs and automatic meshing functions to help users quickly get started with CFD simulations. The FFD method was integrated into the architectural design platform to support the rapid airflow simulations.
• The structured hexahedral meshes generated by Butterfly are relatively easy to achieve mesh independence. The unstructured tetrahedral meshes automatically generated by Autodesk CFD can achieve mesh independence with a small number of the total meshes. However, the uniformly distributed voxelized meshes generated by GH_Wind do not support local encryption, making it a challenge to achieve grid independence.
• All CFD plug-ins underestimate the wind velocity at the pedestrian level inside the urban block in any wind direction. Butterfly had the best simulation accuracy under 0° wind direction. Autodesk CFD has the best simulation accuracy at 22.5° and 45° wind direction. With the increase in the incoming wind angle, the simulation accuracy of Autodesk CFD tends to improve, while the simulation accuracy of GH_Wind decreases continuously.
• The CFD plug-in generally has a low simulation accuracy for the leeward area of the obstacles at the pedestrian level, especially GH_Wind. Butterfly had good prediction accuracy in the strong wind regions, while its prediction ability was weak in the entrance and exit areas of the streets, but this problem improved with the increase in the inflow angle. Autodesk CFD maintains good prediction accuracy in low wind speed regions.
• Autodesk CFD achieves a good balance between simulation accuracy and speed. However, GH_Wind using the FFD method does not support a local adjustment of the grid size, which leads to a large number of generated grids and a sharp increase in computational time.
• Of the three plug-ins, Butterfly is the most difficult for urban planners and architects to learn and operate, while Autodesk CFD and GH_Wind are architect-friendly. The combination of the FFD method and the ML algorithm on an architectural design platform can effectively improve the simulation accuracy of GH_Wind.

This study analyzed the variation of CFD plug-in simulation accuracy and its rules under different inflow directions by comparing the simulation results of wind speed, to guide the validation and evaluation of the reliability, advantages, and disadvantages of a CFD plug-in simulation. However, this study has certain limitations, such as not considering and choosing the CFD plug-in on SketchUp for studying. In future studies, the authors will comprehensively evaluate the simulation accuracy and efficiency of different CFD plug-ins on different platforms from more aspects, to provide better support and suggestions for urban planners and architects to effectively study and improve urban environmental problems by using CFD plug-ins.