CGAN-Assisted Renovation of the Styles and Features of Street Facades—A Case Study of the Wuyi Area in Fujian, China

Abstract

With the development of society and the economy, the unified planning of architectural styles has become a significant problem in the balance between urban expansion and the protection of traditional buildings in villages and towns. This also allows people to re-examine the appearance of and quality of life, experienced by those in traditional village buildings. This research employs a conditional generative adversarial network (CGAN) to develop a generative technique for designing building facades in villages and cities. The provided results can be used to develop schemes and as design references for building facade design, enhancing the design efficiency of building facades. Simultaneously, we utilized this model for the rehabilitation of building facades in villages and towns, as well as in the visual design of rural tourism products, demonstrating its practical usefulness and design-related potential. We took villages and towns in the Wuyishan area of China as an example and carried out model training, image generation, and a comparison of the derivation results of different assumed buildings and product contours. The research shows that: (1) CGAN can be used to produce and supply reference schemes for conventional civil construction facade design in rural and urban areas. (2) In terms of adaptability, CGAN may develop architectural facade design schemes with a reference value for the hypothetical experimental building facades, and it can play a role in other design domains, as well. (3) The construction of this method is not only applicable to villages and towns in the World Heritage es Cities Programme, but can be further promoted and used in the future for cities and villages that have a demand for architectural style consistency.

1. Introduction

Global natural and cultural heritage are precious resources for mankind. In the face of the rapid urban modernization, large-scale modern buildings are emerging in an endless stream. It is also inevitable that a large amount of public transportation infrastructure, such as high-speed railways, need to be built, in order for a city to develop. In traditional villages within sight of high-speed railways, the coordination and unity of building facades has become a key issue. The unified management and control of the architectural styles of hundreds of villages and towns requires a lot of time and energy. Therefore, how to control the coordination and unity of traditional village architectural styles and improve the efficiency of architects, in the face of a large amount of repeated design work, is a question worth exploring.

This paper’s research objectives are as follows: (1) To address the genuine requirements of the Wuyishan region. In order to match the Wuyi Mountains’ position as a double world heritage site and the creation of national parks and national archaeological sites, China has a strong requirement for the street facades surrounding the Wuyi Mountains to have a uniform appearance. (2) To develop approaches for optimizing the design of the village and town architectural styles. In recent years, China has continued to provide substantial financial assistance for the Wuyi Mountain region’s architectural development. Due to the high number of buildings involved, it has been impossible to complete the design work in the allotted time, and a more efficient design process is required. (3) To make Wuyi style architecture more sustainable. The Wuyi region is home to an important regional architectural style known as the Wuyi style. A methodical and consistent design approach can enhance the regional features and material resources of the cultural tourism in the Wuyi Mountain region. This will allow designers to eliminate repetitive work. For a large number of simple, old buildings that are not difficult to construct, batch processing techniques, based on machine learning, can be utilized to develop a sequence of designs for experts to evaluate, transforming the design process from manual to man–machine collaborative design.

1.1. Research Background

The Wuyi Mountains are located between Wuyishan, in the Nanping prefecture in the Northwest Fujian province, and the town of Wuyishan, within Shangrao in the Northeast Jiangxi province. The mountain range is known worldwide for its status as the habitat for several rare and endemic plant species, its dramatic river valleys, and the abundance of important temples and archaeological sites in the region, and it is a UNESCO World Heritage Site [1]. There have been three movements, known as “Jiageng style”, “Wuyi style”, and the “regional high-tech building”, in the development of Fujian regional architecture, since the 1950s [2]. Around the Wuyi area with the Wuyi Mountains as the core, many buildings with regional styles have been built and a local architectural school in Fujian, China was formed. The narrow sense of the Wuyi style refers specifically to the buildings with the local characteristics of the Wuyi Mountains, and which were recreated by architects in the Wuyi Mountain area after 1979, by absorbing local traditional architectural styles and combining local natural and cultural resources (recognized as a UNESCO World Heritage Site) and specific environments with modern design techniques [3]. At the same time, the Wuyi style emphasizes the integration of architecture into nature, that is, the integration of architecture and the mountains, and thus promotes the formation of world heritage. The Wuyi style can meet the call for the modernization of vernacular architecture, and it is one of the representative works of the new vernacular architecture. The unification of its overall architectural style is conducive to the sustainable development of the world heritage city.

In architectural heritage-related machine learning research, machine learning methods are typically used for four components of architectural heritage: (1) To anticipate and restore missing architectural historical components. Using architectural heritage data as the input for machine learning, enables models to identify missing elements of the target architecture, to document damaged and destroyed architecture [4]. (2) The classification and processing of photographs of cultural heritage. During the duration of architectural heritage surveys, a vast number of photographs are classified and organized using machine learning techniques [5]. (3) Three dimensional point cloud data processing. Following the scanning of architecture with 3D laser equipment, a vast amount of point cloud data will be generated for the job of applying 3D digital technology, to conserve the architectural heritage [6]. Machine learning techniques may recognize architectural aspects, protection status, and building materials in point cloud data, allowing for a more efficient use and processing of the point cloud data. (4) The development of virtual architecture. In the Virtual Architectural History Project, the architectural components are tagged by utilizing machine learning techniques, to generate architectural examples in batches, utilizing semantic labels that have been annotated [7]. Thus, machine learning has been studied in numerous areas of architectural research. This study will continue to investigate the feasibility of using machine learning in this field. To protect the architectural style and the characteristics of the villages and towns surrounding the Wuyi Mountain region, a method of identifying and generating building facades is proposed, as well as a method of machine learning to aid in the architectural facade design.

1.2. Literature Review

There are many design practices in China aimed at the renovation and restoration of traditional architectural styles. Representative styles are: the characteristic Dali ethnic towns in Yunnan [8], the historical towns in Hubei Province [9], the historical landscape along the river in Jiangsu Province [10], the traditional village landscape in Henan Province [11], and the historical and cultural district in Shanghai [12]. However, architects have to execute all of these schematic designs, and spend a lot of time and effort doing so, potentially duplicating unordered work. Although the style of traditional buildings can guide the transformation of urban landscapes, it also challenges architects to comprehensively consider the transformation plan, from the perspective of understanding history, and presenting it in the design. It is a time-consuming task for architects to gather local information on a macro-city scale and organize important information. Furthermore, dealing with buildings with different urban backgrounds in the face of different historical areas requires more expertise and experience [13].

It is inevitable that in architectural design, individuals have diverse understandings of architecture. The design process will subtly add personal preferences, and this spontaneous development often leads to the loss of the regional cultural characteristics of the building, thus affecting the coordination of the overall style of the streets. Therefore, a design method, based on the objective information, is needed to reduce the influence of individual design on the street style [14]. Traditional architectural design is a time-consuming and labor-intensive process. With the advancement of computer technology, machine learning, as the core component of artificial intelligence, it provides innovative methods for the above process in the following ways: (1) Image recognition quickly identifies elements in architectural images, thereby leading to the batch extraction and evaluation. (2) Image generation and image conversion efficiently generate new architectural images and perform style transfers on existing buildings. The above methods have shown a great potential in the field of architecture, such as in building facade generation [15], building volume generation [16], building floor plan optimizations [17], street view quality assessments [18], street view image generation [19], and architectural style migration [20]. All of these applications can greatly improve the efficiency of the building evaluation and building design.

In the field of building facade generation, the current research is mainly based on generative adversarial networks (GANs) [21]. This method artificially marks the elements of the building facade with color blocks, as a color block map, and then carries out the confrontation training between the building facade image and the color block image, to generate a building elevation map through the color block map [22,23]. This method is suitable for the stylistic assessment and facade restoration of existing buildings. However, in the architectural design stage, how to guide the design of the building facade, generate the building facade through the given design scope, and conform to the coordination of the surrounding style of the site, still needs in-depth research.

1.3. Problem Statement and Objectives

In October 2016, the Nanping municipal government clearly stated that the environment and landscape, along the high-speed rail and expressway, are key in displaying the urban and rural features and civilization of Northern Fujian. This is an important part of the construction of a national green development demonstration zone, which requires the improvement of the landscape along the high-speed rail, expressway, 303 expressway, and light rail (Figure 1).

Nanping City has also issued several policy documents to create landscapes with Northern Fujian characteristics. The Wuyishan municipal government has also set up a leading group to strengthen the environmental improvement and landscape construction. The remediation scope includes the high-speed rail, expressway, 303 expressway, and the light rail within Wuyishan city. The planned expressway is divided into three sections, namely: (1) The G1514 expressway, from the junction of Fujian and Jiangxi (279) to the Xingtian hub (232), a total of 47 km. (2) The G3 expressway, the Wufu Section (Tangbian to Chatouling), a total of 12 km. (3) The S0311 expressway, the Xiaba hub connecting to Hongqiao Village, for a total of 10 km. All of the villages in the visible area and the rural landscape along the line that affect the villages’ architectural styles are the targets of remediation (Figure 2).

Due to the large range of villages to be rehabilitated, it is necessary to maintain the consistency of the traditional urban dwellings. This study explores the technical methods of machine learning, to assist in this.

2. Materials and Methods

2.1. Methodology

This paper uses a method, based on CGANs for the style transfer from the local architectural style to the village and town building facade, to provide the design, decoration, and the restoration of the building facades to the village and town architectural style (Figure 3). This has an important practical value for the unified-style governance of large-scale and multi-building villages and towns. The method of this study consists of five aspects: data collection, data processing, model training, model evaluation, and data application. Please refer to Appendix A for the computer environment of model training.

(1) Data capturing. First, it is necessary to clarify the positioning and goals of the village and the town street style renovation, i.e., what kind of village and town do we want to design. In this study, through several field investigations and village interviews, we learned that the target area has a profound regional culture. The Wuyi style is a characteristic architectural style in this area; it is mostly found in tourist areas, and has not reached economically underdeveloped villages and towns. Therefore, the researchers photographed 164 Wuyi style buildings in the tourist area where Wuyi style buildings were concentrated as samples for the experiment.

(2) Data processing. As a cutting-edge machine learning model, a CGAN has powerful image information processing capabilities and realistic generation effects. However, the training of a CGAN is often accompanied by problems, such as the difficulty in convergence and uncontrollable results [24]. To reduce the training time of the model and to improve the accuracy of the model, we redrew the collected samples into three categories of pictures: building exterior profile (BEP), functional segmentation layout (FSL), and building elevation (BE).

(3) Model training. In this step, the conditional adversarial generative network is trained with paired images. Therefore, the three types of pictures produced after the data processing are, respectively, trained from the building exterior profiles to the functional segmentation layout, and from the functional segmentation layout to the building elevation, corresponding to two weight models, models 1 and 2. These two models cover all of the data to generate the building elevation from the building exterior profiles, which can effectively achieve the experimental goals.

(4) Model evaluation. Once the training is completed, the weight model needs to be evaluated and tested, to determine whether the model meets the expectations of the experiment. By looking at the loss and pictures of the training process, you can make a basic judgment on the accuracy of the model. The CGAN loss value line chart shows the stability of the generator and discriminator during the training process. For better training results, the loss value will continue to decline and stabilize within a small period. In addition, pictures of the training process can be examined to see if the generator produces image noise. This kind of image noise is likely to fool the discriminator’s true and false judgment, thus showing a good result in the loss of value, but the actual effect of the generation is poor. Through the above two judgment methods, it is possible to control the model, in real time, during the training process. If the loss value continues to rise or the effect of the training process becomes worse and worse, the training will be stopped in time, and the training samples and various parameters will be readjusted.

(5) Data application. The model after training can be applied to many aspects of the remediation of villages, towns, streets, and alleys, such as the facade design of new buildings and the facade decoration and restoration of old buildings, and at the same time, analyze and judge the effectiveness and feasibility of these applications.

2.2. Materials Handling

The test material samples were collected by the researchers in the field. The buildings around the collection site conform to the regional style of the area, and included Wuyi Mountain Villa, Song Street, Jade Girl Villa, Jiuqu Hotel, and Wuyi Style Commercial Garden. The samples covered 32 building groups and 256 individual buildings, from which 164 representative facade pictures were selected. According to the different building heights and widths, uniform scaling is performed in 512 × 512 pixel images for registration.

The experimental materials are shown in Figure 4. According to the needs of the experiment, the facade picture is further divided into building exterior profile (BEP), functional segmentation layout (FSL), and building elevation (BE). There are 164 images in each category, for a total of 492 images. The BEP samples are grayscale images in black (R0, G0, B0). In the processing of the FSL samples, to simplify the data, various elements in the building facade are represented by different colors and presented in the form of color pictures. In this study, five colors are used to represent the elements in the facades: the doors are red (R255, G99, B128), the windows are green (R123, G132, B61), the walls are yellow (R243, G219, B181), the guardrails are blue (R3, G172, B213), and the roofs are brown (R171, G73, B64). These colors represent most of the content in the building facades. The BE sample is cleaned and simplified, the unnecessary advertising signs and pollutants on the facades are cleaned up, and the texture of the material is simplified.

2.3. CGAN Model

Conditional generative adversarial networks (CGANs) are one of the variants of generative adversarial networks (GANs). Consistent with the original GANs, the CGAN is mainly composed of two adversarial models: a generator responsible for generating pictures and a discriminator for judging the authenticity of the generated pictures. As shown in Figure 5, the main principles are: (1) The generator generates a fake image, based on the input image (Train A) and a random vector (Z). (2) The discriminator determines another set of corresponding pictures (Train B) and the random vectors as true pictures. At the same time, it discriminates against the fake pictures input by the generator; the real pictures are marked as 1, and the fake pictures are marked as 0. (3) If the generated image is judged to be false, the discriminator returns the deviation value between the fake image and the real image, to the generator. It then upgrades the generator so that it can generate pictures that are closer to the real thing. On the contrary, if the discriminator judges that the generated image is real, the discriminator will continue to learn from the training set to improve the recognition ability. (4) Through adversarial training, the generator can finally generate fake and real pictures, so as to achieve the goal of generating building elevations.

3. Results and Discussion

3.1. Training Process

The loss of the generator and discriminator of each generation in the model training process were counted (Figure 6). Model 1 represents the weight model, from the BEP to the FSL. Model 2 represents the weight model from the FSL to the BE. Following 200 generations of training, the loss gradually decreases, and the fluctuation stabilizes in a small range. This shows that the model is basically fitted and the training results are good. The loss fluctuation of Model 2 is smaller than that of Model 1 and the training is basically stable around the 80th generation. However, Model 1 still has significant fluctuations around the 130th generation, indicating that the training from Model 1 is more difficult and more complicated.

The weight model of every 50 generations, in the model training process, is generated into pictures, to give the results shown in Figure 7. Overall, the images generated during the training process are almost identical to the real images, with a high accuracy. Model 1 is not as effective as Model 2, and in the results generated by the 100th generation of Model 1, the windows on the building facade have obvious errors. However, the 200th generation results are all up to the real level.

3.2. Model Assessment

We randomly selected 20% of the images (99 images) from all images (492 images) in the training set, as the test set to test the trained model. As shown in Figure 8, buildings A to C are the test results of Model 1, and buildings D to F are the test results of Model 2. The generated results are relatively accurate, and the ideal results can be perfectly predicted with small errors. In Model 2, the generated results can not only generate doors, windows, railings, roofs, and other components according to the functional division of the FSL, but it is also possible to additionally generate facade styles and decorative lines, reflecting the local style of the building.

3.3. Model Application

Once the model training is completed, it can be applied to our project, regarding the village street style renovation. In this study, the project has three main problems to solve: (1) How to preserve the regional style in the design of the new building. (2) How to modify the exterior decoration of unadorned structures, according to the existing door and window openings. (3) How to restore unfinished buildings, according to the regional style. In addition, we propose that the model can be applied to some cultural products in villages and towns, such as road signs, house numbers, and souvenirs.

3.3.1. Application in the New Building Design

As shown in Figure 9, the feasibility of the application of a new architectural design is explored through the newly drawn six BEP samples. Figure 9A–C represent the impact of the building extension on the building’s facade design. Figure 9D explores whether the machine can divide floors according to the height of the building. Figure 9E,F show some of the building outlines commonly found in the site. The above samples are generated using Model 1. It can be seen from the overall generated results that the effect of the model generating the FSL is general, and there are obvious errors. Except for image D2, the rest of the images have different degrees of offset. However, the position of each component of the building facade can be roughly judged, and further manual processing is required. In addition, we also found that the machine can adjust the FSL, correspondingly, according to the change of the building control, and the result has a certain reference value.

The results generated in Figure 9 are manually processed and then input into Model 2 for the generation (Figure 10). Overall, the results are reasonable and there are no obvious errors. The machine can adjust the final BE, according to the size of each different function in the FSL.

(1) Automatically adjusting the style of the window. As shown in Figure 10A1,A2, the machine can generate single-opening windows and triple-opening windows, corresponding to the different sizes of windows, and the triple-opening windows are also equipped with shutters on both sides, which conform to the characteristics of the regional styles.

(2) Automatically adjusting the style of the door. In Figure 10A2,B2, it can also be seen that the machine generates single and double doors, according to the different dimensions of the door. The materials of the doors, generated by Figure 10B2,C2, are also significantly different. Figure 10B2 is a grid door, and Figure 10C2 is a wooden door.

(3) Automatically increase the wall decoration. All of the generated pictures show that not only are white painted facades generated, but the line decoration of the plinth and the edge of the wall are also generated, which improves the aesthetics of the building facade.

(4) Automatically generate guardrails. As can be seen from the generated pictures with guardrails, although the color blocks of the guardrails are independent in the FSL, the generated guardrail renderings are not isolated, but are fused to a certain extent, according to the positions of the doors and windows.

(5) Automatically adjusting the style of the roof. In Figure 10B2,E2, it can be seen that the machine generates pitched roofs with Howe style truss decorations and dark tiled roofs for different roof shapes.

3.3.2. Building Facade Decoration

As shown in Figure 11, six main buildings on the site, Figure 11A–F, which lack facade decoration, were selected as examples. According to the openings of the doors and windows on the facade of the existing buildings, the FSL is drawn manually and used as a sample for the BE generation. As a result of the simplicity of the village’s architecture, the CGAN facade has no major advantage over the artificial design. It only takes around one minute to generate the results for the next six examples in large-batch design assignments, demonstrating its efficiency advantage. Additionally, due to the influence of the existing building facade form and elements, if it is generated completely, according to the status quo, the final effect is relatively common. Only turning the walls white and adding a small amount of facade decoration cannot fully reflect the regional style. In practical applications, in addition to directly generating the existing facades, it can be further artificially optimized, to add more facade decorations to strengthen the embodiment of the regional styles with respect to the economic considerations.

3.3.3. Unfinished Building Restoration

During the on-site investigation, it was found that there are still some unfinished buildings on the site, with only floor slabs and frame structures, and a lack of external walls. The generative design of the facade can be carried out, according to the general outline of the building. As shown in Figure 12, after the generation, according to the above method, the generation effect is better on the facades with a large volume and complex outlines (Figure 12B5). Moreover, a facade with a smaller volume and a simple outline has a more general effect (Figure 12A5). In the Fix-FSL step, the FSL with general effect can be manually adjusted, which is beneficial to generating the optimal BE. Notably, the CGAN method determines the positions of the building facade openings, based solely on the angles of the ornaments. In practical projects, the window size, orientation, ventilation, lighting, privacy, and other building qualities and site conditions, must be thoroughly examined. Therefore, the generated results are only appropriate for a theoretical demonstration.

3.3.4. Product Application

In addition to the applications regarding the buildings themselves, based on the future tourism planning of the village, this study attempts to apply the model to the graphic design of tourism cultural products.

As shown in Figure 13, Figure 13A,B are common road signs and house numbers. Once these are generated twice, the result is messy and some text and icons are blurred. The overall style is consistent with the building, but some corrections and adjustments are required. Figure 13C,D correspond to the common souvenirs of fans and water cups. Once these are generated twice, the final effect is quite impressive. Except for some marks, these could be used as finished products almost immediately. The generation of the Wuyi style architecture-related rural tourism products may greatly promote the Wuyishan World Heritage culture.

The tests of the above two types of products show that, for road signs and house signs with text, the effect of the model generation is general. However, for the fans and cups, which have a more creative potential, the effect of the model generation is better. Because the graphic design of more artistic products is more intensive, the model can focus on the further application of such products, which have a significant value potential.

4. Conclusions

With the development of the economy and higher education, citizens and governments at all levels have paying more attention to the style of villages, towns, streets, and alleys, as well as to the quality of life of the citizens. This study took a village and town project in China as an example, to build a method of building facade generation in villages and towns, based on a CGAN model. This was carried out with the aim of meeting the needs of local governments. We drew the following conclusions:

• Strengths: A CGAN has the capacity to efficiently generate building facades with a particular regional style in large quantities,
• Weaknesses: A CGAN is limited to the rudimentary image processing and cannot properly convey the regional style’s cultural qualities and local spirit,
• Opportunities: Increasing the training materials and enhancing the model design of the CGAN could enhance the effect of the building facade creation; this would require additional study. The technique also proved useful in other design-related domains,
• Threats: A CGAN cannot operate effectively without the required quantity of training materials. A CGAN cannot substitute human labor for architectural cultural expression-intensive projects.

This method has significant efficiency advantages for the mass generation of hundreds of village buildings, and the generation effect is ideal. In addition to the generation of building facades, the product graphic design, especially of the common tourist and cultural products, such as fans and water cups, shows a significant potential.

In this study, the collected building facade pictures are subdivided into three types of pictures, i.e., BEP, FSL, and BE, as training materials. The combined application of Model 1 and Model 2 led to the ability to directly generate the BE from the BEP, which further expands the application scope of the building facade generation in machine learning.

In this study, limited by the number of samples of the CGAN model, the generated effect is still insufficient. In the future, we can continue to increase the samples of the training set, and specifically optimize the CGAN model to make up for the shortcomings of the experiment.

This study takes traditional Chinese villages and towns as an example to thoroughly examine the cultural value of the site. Extracting the Wuyi style under the influence of the World Natural Heritage and Cultural Heritage List as the target, has reference value in the value orientation of the village style control. For urban and village projects with the same style of management and control needs, the research method in this paper can be applied. Combined with the method of machine learning CGAN model-assisted building facade design, it can inspire more application scenarios for architects and related practitioners, leading to improving the interdisciplinary cooperation and work efficiency.