Three-Dimensional Documentation and Reconversion of Architectural Heritage by UAV and HBIM: A Study of Santo Stefano Church in Italy

Abstract

Historic buildings hold significant cultural value and their repair and protection require diverse approaches. With the advent of 3D digitalization, drones have gained significance in heritage studies. This research focuses on applying digital methods for restoring architectural heritage. It utilizes non-contact measurement technology, specifically unmanned aerial vehicles (UAVs), for data collection, creating 3D point cloud models using heritage building information modeling (HBIM), and employing virtual reality (VR) for architectural heritage restoration. Employing the “close + surround” oblique photography technique combined with image matching, computer vision, and other technologies, a detailed and comprehensive 3D model of the real scene can be constructed. It provides crucial data support for subsequent protection research and transformation efforts. Using the case of the Santo Stefano Church in Volterra, Italy, an idealized reconstructed 3D model database was established after data collection to preserve essential resources such as the original spatial data and relationships of architectural sites. Through the analysis of relevant historical data and the implementation of VR, the idealized and original appearance of the case was authentically restored. As a result, in the virtual simulation space, the building’s style was realistically displayed with an immersive experience. This approach not only safeguards cultural heritage but also enhances the city’s image and promotes tourism resources, catering to the diverse needs of tourists.

1. Introduction

Architectural heritage encompasses three key categories: urban heritage, landscape heritage, and architectural heritage. Each of them are imbued with cultural, material, and social significance [1,2]. Not only does architectural heritage embody a local area’s unique cultural essence, but it also serves as a vital conduit for preserving beliefs, documenting local characteristics, and chronicling societal norms and customs, thereby playing a pivotal role in the ongoing evolution of human society [3,4]. In recent years, architectural heritage, being a finite resource, has encountered numerous challenges. These include degradation stemming from social urbanization, deterioration and collapse induced by natural disasters, and inappropriate alteration of materials and color schemes due to anthropogenic development and conservation efforts [5]. Since the release of the UN’s Sustainable Development Goals (SDGs) in 2015, the world has been explicitly called upon to make efforts to protect cultural and natural heritage [6]. Furthermore, the UN’s New Urban Agenda 2030 clearly emphasized the importance of tangible and intangible cultural and landscape heritage; it also proposed that research on the application of digital technology to heritage conservation be encouraged [7]. It has been demonstrated that amid the era of rapid technological advancement, the integration of digital tools such as 3D scanning, modeling, VR, 3D point clouds, and other digital resources holds increasingly significant importance in the preservation and documentation of cultural heritage [8,9]. Furthermore, these technologies enable the recreation of historical sites that were previously unable to be fully represented [6]. Presently, this stands as the most effective method for showcasing lost artifacts without impacting the physical integrity of architectural sites, particularly through virtual mediums like digital displays [10]. An architectural site is an important type of artifact, and its value is expressed through the site itself. Reconstruction of destroyed buildings on the site is not allowed, and virtual methods can fulfill this requirement [11]. To accurately build the virtual restoration model, it is important to obtain accurate site data, which will affect the accuracy of the restoration model data [12,13].

HBIM, as a new paradigm for digital design and management [14], was proposed by Murphy in 2009 for the first time. The purpose of his research was to outline, in detail, the procedure of remote data capture using laser scanning and the subsequent processing required in order to identify a new methodology for creating full engineering drawings (orthographic and 3D models) from laser scan and image survey data for historic structures [15]. Coşgun et al. [16] defined HBIM as a parametric object prototype repository based on historical architectural data. With the deepening of the research and updates in the technology, Barrile et al. [17] stated that HBIM is a 3D computer model based on digital technology with the means to embed heritage asset data throughout the preservation lifecycle.

The traditional surveying method for architectural heritage typically involves data recording in various formats, such as manual measurements, textual descriptions, hand-drawn sketches, photographs, videos, and audio recordings. However, these methods lack efficiency, are labor-intensive, and yield more varied data records, which hinder subsequent data application and analysis [18]. During the process of establishing architectural heritage archives, surveying and mapping are the core components. Surveying and mapping are the basic premise of establishing archives [19]. Through the observation, measurement, and systematic documentation of architectural heritage, the acquisition of specific data and related information emerges as the cornerstone of heritage protection [20], establishing preconditions for subsequent protective measures. Without the data support obtained through surveying and mapping, research, evaluation, planning, design, and implementation of protection initiatives may be compromised. Thus, it is imperative to prioritize surveying and mapping to protect architectural heritage.

With the rapid development of UAV technology, it has been widely used in military affairs, agriculture, architectural heritage measurements, natural resource surveys, urban and rural planning, and other fields [21]. In the field of architectural heritage protection, when investigating and researching architectural heritage, it is often difficult to show the whole picture using manual surveying and mapping alone because of the complex surrounding environment of the building, and some detailed structures need to be photographed and measured by setting up a frame [22,23]. The surveyed and mapped buildings may be damaged due to the surveyors’ behavior. On the other hand, since the ancient buildings have been exposed to the natural environment for a long time and most of the main structures have problems, there will be certain safety hazards for the surveying and mapping personnel [24]. UAVs can perform close-range photogrammetry of ancient buildings, obtain detailed information in all directions, and make up for the limitations of ground photography modeling in terms of perspective, creating refined 3D models. They can accurately record the material information and damage of a building and are suitable for the recording of buildings or components of various scales. They can also be used for tilt analysis, like the results of 3D laser scanning.

Although the model generated by drone modeling is less accurate compared to 3D laser scanning, drones are low-cost, simple to operate, and can record the shape, color, and other information of the measured object in all directions. The application of drones can help conservators understand and present the characteristics and values of historical buildings more comprehensively and accurately while also providing the public with a more intuitive, vivid, and rich heritage visiting experience.

2. Literature Review

HBIM, as a specific subcategory of building information modeling (BIM), has received a lot of attention in the field of archaeology and architectural heritage since its implementation [25], importantly because of its significant breakthrough in modeling accuracy. The HBIM models developed by Brumana et al. [26] demonstrate a diverse spectrum of precision levels and geometric intricacies and are precisely tailored to the specific functional demands of each target. Moreover, Antón et al. [27] have further enhanced HBIM precision by leveraging remote sensing technologies. Additionally, other techniques such as laser scanning [28,29], computer science, GIS, 3D modeling, and 3D printing have been employed to meet the intricate modeling requirements of various architectural component shapes and complex surfaces [30,31,32,33]. In addition, HBIM, as a quintessential parametric design tool, facilitates element modifications through parameter adjustments, thereby enabling seamless modification and information interaction [34]. The HBIM modeling workflow typically integrates descriptive and reality-based data to document architectural heritage, as illustrated in Figure 1. This framework divides HBIM implementation into seven steps. The first step is initial data collection and processing, followed by point cloud generation, building block creation, and automation, culminating in the formation of the final HBIM structure.

2.1. Methodology of Modeling

Three-dimensional computer graphics was the first technology used to model heritage properties, often dedicated to virtual reconstruction [35]; combining it with BIM techniques gave birth to HBIM. Wang et al. proposed a new approach to HBIM that allows for virtual reconstruction based on the collection and processing of laser/image measurement data [36]. Virtual reconstruction usually refers to a virtual replica of a heritage object (disappeared, damaged, or present), where the data sources may come from digitized historical materials documenting the heritage, reality-based photographs, and measurable drawings representing surfaces. Thus, virtual reconstruction may resurrect the past, virtualize the present, and predict the future. Yang et al. [37] have discussed how to solve the problem of 3D modeling of historic buildings and apply it to HBIM documentation and analysis.

As a 3D information modeling environment, BIM software increasingly supports 3D point clouds representing solid surfaces, which is called the scan-to-BIM process [38]. In the last decade, reality capture technologies that generate 3D point cloud data have become increasingly sophisticated and available, enabling more accurate and cost-effective point cloud data acquisition. This means that the points in three-dimensional space defined by X, Y, and Z can accurately characterize the exterior surfaces of architectural objects [36]. Three-dimensional point cloud data can, in turn, be obtained by means of laser scanning, 3D computer graphics, and videos, among others. This semi-automated modeling method has been studied several times to obtain a certain accuracy [39,40]. The accuracy of the model was enhanced by combining other methods such as using NURBS surfaces and curves to solve more complex building component shapes [41]. However, this semi-automated modeling still requires some manual effort to improve its efficiency, and the process is time-consuming, tedious, and laborious as well as somewhat subjective. Therefore, many scholars’ studies have focused on developing a system/process that takes the point cloud of a facility as input and combines it with other techniques to automatically generate a fully annotated as-built BIM of the facility as output. For example, Pađen et al. [42] proposed a workflow for automatic reconstruction of simulated 3D city models by combining 2D geographic datasets and elevation data based on aerial point clouds. Moyano et al. [43] explored the insertion of point cloud data into a BIM environment, classifying the data and applying digital twin probabilities for automatic modeling of HBIM. The use of artificial intelligence and deep learning to improve modeling accuracy and automated modeling will be a research trend in the future [44,45].

2.2. HBIM and Drone Technology Modeling

The emerging field of drone technology as a digitization methodology offers new perspectives on digitizing and preserving architectural heritage [39]. A study by Zaker et al. [46] emphasized the importance of drones in the documentation of architectural heritage. Drones can carry various sensors, such as LiDAR and cameras, which are used to acquire high-resolution 3D data from buildings [47]. Drones offer significant advantages over traditional surveying methods. Firstly, they can measure large areas in a shorter period, thus increasing efficiency. Second, UAVs can perform non-invasive measurements, reducing damage to buildings [48]. This is particularly important for old and vulnerable built heritage. Liu et al. [49] highlighted the complementary nature of drone data and HBIM modeling. High-quality 3D data acquired by drones can be directly integrated into HBIM to create a more comprehensive record of built heritage [50]. This integration provides not only the geometric data of the building but can also include its exterior, structural, and material characteristics. This provides more accurate information for the maintenance and restoration of built heritage, helping to reduce unnecessary damage and costs.

UAVs can also meet different measurement angle requirements and greatly enrich modeling data. Özdemir and Remondino [51] used aerial RGB orthophotography to measure architectural objects and combined it with GIS to obtain detailed object information. Moreover, in this paper, we adopt the “close + surround” tilt photography technique, using a UAV for data collection, create a 3D point cloud model from UAV data, and construct a detailed and comprehensive 3D model of the actual scene using VR technology, image matching, and other techniques for architectural heritage restoration.

3. Materials and Methods

3.1. Study Area

Volterra city in Italy was established more than 3000 years ago and contains historic sites dating back to the fourth century BC. It has been tentatively recognized as part of the World Heritage List since 2006. Being one of the oldest cities in the world, it is also home to the world’s oldest standing Etruscan arch, as seen in Figure 2. The authors chose the Santo Stefano Church, located on the edge of Volterra city, as the case study. It was built in its present form in 1784 on an ancient medieval church dating back to the 11th century. However, it is heavily damaged; only the lower part of the façade and part of the side walls remain from the original phase. The church has three access portals, of which the central one is the largest. The jambs are made up of red stone pillars and support an architrave. They are surmounted by capitals decorated with different decorative motifs, as is typical of Romanesque churches. The architraves are also decorated. Today, only the portal on the right, which has a floral motif, has been preserved, as shown in Figure 3. The façade’s high degree of decorative accuracy, which is indicated by the use of numerous materials to create a chromatic effect of strong impact and by the architectural articulation, undoubtedly informs us of the wealth and prestige enjoyed by the church during the Middle Ages.

Many architectural heritage sites have fallen into disuse due to many economic, technological, social, and cultural factors. A lack of documentation is an essential factor that makes restoring architectural heritage difficult. Architectural restoration often requires detailed documentation and records to understand the history of the building, its structure, its design intent, and its original materials. The present study case, which can be traced back to the 12th century, is of great significance for the study of architectural heritage in the Romanesque style. This case’s point cloud data can provide ideas and reference points for less costly restoration work. At the same time, this low-cost way of restoring architectural heritage helps to preserve and pass on the architectural and cultural heritage of a community, country, or region; these buildings usually carry traces of history, and through restoration, the buildings act as witnesses to history. Restoration of architectural and cultural heritage contributes to the preservation of the common cultural heritage of humanity, the promotion of cultural diversity, and the sustainable development of society.

3.2. Workflow

The structure and appearance of Santo Stefano Church have changed greatly after hundreds of years of wind and rain. Without drawings and historical documents, it is even more difficult to trace its history. In addition, architectural heritage itself has a certain function of cultural heritage and artistic edification, and both carving details and color details need to be accurately restored. Traditional surveying and mapping methods lack precision, require a lot of manpower and material resources, and are more likely to damage the building itself during the process. Therefore, this research focuses on using digital technology to protect and revitalize the Santo Stefano Church in the city of Volterra.

Drones are mainly used as aerial photography tools for obtaining the orthographic projection of building roofs and orthographic projection photos of buildings and surrounding environments. They are an auxiliary tool for building information collection in areas with low visibility and accessibility and for spatial analysis. The church was originally a single-family Romanesque building. After demolition and damage, the surrounding urban residences have undergone many expansions and repairs, and now they are closely connected with the church. Therefore, photos taken using the orthographic projection capabilities of the drone can not only identify the details of the main body of the building remains but also enable the relationship between the entire building and the surrounding environment to be studied.

In this study, we chose to use Mavic 3 Pro UAV equipment to conduct low-altitude photogrammetry for the research case. The drone has a 24 mm Hasselblad camera, a high-precision anti-shake gimbal, and a 20-megapixel image sensor. The photo formats that can be taken are JPEG and RAW. The maximum flight speed is 20 m/s, and the battery can last for about 30 min. In this photogrammetry project, 200 UAV remote sensing images were obtained, with a heading overlap rate of 80% and a side overlap rate of 70%. The UAV flight height was set to be two meters above the building. The three main steps are data acquisition, information management, and application. They will be implemented through a suite of work packages fulfilling the following workflow (Figure 4).

In the first step, control point measurement is integral to the UAV low-altitude photogrammetry technology. A GPS is used to measure the control points in the research area. To ensure the accuracy of the data, the control points are evenly distributed throughout the research area. Therefore, five control points are arranged at the four corners of the surveying and mapping object, targets need to be placed, and the color of the targets must be obvious to ensure that the center of the target can be clearly seen during the information collection process. To measure the target coordinates, it is better to use a prism-free electronic total station to ensure accuracy. RTK (real-time kinematic) measurement technology may produce large errors due to small-scale measurement, and the accuracy could be better, so it is not recommended for use. For investigating the weather environment in the aerial survey area, researchers should consider whether there are any obstacles that may affect the safe flight of drones in the surrounding area and whether there are any interference sources that may affect information transmission. They should also choose a cloudy day with better light to avoid overexposure of the lens and distortion of the picture caused by direct sunlight, significantly increasing the workload and difficulty of image processing, and select the operating point to provide a reference for UAV aerial survey planning. This is then followed by formulating the aerial survey plan, determining the UAV’s take-off and landing point, and setting the route based on the UAV parameters. After setting the route, the test flight should be carried out first, and the aerial survey should be officially started after the relevant parameters are calibrated. During the aerial survey, the researcher should pay attention to the drone’s power and flight attitude to ensure that it can return safely. A stepped route can solve the problem of different heights in the shooting area. When shooting, the height difference of each area changes as little as possible. For this project, the vertical distance between the UAV and the church façade is two meters, and it flies vertically along each façade from bottom to top. Figure 5 shows a working process by which the Santo Stefano Church data was scanned by UAV technology.

After capturing images, it is essential to review them promptly for missed shots. Subsequently, the images are uploaded onto a computer for processing (Photoshop, v.24.7.1). It utilizes batch processing to standardize parameters (saturation and brightness) while eliminating flawed images. Next, the data are transferred to the specialized software Pix4Dmapper 4.6.1 for comprehensive UAV data and aerial image processing. It efficiently processes hundreds of images, minimizing manual intervention. The software automatically identifies and reads the data after importing the JPG aerial photos and corresponding coordinates of the control point into Pix4Dmapper. After setting parameters for feature point extraction, product specifications, and contour line parameters, Pix4Dmapper generates orthophoto maps, point cloud data, digital surface models (DSM), and digital terrain models (DTM) with high precision and accuracy. Finally, the point cloud data obtained from the aerial photography data are processed as follows (Figure 6).

4. Results and Discussion

HBIM is a technical means of heritage building information management. The standard parametric component library is built through a cross-platform program that maps point cloud and photographic modeling data to parametric components based on existing historical building data. Compared with traditional two-dimensional information management methods such as text, pictures, and drawings, HBIM can express the original appearance of buildings more intuitively and accurately and create three-dimensional records of non-geometric data, improving data management and providing information on the current performance of architectural heritage.

4.1. Acquisition and Recovery of Individual Object Data

For this article’s research case, the Santo Stefano Church, the authors analyzed the point cloud data model obtained by the drone. The model can not only analyze and evaluate the status of each part of the remaining building but can also assist in investigating the structural problems of the current building, including crooked wall flashing, tilting, foundation subsidence, etc. Using HBIM technology, we were able to establish a parametric component library, build a building information model, input survey data, realize the visual management of Santo Stefano Church’s survey data, and use the output to draw the general plan, elevation, and structure of the building, a large sample diagram of nodes, and other archival information technology drawings. Figure 7 shows the creation of a family of HBIM models of the Santo Church from point cloud data. However, due to the lack of historical documents, this project’s restoration model is an idealized model based on the partially preserved wreckage.

The family library created based on point cloud data contains standard historical architectural elements, such as windows, doors, columns, etc., so that experts and scholars can reuse these elements during the conservation and restoration process. Figure 8 shows a built-up combined member based on the columns and beams of Figure 7. From repairing individual components to systematic restoration of progressively assembled elements, specialized personnel undertake manual repair processes, ensuring precision and accuracy. This meticulous approach contributes significantly to enhancing the HBIM database but also aids in preserving the style and coherence of historic structures. The model’s comprehensive family library is a repository for detailed information concerning historic building elements. This encompasses material specifications, historical context, maintenance records, and more. Such extensive documentation enables architects, engineers, and preservation experts to gain deeper insights into the unique requirements of historic buildings throughout the design and maintenance phases. At the same time, the family library of HBIM models can be used to create virtual simulations of various aspects of historical buildings, including structure, lighting, ventilation, etc. This helps evaluate the effectiveness of retrofit design options while maximizing the preservation of the historic building’s original appearance. In general, the model family library created based on point cloud data plays a crucial role in HBIM, helping to protect and maintain these precious cultural heritages by standardizing, sharing, and managing the information.

4.2. Partial Restoration of the Church

Expert technicians meticulously engage in the manual repair processes, methodically addressing individual components before systematically restoring progressively assembled elements, guaranteeing utmost precision and accuracy. This painstaking methodology significantly enriches the HBIM database and plays an instrumental role in safeguarding the integrity and authenticity of historic structures. Within the HBIM framework, an exhaustive family library stands as a veritable treasure trove of detailed insights into historic building elements (Figure 9). This repository encompasses a wealth of information ranging from intricate material specifications and nuanced historical contexts to meticulous maintenance records and beyond. Such comprehensive documentation is invaluable, empowering architects, engineers, and preservation experts to delve deeply into the intricate nuances and unique demands of historic edifices at every stage—from initial design conceptualization to ongoing maintenance endeavors. Through this holistic understanding, practitioners can navigate the complexities of preserving and revitalizing these architectural treasures with heightened sensitivity and expertise.

Without historical materials and archeology, surveying and mapping data and current models should be fully utilized to carry out reasonable and hierarchical restoration design based on the building’s traces and other architectural styles of the same period. The idea of restoration is to divide it into two parts, the well-preserved part and the poorly preserved part, according to the overall preservation status, and to restore the better-preserved part before the poorer-preserved part. Individual pieces of the brick decoration of the church of Santo Stefano, found in various locations and often in multiple pieces, are being recorded in 3D using photogrammetry. They are being virtually restored to their original location in the 3D model of the structure (Figure 10). Due to the lack of historical documents in this case, apart from the remains that can be restored to an identical HBIM model, such as the data on the upper part of the façade, we can only estimate the original appearance of the entire church based on the style of the remaining parts and by using churches of similar style from the same period as reference cases for restoration.

4.3. Application of VR Tools

VR offers a promising solution by integrating 3D UAV data with HBIM to provide users with immersive experiences. Users can explore indoor and outdoor architectural heritage sites within virtual simulation spaces, including virtual reconstructions of incomplete areas, fostering a deeply immersive experience. HBIM realizes the integration of all-round information on historical buildings. Therefore, the combination of HBIM and VR can provide a visual reference for the restoration and protection of architectural heritage. Unity 2019.4 is a game engine software that effectively realizes this link using Revit 2023.1 software. Unity 3D supports multiple virtual reality devices: HTC Vive, Oculus Rift, and PlayStation VR, and provides rich interactive control options. It also supports technologies such as head tracking, spatial sound effects, and high-quality graphics rendering to improve the realism and immersion of the virtual environment while focusing on performance optimization to ensure smooth operation. The workflow of interactive virtual architectural heritage based on Unity mainly includes the following steps: realization of interactive functions, design of dynamic navigation system, application of intelligent interactive technology, use of an AI-driven virtual tour guide, and eye tracking. During the VR roaming process, relevant building information can be queried in real time, edited online, and the real-time effects displayed in a visual way, as shown in Figure 11a,b.

This approach enhances the preservation of cultural relics and diversifies the dissemination of the city’s image and tourism resources, catering to a wide range of user interests. The virtual restoration of architectural heritage enhances the city’s image and boosts tourism resources through online promotion. Additionally, the significant human–computer interaction capabilities facilitate research in tourism planning and drive the advancement of virtual tourism.

Furthermore, the integration and standardization of diverse cultural relic resources through computer networks enable virtual technology to depict cultural relics comprehensively and realistically, transcending geographical constraints and facilitating resource sharing. This strategy enables cultural relics to reach broader audiences and fosters a deeper appreciation for heritage preservation efforts.

5. Conclusions

Architectural heritage is a crucial part of cultural heritage, and its preservation and recording are essential for transmitting historical, cultural, and artistic values. This paper focuses on the use of drone close-range photogrammetry technology to collect basic information about some architectural ruins that cannot be repaired and restored. In the case of Santo Stefano Church, drones and HBIM are used to improve the efficiency of recording and conservation of architectural heritage, thereby reducing labor and time costs. Through data collection, data processing, HBIM model creation, virtual restoration of the original plan, and VR application, the church’s original façade effect was restored. This technique not only provides digital protection data for architectural heritage but also meets various needs, such as tourism resource promotion, heritage cultural education, etc. The results show that drones are powerful tools for acquiring high-resolution three-dimensional data of buildings in a way that is more efficient than traditional surveying methods and avoids damaging contacts. As computer science continues to develop, related digital technologies will become more available for heritage studies. In the foreseeable future, by combining technologies such as 3D printing, big data, AR, artificial intelligence, robotics, and computer vision, the digital protection means and methods of architectural cultural heritage will become more systematic, which will surely promote the informatization of architectural cultural heritage and provide more innovative methods for architectural heritage protection.