Combining Multiband Imaging, Photogrammetric Techniques, and FOSS GIS for Affordable Degradation Mapping of Stone Monuments

Abstract

The detailed documentation of degradation constitutes a fundamental step for weathering diagnosis and, consequently, for successful planning and implementation of conservation measures for stone heritage. Mapping the surface patterns of stone is a non-destructive procedure critical for the qualitative and quantitative rating of the preservation state. Furthermore, mapping is employed for the annotation of weathering categories and the calculation of damage indexes. However, it is often a time-consuming task, which is conducted manually. Thus, practical methods need to be developed to automatize degradation mapping without significantly increasing the diagnostic process’s cost for conservation specialists. This work aims to develop and evaluate a methodology based on affordable close-range sensing techniques, image processing, and free and open source software for the spatial description, annotation, qualitative analysis, and rating of stone weathering-induced damage. Low-cost cameras were used to record images in the visible, near-infrared, and thermal-infrared spectra. The application of photogrammetric techniques allowed for the generation of the necessary background, that was elaborated to extract thematic information. Digital image processing of the spatially and radiometrically corrected images and image mosaics enabled the straightforward transition to a spatial information environment simplifying the development of degradation maps. The digital thematic maps facilitated the rating of stone damage and the extraction of useful statistical data.

1. Introduction and Background

A substantial fraction of the historic built environment is made of stone. Thus, stone monuments are an integral part of our heritage and our societies’ continuity. Moreover, their presence affects everyone because of the essential historical values they possess tied to religion, culture, aesthetics, and building techniques of their construction period. Therefore, preserving stone monuments is critical in maintaining an essential historical resource and upholding a societal identity vital for future generations.

Natural stone materials (as, for example, limestone and marble) suffer from complex weathering processes that lead to degradation [1,2], which imposes significant challenges to ensuring the preservation of the historic build environment. Weathering of building and construction stone materials depends on intrinsic and extrinsic factors, which interact, causing degradation at different rates [3]. Environmental elements (past and present), anthropogenic parameters, physical properties of stone, and previous conservation interventions are all factors that contribute to stone-built heritage weathering [4,5,6]. However, understanding the weathering processes of stone is an intricate task that requires continuous interdisciplinary monitoring of the above parameters [7] and proves essential for the decision making regarding suitable and compatible conservation interventions [8].

Documentation is a prerequisite for any investigation aiming to reveal information about a historic monument’s preservation state and precedes the weathering diagnosis. Essentially, documentation serves a crucial role in monitoring degraded stone surfaces. It provides the necessary background for analyzing the condition of stone materials, both in terms of acquiring and evaluating comprehensive anamnestic information and creating the spatial reference upon which interdisciplinary information can be effectively integrated.

Mapping the surface patterns of stone has long been established as a non-destructive technique for the detailed documenting of lithotypes and degradation [9]. Mapping is performed for various materials and produces data that can be processed through computational systems to support and accelerate the documentation work. More specifically, the degradation patterns can be identified—supported by visual inspection—and mapped in a computer-assisted design (CAD) or geographic information systems (GIS) environment at different scales. The derived thematic data (spatial entities with semantic annotations) have spatial attributes, in a relative or absolute reference sense, and can therefore be spatially referenced, facilitating spatial query execution, visualization, and calculation of statistical quantities (quantitative valuation) to be performed [10]. In addition, mapping is a relatively cost-effective technique and, when combined with other on-site investigations, can reduce the cost of diagnostic and conservation procedures by assisting the planning of sampling and laboratory testing [11].

1.1. Scope and Research Aims

Stone heritage degradation mapping is a procedure often performed manually, which proves very labor intensive. Having identified a lack of affordable, agile, and relatively uncomplicated solutions to perform degradation mapping for stone conservation purposes in digital environments, this work focuses on recommending a combination of techniques to tackle the above issues. A methodology integrating low-cost sensing techniques and geomatics tools is proposed to produce digital thematic maps of degradation cost-effectively. The proposed methodology utilizes GIS software to allow the quantitative evaluation of degraded stone surfaces through spatial and statistical analysis and the representative visualization of the preservation state.

1.2. Article Structure

This article is structured into five sections. Section 2 delivers a review of relevant literature. Section 3 provides an overview of the workflows followed, alongside specifications about the low-cost instrumentation, techniques, and software used. In Section 4, the case studies are briefly described, followed by a comprehensive presentation of the methods’ implementation and the results. Finally, the discussion of the results follows in Section 5, together with some concluding remarks.

2. Related Work

True color images, in particular rectified images and orthomosaics, provide a useful and rich-in-information background for performing manual-digitalization mapping procedures [12,13]. The background quality is important to acquiring quantitative degradation mapping based on visual information [14]. Consequently, the recording of suitable images and the generation of suitable ortho-derivatives is crucial, considering the required spatial analysis and the radiometry that will facilitate the digitalization of all surface patterns [15]. Photo-based and photogrammetry-based mapping allows thematic comparisons [16,17] and contributes to the calculation of damage indexes [10,18]. It helps estimate the cost and analyze the compatibility of interventions [17]. Mapping can be applied for degradation patterns induced by a wide range of weathering phenomena [19,20,21,22,23]. The analysis of the spatial association between different patterns and geomaterials leads to observations regarding the interactions that occur on the historical surfaces [24,25]. The mapping implementation provides significant insight into ongoing weathering processes, especially after supplemented with information from analytical techniques and environmental measurements [26,27,28,29,30]. Manual photo-based mapping is a technique widely used in architectural conservation.

An essential issue in semantic annotation of thematically mapped historical surfaces is the automation level [31]. Digital image processing (DIP)-based approaches have been implemented to automatize the mapping process, offering a time-efficient alternative to manual mapping procedures and supporting a more detailed, quantitative, and comprehensive evaluation of the state of stone preservation [32,33,34].

Recorded reflectance intensity data in the near-infrared (NIR) and thermal-infrared (TIR) spectra can improve the automatic classification of the surface patterns, and thus, multiband imaging has been evaluated in this direction [35]. The complementarity of these spectra stems from the dissimilar radiometric signatures that different stone materials, natural weathering forms, and moisture content levels display [36,37]. NIR reflectance imaging proves favorable for detecting vegetation or biodeterioration [38]. Combining true color and TIR imaging increases and mutually enriches their efficiency, particularly when thermograms are presented comprehensively as thermal orthomosaics produced by rectified or orthorectified images. Moreover, infrared thermography has extensively been used as a non-destructive technique for the condition monitoring of historical stone structures [23,37,39,40] that can reveal valuable information about moisture content, material incompatibility, and, in general, the subsurface heterogeneities not observable on optical images. Nevertheless, as a non-destructive technique, reflectance imaging has many defects, such as the cost, resolution, and calibration needs of sensors.

The combination of degradation mapping and multiband close-range recording with 3D technologies does not merely augment the mapping results from a thematic data management point of view [41]; it also enhances the multidimensional interpretation of weathering [42,43]. Moreover, geometrical characteristics, such as features caused by the material loss [44], cracks, and the distance from the surfaces’ original shape [45], can also be mapped to enrich the documented thematic information.

3. Methods and Materials

In this research, low-cost and easily implementable methods are presented for stone condition mapping of weathered architectural monuments. Taking into consideration the capacity of the approaches based on imaging, photogrammetry, computational visualizations, and spatial analysis tools to automatize the mapping process described in related work, hybrid workflows combining these techniques are followed. Figure 1 presents the overall methodology design, which branches out into two similar workflows depending on the observed deterioration forms. The two workflows differentiate only during the step of extracting the thematic information using DIP but otherwise follow the same procedure. The methodology starts with the data collection and correction, continues with the multiband data processing generating the necessary background data for the mapping process, leading to the development, and finally, the analysis of thematic degradation maps in free and open source software (FOSS) GIS environment.

3.1. Multiband Image Acquisition

The proposed mapping procedures employ low-cost imaging sensors to capture multiband data. All involved close-range recording techniques fall within the category of passive sensing, meaning that no external radiation sources are used, but rather that captured images are a digital representation of the distribution of reflected radiation coming from ambient sources. For this reason, the reflectance images were acquired with as homogeneous illumination conditions as possible. In addition, the utilization of low-cost cameras allowed increased versatility during acquisition [46]; however, extracting suitable radiometric information from the spectral images in order to perform any quantitative processing requires spatial and radiometric corrections [35].

True color (RGB) images were acquired with a commercial off-the-shelf (COTS) P30 smartphone (Huawei Technologies Group Co. Ltd., Shenzhen, China) with a camera employing a complementary metal-oxide semiconductor (CMOS) 40 MP Exmor RS IMX600y sensor (3648 × 2736 pixels resolution in RAW capturing mode, 1.01 μm pixel pitch). NIR reflectance images were acquired using an 18 MP digital single-lens reflex (DSLR) camera Canon (Canon Inc., Tokyo, Japan) Rebel-SL1 (4.38 μm pixel pitch) modified via removing the internal near-infrared blocking filter (LifePixel Infrared, Mukilteo, WA, USA) and by placing an external NIR-pass filter (‘Standard Infrared’ 720nm filter from LifePixel). Raw reflectance images were exported in the “RawDigger” software (LibRaw LLC, Maryland, USA). After reversing the demosaicing to create raw RGB composite images, which were color balanced, and single-channel NIR images, the results were exported with an uncompressed format. These images were undistorted using the ImageJ2 Lens Correction plugin [47]. An additional step for ensuring the quality of RGB and NIR images consisted of checking the sensors’ errors. In terms of background noise and vignetting, the observed noise was less than 2% of the actual intensity digital values for the two CMOS sensors, and therefore, no intensity corrections were performed.

Thermograms were acquired with a FLIR ONE Pro for Android (Teledyne FLIR LLC, Wilsonville, OR, USA) thermal camera attached to the Huawei P30 smartphone. This lightweight and low-cost instrument (retail cost 400$ at the time of this research) has spectral range 8 μm–14 μm, 160 × 120 pixels resolution, 12 μm pixel pitch, <70 Mk thermal sensitivity, and object temperature range −20 °C–+120 °C.

3.2. Photogrammetric Techniques

Digital close-range photogrammetry involves different recording techniques, and the selection between them largely depends on the complexity and dimensions of the surveyed object. For relatively simple geometries, digital image rectification provides a non-expensive solution for orthoimage generation, which can be used as a background for degradation mapping. A projective transformation-based rectification can be chosen for nearly planar surfaces, while polynomial rectification approaches can be helpful for more complex surfaces. For this research, RGB and NIR images were rectified with the “HyperCube” spectral imagery analysis software Version 11.52 (USACE ERDC Geospatial Research Laboratory, Alexandria, VA, USA).

Orthorectification is often not suitable for creating complete photorealistic representations for complex architectural surfaces or entire historical structures. In these cases, large numbers of images must be simultaneously processed to cover the object completely, producing homogeneous geometric results and ortho-textures. The advancements in dense image matching and the improvements in camera sensor manufacturing have drastically improved photogrammetric solutions [48,49]. The current photogrammetric approaches are based on computer vision algorithms. They are generally robust, affordable, and agile, both in terms of implementation and of the flexibility of ground-sampling distances (GSD) and other parameters that can be adjusted according to requirements. These approaches allow the use of non-metric and even lower-end sensors and have widened the application scope of photogrammetry for cultural heritage because of their increased automatization. The standard workflow followed for architectural recording via dense multiview reconstruction (DMVR)-based approaches starts from an image dataset where every part of the object under study is depicted at least at three overlapping images. Image features are detected and described on every image so that the camera positions and 3D point coordinated can be estimated through a structure-from-motion (SfM) implementation, to produce a sparse point cloud. The 3D point cloud is densified by dense-matching algorithms and then meshed into a 3D model using triangulation algorithms. Finally, the model is textured by interpolating color or intensities from the image dataset and can be further exploited to generate orthoimage mosaics.

The photogrammetric software utilized for the orthoimage generation for this research was MetaShape Standard Edition 1.6.6 (Agisoft LLC, St. Petersburg, Russia). Images inserted in Metashape were radiometrically corrected, as defined in Section 3.1. The high-resolution orthoimages were produced with a mosaicking approach (instead of averaging) to avoid compromising the radiometric quality of results.

3.3. Digital Image Processing—Thematic Layers Generation

The methodology applied for the thematic degradation mapping depends on the pathology of the studied stone surfaces. For this work, the generation of thematic data was based on digital image processing tools for feature extraction. Different approaches were followed depending on the multiplicity of the observable degradation patterns. Specifically, two distinct procedures were applied. For surfaces characterized only by material loss and discolorations, the feature extraction approaches were mainly based on histogram manipulation, thresholding, and edge detection (producing binarized image outputs). For surfaces characterized by more complex degradation, meaning that also vegetation and biological colonization could be observed, unsupervised image classification was performed with ImageJ2 [50] (producing labeled image results). For all the involved case studies, the ICOMOS Glossary [51] was followed as a guide to identifying the individual degradation patterns during visual inspection of the monuments involved.

3.4. Development and Analysis of Thematic Maps

Detailed thematic maps, which presented the architectural surfaces’ degradation for characteristic parts of the investigated case studies, were developed in QGIS 3.18.2 [52]. The rectified images and orthoimage mosaics were utilized as the GIS base maps to annotate the geometric shape of building stone elements. In addition, an individual block approach was considered as it would allow for the description and analysis of degradation at a structural element level [53]. The spatial data input regarding degradation patterns was performed by automatically vectorizing the digital image processing raster results into different layers. The topology of the thematic data was essential for establishing digital connections between spatial entities to implement spatial analyses successfully [15,17]. Therefore, after the polygonization of different degradation forms, the generated spatial entities were thoroughly checked and corrected. At the end of the digital map development process, the structural elements’ outlines and each degradation typology were represented by a different thematic layer.

The constructed thematic maps were used to visualize the architectural surfaces’ pathology and further exploited to extract numerical information and present the overall condition of building stone elements. Spatial and logical analysis based on the thematic layers was used to extract statistical information and damage indexes. The first step for determining the indexes was calculating the spatial join of all biological colonization-related crusts, vegetation, and material loss thematic layers. Then, depending on the percentage of each stone block’s coverage by this joined layer, the stones were categorized according to damage severity in five categories.

4. Case Studies

The above-described digital techniques were integrated for three application case studies that presented different surface degradation. The historical stone monuments are located in archaeological sites in Peloponnese (Greece), in a region that during classical antiquity was part of Elis and Arcadia (Figure 2).

4.1. Temple of Apollo Epikourios

The Temple of Apollo Epikourios (Figure 3) is one of the best-preserved monuments of classical antiquity and an imposing monument of ancient Greek architecture. The temple is positioned at the archaeological site of Bassae and inscribed on the UNESCO World Heritage List since 1986. It was constructed toward the end of the 5th century BC in the rocky heights of the Arcadian mountains, mostly of local limestone. The classical temple was founded on bedrock, on a specially built terrace. It is the only known temple that combines elements of all three architectural orders of antiquity. The temple is Doric, peripteral, distyle in antis, with pronaos, cella, adyton, and opisthodomos. A Doric frieze of undecorated metopes and triglyphs ran along the outer facades. Parts of the temple were excavated and restored in 1902–1906. The Ministry of Culture has overseen the conservation of the monument since 1982. A temporary shelter, erected in 1987, is covering the monument to protect it against the region’s harsh weather conditions and acid rain [54,55].

The thematic data about characteristic surfaces at the Temple of Apollo Epikourios were produced by digitally manipulating NIR reflectance images. These images presented more homogeneous radiometry, which was extremely important in order to extract suitable thematic information. In particular, the original and restoration materials that had been previously used presented more similar reflectance characteristics in the near-infrared than in the visible spectrum. Thus, by using NIR reflectance images, the misinterpretation of other degradation forms could be avoided (Figure 4). The architectural surfaces had been mostly cleaned, and therefore, the observable categories of degradation were only loss of material, cracks, and in certain places discoloration. After the NIR images had been corrected and rectified, binary images representing these different categories were produced. Thresholding was applied for identifying discoloration after flat-field correction, which eliminated any remaining inhomogeneous illumination problems. Edge detection and afterward binarization was performed on the first principal component of the images to detect cracks. Finally, histogram equalization and thresholding were performed to identify the loss of material. Figure 5 presents the generation of the binarized category-specific degradation images for an architectural surface and the production of thematic layers in the FOSS GIS environment. The thematic maps produced for investigated characteristic surfaces are presented in Figure 6 and Figure 7.

4.2. Temple of Athena and Zeus Sotiros

The Temple of Athena and Zeus Sotiros is the most notable excavated monument of the ancient city of Phigalia. It is located on the hill “Kourdoumbouli” at the southwest end of the Agora of the ancient city. It is a rectangular temple with pronaos and cella. The life of the temple spans from the 4th century BC until the Roman Era, while there must also have been an Archaic phase. It was constructed of large stone blocks according to the irregular isodomic system and is preserved up to a height of three building blocks (Figure 8). The stone cubic pedestal of the worshiping statue was found in the cella, while there was a table for offerings in front of it. The large number of inscriptions found inside the temple testifies to its political significance for the city of Phigalia [56].

For this case study, the use of both visible and NIR-spectrum images was opted due to the existence of biological colonization (moss, lichens, biogenic crusts, vegetation), benefiting from the dissimilar reflectance characteristics that it presents at different wavelengths (Figure 9). Using a photogrammetric approach, orthoimage mosaics were generated for parts of the cella and the cubic stone pedestal with the RGB and NIR-reflectance images. Then, they were blended to create pseudo-color orthoimage mosaics (using the Green, Red, and NIR bands) in the HyperCube software. Next, the resulting base maps were segmented to the areas presenting different degradation forms following a k-means unsupervised clustering approach with centroids initialized using the k-means++ algorithm. Finally, the classified images were digitized, and the degradation was semantically annotated in QGIS to the corresponding thematic layers (Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14). A map of damage categories regarding the Temple of Athena and Zeus Sotiros remains could also be produced. Figure 15 additionally provides statistics of the coverage of the historical stone surfaces by various degradation forms. Both investigated surfaces were in their entirety covered by biogenic material or discolored.

4.3. Archaeological Site of Lepreum

The ancient Lepreum constituted one of the most important cities of Elis’s Trifylia. Lepreum had two citadels, the prehistoric acropolis on the naturally fortified hill of Agios Dimitrios, east of the modern village, and the acropolis of the historical period to its north. The peripteral temple of Demeter is the most important monument on the Classical acropolis. The historical city extended up to the prehistoric acropolis. The remains on the Classical and Hellenistic acropolis (Figure 16), particularly the well-preserved enceinte of isodomic and polygonal masonry with its square towers, show that the city thrived during these periods. Inside the enceinte are several buildings, including the Classical Doric peripteral temple of Demeter, mentioned by Pausanias [57].

For this case study, photogrammetric approaches were followed using multiband datasets (examples of reflectance images shown in Figure 17), as described in Section 3.2, to generate the necessary base maps for the mapping process. The thermal orthoimage mosaic was produced following a hybrid approach exploiting both the visible and thermal images captured by the FLIR One Pro camera [58] (Figure 18). The false-color multiband image mosaic composed of three single-band mosaics (RGB, NIR, and TIR) was classified to create the thematic map (true-color image shown in Figure 19 and final degradation pam in Figure 20). Statistics were again calculated for the coverage of the historic surface by various degradation forms (Figure 21), while the damage map (Figure 22) considered the presence of vegetation, moss, and biocolonization. Additionally, in this case study, the historical surface was almost entirely covered by biogenic material or discolored.

5. Discussion and Conclusions

The development of an appropriate methodology for the implementation of close-range sensing technologies and FOSS GIS for stone degradation mapping has been outlined and tested on weathered historical monuments and remains. Affordable, accessible instrumentation is employed, which adds to the practicality and flexibility of the methodology. However, the implementation of low-cost sensor technology implies that the collected data should be thoroughly checked and comprehensively corrected. The quality verification of the base maps used for mapping purposes is essential to the accuracy of results purposed for thematic visualization and spatial analysis.

Multiband reflectance imaging demonstrates an important non-destructive approach to enrich the thematic mapping process. The plethora of degradation forms, as showcased, presents distinct reflectance characteristics when imaged at different ranges of the electromagnetic spectrum. Mainly, stone patterns caused by biogenic factors present a unique spectral footprint in the infrared range. Thus, involving infrared images for the detection and mapping of degradation enhances the identification capacity for surface patterns such as vegetation, moss, and various biogenic crusts. Nevertheless, when mapping stone that presents less complex surface characteristics, the combination of images captured at multiple spectra may not be necessary, which can speed up the mapping process. The application performed at the Temple of Apollo Epikourios presents such a case, where only images captured in the near-infrared spectrum were used as base maps since they maximized the reflectance contrast among degradation characteristics that we wanted to map and minimized the contract between original and restoration materials that we did not want distinguished. Furthermore, for this case study, a single-image approach could be followed due to the narrow extent of the investigated areas.

Photogrammetric techniques, applied for the generation of high-resolution ortho-mosaics from image datasets, are essential for producing the background for mapping extensive stone surfaces. Their application proved useful when large multiband datasets needed to be integrated to produce base maps in a cost-effective way. Spectral orthomosaics can more easily be fused together than individual photos. Additionally, the prospect of involving metric data introduces a spatial reference to the thematic maps, which allows for subsequent spatial analyses to be performed.

Processing rectified images and orthomosaics for extraction of the degraded stone surface features (pattern recognition) has often been performed manually. In the developed methodology, unsupervised clustering-based segmentation has been performed, which speeds up the process but requires the interpretation of the different patterns after they have been segmented on the images. Implementation at the Temple of Athena and Dios Sotiros, and the archaeological site of Lepreum showed that this method is very effective when a large number of degradation forms are present but underperforms when there is overlap between different weathering damages. Additionally, this method of extracting surface patterns cannot detect at the same time features induced by material loss. The combined use of accurate range/depth data and multiband data should be researched in this direction. The proposed methodology also offers a more simplistic alternative for cases of stone monuments where there is an absence of extensive biodegradation, based mainly on image histogram manipulation and edge detection.

Transferring the thematic data extracted from rectified images and multiband ortho-mosaics to a GIS environment offers many advantages. This work presented that the visualization of thematic degradation results becomes more effective through a spatial information management environment and also that damage index maps and statistical results can be easily extracted through the heritage GIS implementation. This approach helps with the weathering interpretation and subsequently can assist conservation measures. The damage categories can be qualitatively and/or quantitatively defined. The potential availability of descriptive and qualitative information from other on-site investigation techniques and historical documentation can enhance the results.

Direct outlooks concern the evaluation of different spectral band composites and image segmentation algorithmic implementations to improve the accuracy of degradation pattern recognition. As intelligent methods based on machine learning have the capacity to support damage detection on historic stone monuments, the application of different classification methods utilizing clustering, dimensionality reduction, decision tree, and ensemble learning methods should be further investigated.