Multispectral Sensing and Data Integration for the Study of Heritage Architecture ^†

Abstract

The recording and processing of terrestrial multispectral information can have significant value for built heritage studies. The efficient adoption of active and passive sensing techniques operating at multiple wavelengths and the integrated analyses of the produced data is essential for enhanced observation of historical architecture, especially for the implementation of rapid non-destructive surveys, which can provide an overall assessment of the state-of-preservation of a historical structure to indicate areas of interest for more detailed diagnostics. Based on this rationale, the presented work aims at providing methods for prompt recording, fusion, and integrated visual analysis of two-dimensional multispectral results to study architectural heritage. Spectral images—captured with a modified digital camera—thermograms, photogrammetrically produced orthophoto-maps, and spatial raster data produced from point clouds are integrated and analyzed. The results are evaluated within the scope of studying building materials, deterioration patterns, and hidden defects, towards the employment of advanced geomatics approaches to monitor built heritage effectively.

1. Introduction and Objectives

Historical buildings are particularly valuable cultural heritage assets. The necessity of the monitoring and conservation of built heritage, through holistic recording and integrated data management approaches, is profound, due to its vulnerability to environmental pressures, natural and human-caused disasters, neglect, and inappropriate conservation interventions [1]. The comprehensive understanding of historical monuments or groups of heritage architecture requires detailed documentation [2], which can be achieved by applying multidisciplinary investigative procedures [3]. These studies should be performed in non-destructive ways [4], to a great extent, to avoid further damaging the historical buildings. Terrestrial-based recording techniques, employing active and passive sensors, are essential for the non-destructive study of heritage architecture’s physical characteristics. Capturing information in multiple wavelengths ranging from the ultraviolet to the infrared provides essential input for detecting historical materials [5] and induced damage or weathering [6]. Optical inspection [7], infrared thermography [8], and recently, Multiple-view-Stereo (MVS)-based digital photogrammetry [9,10], and terrestrial Light Detection and Radar (LiDAR) [11] are being applied—to produce records containing necessary metric spatial and radiometric information or meta-information—to assist non-destructive observation. The combination of metric and qualitative data is often highlighted [12,13] as a means to collect products that can serve as the basis for accurate quantitative measurements. The adoption of multispectral methods allows for the efficient recording and integration of multimodal data [14] to enhance observation and facilitate further non-destructive diagnostical investigations by identifying specific areas of interest [15].

The presented work aims to implement efficient terrestrial-based multispectral sensing and data integration methods and repurpose visual analytics procedures commonly applied for remote-sensing datasets to evaluate them for architectural heritage investigations. The assessment of the involved methods is performed from the scope of detecting and classifying historical materials, surface alteration, and hidden defects, to provide preliminary diagnostical assumptions for the state-of-preservation of built heritage, simultaneously proving the necessity of the employment of advanced geomatics approaches for structural health monitoring.

2. Materials and Methods

2.1. Methodology

The proposed workflows include multi-sensor two-dimensional (2D) scenarios to produce localized measurements and metric three-dimensional (3D) procedures to provide larger-scale accurate ortho-derivatives. The investigated approaches were applied to study the façades of Castello del Valentino, one of the historical residences of the Royal House of Savoy in Turin (Italy), and inscribed as UNESCO World Cultural Heritage Site since 1997. The main (west) façade is overall conserved in good condition; however, parts of the river-facing (east) façade are significantly degraded due to the direct and indirect exposition to moisture.

2.2. Imagery Data

The first phase of the terrestrial sensing process was the acquisition of reflectance and emissivity data. Visible and near-infrared (NIR; 0.7–1.4 μm) reflectance images were taken with a Digital Single-Lens Reflex (DSLR) camera, a Canon Rebel-SL1 (CMOS 22.3 × 14.9 sensor, 3456 × 5184 pixels spatial resolution, 4.4 μm pixel size), using externally mounted filters for the different spectral bands. Passive thermography images were taken with a FLIR T1030sc camera, which employs an uncooled 1024 × 768 Long-Wavelength Infrared (LWIR) detector array (1024 × 768 pixels spatial resolution, 17 μm pixel size, measurement accuracy ±1 °C, thermal sensitivity 20 mK at +30 °C, spectral range 7.5–14 μm). The images were corrected from lens distortion after performing calibration for the optical and thermal sensors, using custom-made checkerboard-pattern targets.

2.3. Multispectral Image Composition

Image composition involved the creation of multispectral cubes from the images collected with different techniques. Undistorted NIR and thermal images were matched manually with the undistorted RGB images, by identifying common features (on the planes of interest). The necessary projective transformation parameters were calculated in the HyperCube hyperspectral research freeware, and affine transform was used to warp the infrared images using as reference the RGB images. Additionally, RGB images were decomposed to the individual bands, thus producing multispectral cubes with a total of five spectral channels (blue, green, red, NIR, LWIR).

2.4. Multispectral Orthophoto-Mosaic Composition

An image-based approach involving Structure-From-Motion (SFM) and Dense Multi-View 3D Reconstruction (DMVR) algorithms was also considered to produce larger-scale multispectral results. The RGB imagery from the Rebel-SL1, the NIR imagery, and the RGB imagery from the optical sensor of the T1030sc were combined in a single photogrammetric dataset for the accurate estimation of the external orientation parameters of the cameras during acquisition. Then, the 3D geometry for a part of the main façade was densely reconstructed using only the Rebel-SL1 high-resolution RGB imagery. The produced point-cloud was meshed into a model using the Delaunay triangulation algorithm. RGB, NIR, and LWIR images were subsequently used to create three orthophoto-mosaics containing spectral information in combination with the resulting geometry. An adaptive ortho-photo texturing approach was followed to avoid inclination and convergence effects concerning the thermal-range imagery. For the single-image approach, spectral ortho-imagery was loaded in the HyperCube (without any need for further geometrical corrections due to the common spatial reference of the orthophoto-mosaics) and five-band multispectral cubes were produced after decomposing the RGB images to individual bands.

2.5. Raster Range Data

Point-clouds collected with a FARO Focus 3D × 330 laser scanner were used to generate 3D models of the façades. For each surface under investigation, an artificial plane was calculated best fitted to the flatter/less deteriorated areas and used in CloudCompare to calculate distances from the 3D model, thus creating a raster range map, which could be treated as an imagery product.

3. Results and Discussion

Multi-spectral and range images regarding twelve areas on the west and easter facades of Castello den Valentino were exploited through:

• Principal Component Analysis (PCA);
• Generation of pseudo-colored images using the five spectral bands and the calculated principal components (PCs);
• Generation of Color Quantized Images (CQIs);
• Unsupervised K-means, clustering-based image segmentation;
• Edge detection filtering.

To facilitate segmentation procedures and interpretation of visual analytics, the results were cropped according to the thermal images’ dimensions. In the following section, some of those areas with the most interesting results are presented.

Figure 1, Figure 2, Figure 3 and Figure 4 show how image analyses facilitate the identification and classification of decay. While on area 2, we could identify only detachments, discolorations, and small cracks, in area 3, we could observe the severe loss of material, biodeterioration, discoloration, erosion, and large cracks.

The radiometric characteristics identified on the lower part of area 2 (Figure 1 right, and Figure 2 left), combined with the minimal surface degradation observed, led us to identify it as an area of interest for further non-destructive experimentation.

Figure 5 and Figure 6 show how the image analyses assisted the classification of the materials for parts of the main façade, such as the columns. Figure 6 also presents how the existence of anomalies (lower area 7), which leads to anisotropic radiometric characteristics, is resulting in errors regarding the classification of materials. In this case, the visual analytics reveal the position of replacement materials, which will be further studied with additional non-destructive techniques.

Figure 7 presents how pseudo-colored raster images of a shaded visualization and range images can help the interpretation of material loss (area 2).

4. Conclusions

Image processing techniques applied to integrated multispectral datasets, and visualization techniques applied to laser scanning-derived data, were used to identify and map historical materials and their decay. On occasion, the methods involved proved useful to point out areas where additional non-destructive surveys or minor-destructive tests, involving additional sensors, should be performed. This work shows the potential of terrestrial multispectral sensing for architectural heritage investigations and how techniques otherwise standard for aerial and satellite investigations can be repurposed in this scope. The author’s future research will concentrate on calibrating the various sensors to acquire corrected radiometric data in order to perform a more accurate mapping of the materials and decay. The integration in 3D of the data described here with ground-penetrating radar measurements will also be a focal point for future research.