Building Change Detection Based on a Gray-Level Co-Occurrence Matrix and Artificial Neural Networks

Abstract

The recovery phase following an earthquake event is essential for urban areas with a significant number of damaged buildings. A lot of changes can take place in such a landscape within the buildings’ footprints, such as total or partial collapses, debris removal and reconstruction. Remote sensing data and methodologies can considerably contribute to site monitoring. The main objective of this paper is the change detection of the building stock in the settlement of Vrissa on Lesvos Island during the recovery phase after the catastrophic earthquake of 12 June 2017, through the analysis and processing of UAV (unmanned aerial vehicle) images and the application of Artificial Neural Networks (ANNs). More specifically, change detection of the settlement’s building stock by applying an ANN on Gray-Level Co-occurrence Matrix (GLCM) texture features of orthophotomaps acquired by UAVs was performed. For the training of the ANN, a number of GLCM texture features were defined as the independent variable, while the existence or not of structural changes in the buildings were defined as the dependent variable, assigning, respectively, the values 1 or 0 (binary classification). The ANN was trained based on the Levenberg–Marquardt algorithm, and its ability to detect changes was evaluated on the basis of the buildings’ condition, as derived from the binary classification. In conclusion, the GLCM texture feature changes in conjunction with the ANN can provide satisfactory results in predicting the structural changes of buildings with an accuracy of almost 92%.

1. Introduction

Within a disaster management scheme, the recovery phase aims to restore the landscape to the pre-destruction level and reduce vulnerability in the future. Following a catastrophic earthquake event, the progression of the recovery phase depends on the decision makers’ characterization of the disaster-hit areas, which is generally divided into three types: collective resettlement area, original site recovery area and non-disaster area [1]. The usefulness of geospatial data acquired based on remote sensing technology has been extensively demonstrated in recent years. By incorporating both spaceborne and airborne techniques, remote sensing has nowadays been evolved into an extremely useful tool in gathering, processing and analyzing geospatial data with the aim of extracting valuable information for proper decision making in various scientific fields [2,3,4]. Change detection is included among the fields where remote sensing is frequently applied. Due to the unique characteristics of remote sensing data, such as their high temporal frequency, digital format and the availability of several sensors with a variety of spatial and spectral resolutions, they can be efficiently used in order to detect changes in numerous applications, such as land use management and planning, urban expansion and planning, and disaster monitoring and management [5,6,7,8,9].

With the continuous progress of remote sensing technology, change detection can be performed through various platforms, such as satellites, manned aircrafts and Unmanned Aerial Vehicles (UAVs). In particular, the use of UAVs is constantly increasing due to their flexibility, low cost and real-time monitoring, as well as their capability to observe objects from multiple angles [10]. Based on UAV data, change detection algorithms can be applied either in 3D derived data, i.e., point clouds [11,12], or in 2D orthoimages [13]. Furthermore, various methods and techniques based on the analysis of remote sensing images have been developed in order to detect changes. Nowadays, machine learning plays an important role in improving the efficiency of change detection and monitoring because of its high abilities in automatic feature learning and visual pattern recognition [14,15]. Machine learning techniques, Support Vector Machines (SVMs), and Artificial Neural Networks (ANNs), etc., are highly used in change detection, especially in cases where conventional change detection methods such as post-classification comparison are often problematic, i.e., in urban change detection [16,17]. The use of ANNs has received increasing attention, while they have been used for different change detection applications such as vegetation change [18,19], land cover change [20] and urban change [17,21,22]. These studies suggest that the ANN method can improve the accuracy of change detection compared to other techniques.

Another approach for remote sensing change detection is the extraction of a large number of image features, with the goal of improving the discriminative capability of image change information; texture is included among these features [23,24]. Texture is one of the key structural characteristics of an image, used to identify objects or regions of interest in an image, and it is also in line with human visual characteristics [25,26]. Texture extraction is critical as it serves as an input for further advanced processing and has a significant impact on the quality of extraction [27]; therefore, numerous studies on texture extraction from remote sensing data have been conducted (e.g., [27,28,29,30]). Moreover, texture-related research is still a hotspot for researchers in computer vision and image processing, while it is continuously evolving [31]. There are various methods of texture analysis that involve both the color and the arrangement of pixels in an image, i.e., GLCM (Gray-Level Co-occurrence Matrix), fractal model, Fourier and wavelet transformations. Furthermore, different classification schemes of these methods have been developed, including, i.e., statistical, model-based and geometrical/structural methods. Among all the texture analysis methods, the GLCM method has proved to be one of the most useful and powerful texture analysis techniques, since it is straightforwardly simple, easily executable and provides good results [27,32,33,34]. Furthermore, the GLCM has shown its efficiency within previous comparison studies [35,36]. GLCM features (e.g., contrast, correlation, energy, entropy, homogeneity) are calculated based on the occurrence of a pair of gray-level pixels in an image in predefined directions [10,34,37], which are used for different remote sensing applications such as land use classification [38], built-up area extraction [39], damaged building detection [29,40,41] and high-resolution satellite image analysis [30]. Another type of texture feature extraction method that has been developed in the last decade is the learning-based approach well-known as Convolutional Neural Networks (CNNs). CNNs are quite powerful in classification and segmentation problems, applied also in remote sensing, and they have shown their superiority to various studies. Between other applications in remote sensing, CNNs have been applied in urban areas for building footprint extraction and change detection [42,43,44,45,46].

In the case of disaster monitoring and management, remote sensing has proven to be an essential and efficient tool for damage assessment and the detection of building changes after a catastrophic event such as an earthquake [34]. Not only are the observations of the damage level and the spatial distribution after a destructive earthquake of paramount importance in order to plan the first rescue activities and understand the effect of seismic activity on buildings, but the monitoring of the affected area over time is also considered essential in order to study the urban development, such as buildings’ collapse and/or construction, after such a catastrophic event [14,23,47,48]. In this context, in [49], the authors monitored building damage after the Haiti earthquake through satellite stereo imagery and DSMs, while using texture features to optimize accuracy. In [23], the recovery of the Italian city of L’Aquila following the 2009 earthquake was monitored by using spectral, textural and geometric features in order to determine changes in buildings, thus allowing a reduction in the extensive fieldwork required. The Haiti 2010 earthquake was also studied in [50], where authors mapped the damaged buildings by developing an ANN model for classifying the affected area into two classes of changed and unchanged areas, using the extracted textural features calculated using the GLCM from pre- and post-event images as an input vector. The ability of the proposed method to detect building changes was proved by the reported overall accuracy of 93%. In the same context, the authors in [40] used high-resolution satellite images obtained before and after the 2010 Haiti earthquake to extract textural features applying the GLCM method. The results showed that dissimilarity was identified as a better classifier in detecting the collapsed buildings and that about 70% of them were correctly identified. In other research, the authors detected collapsed buildings using three different textural features derived from the GLCM and applying an SVM classifier [51]. The SVM and a synergy of high-resolution optical and Synthetic Aperture Radar (SAR) images were also used for the detection of the collapsed buildings after the 2011 Japan earthquake [52]. Finally, in [41], the authors detected collapsed buildings after the 2017 Iran–Iraq earthquake using ten spectral indices in combination with seven different textural features derived from the GLCM and applying an SVM classifier. Finally, CNNs were also applied to building change detection following a catastrophic event such as an earthquake. This task is usually difficult due to complex data with a heterogeneous appearance and with large variations within classes. When the intra-class variation is low (i.e., [45]), the change to be detected is limited to building/no building; therefore, the building extraction and change detection can be more easily approached. However, the building damages from an earthquake event include multiple sub-classes, i.e., collapsing and partial collapsing. For example, in [53], the detection was based on a pre and a post image; the overall accuracy was limited to 76.85% and the F1 score to 0.761. Projecting this method to a more complex problem, i.e., multitemporal change detection within a recovery phase where collapses, partial collapses, debris removal and new construction can take place, training a CNN would be quite challenging.

The aim of the present study is to present a methodology and the results for multitemporal building change detection after a disaster such as an earthquake. In relation to other studies, the applied methodology aims to monitor the changes that occurred in multiple time steps within the buildings’ footprints during the recovery after a catastrophic earthquake for an entire settlement on the building level. More specifically, the proposed method detects changes in the building sites such as significant changes to damaged buildings, i.e., partial collapses due to weather conditions and aftershocks, the removal of debris and the construction of new buildings, thus monitoring the development and rehabilitation of the settlement after the disaster. The identification of these changes was based on the GLCM features and ANN modelling. Even if various studies have been conducted on this topic, the multitemporal monitoring of a post-earthquake site based on GLCM features and an ANN, to the best of our knowledge, has not yet been fully examined. Therefore, taking a step further from previous studies, herein, the multitemporal monitoring of an entire settlement on the building level using remote sensing data was performed in order to fully identify the building changes in the whole study area, as well as to spatially localize them during the various time steps.

2. Materials and Methods

2.1. Study Area

Lesvos Island, located in the North Aegean region, is one of the most seismically active areas in Greece (Figure 1). Lesvos is thoroughly studied by geologists and geoscientists due to the existence of a rich tectonic and geomorphological background of the broader area in which the island is situated. More specifically, Lesvos is located on the Aegean microplate, which, according to [54], is subject to two different stress regimes. On the one hand, the island is subject to a NE–SW stress from the Anatolian fault, while, on the other hand, is subject to a N–S stress due to the retreat of the subducting Mediterranean plate from the Aegean plate.

The seismic activity on the island has caused both material and environmental damage, as well as human losses, due to which Lesvos is placed as the second greatest seismic hazard zone in Greece [54]. On 12 June 2017 (12:28 GMT), the settlement of Vrissa, situated in the southern part of Lesvos, underwent a catastrophic earthquake of a magnitude of 6.3, which caused one death and, at the same time, destroyed about half of the village [55]. The earthquake epicenter was 35 km away from the island’s capital, Mytilene. Even though the epicenter of the earthquake was not located close to Vrissa, the settlement suffered serious damages from the earthquake event, especially the western part of it. This can be mainly attributed to three factors: the direction of the earthquake, the type of soil and the types of buildings in the settlement.

2.2. Data Collection

In general, two main types of GIS data were used in the present study: (a) 12 orthophotomaps with the same resolution of 0.2 m acquired by a UAV within a 52-month period; and (b) the buildings’ footprints, which were used for the computation of the GLCM texture features and the compilation of the final database.

2.3. General Methodological Framework

2.3.1. Photo Interpretation and Building Classification

The main objective of the photo interpretation of the 12 UAV orthophotomaps was to examine the buildings’ status in the Vrissa settlement so that the buildings were classified into change/no change (1 or 0) for all time intervals, depending on whether changes were observed or not; the classification results were then used as the desired output for the ANN model. During this task, the visual comparison of the orthophotomaps taken in sequential time steps was performed, thus resulting in observations of the buildings’ state for 11 time intervals. The aim of the whole process was to find if the buildings in the study area underwent any changes during each time interval. At this point, it should be noted that, as changes were identified, only changes in the building itself and not changes in other types of factors, such as the exterior surroundings of the buildings, were identified. These changes include: (i) partial or total collapse; (ii) debris removal; and (iii) new construction. The photo interpretation process was performed for a total of 1060 buildings located in the study area and for each time interval, thus resulting in 11,660 recorded values, which are set either to 1 or 0, depending on whether changes were observed or not.

2.3.2. GLCM Texture Features

The GLCM method is a widely applied method for measuring and converting gray values to texture information, which provides an essential understanding of texture analysis and feature extraction [31,56,57,58,59]. The pioneer of this widely used method is Haralick [25], who created the texture analysis algorithm by assigning the textural relationship between the pixels of an image. This method extracts the structural information about the texture pattern to be analyzed at different scales and orientations, making the method more efficient and easier to implement. Based on the literature review, as regards the GLCM texture analysis, it was found that 5 out of 14 indicators, namely, contrast, correlation, energy, entropy and homogeneity, are most commonly used in similar studies; these indicators are defined as follows:

Contrast is a measure of the number of local texture variations in an image. Therefore, if there is a high number of variations, the contrast is high, while, in cases of minimal variations, the contrast is low. It is given by the following equation:

$$Constrast={\displaystyle \sum }_{i,j}{\left|i-j\right|}^{2}p\left(i,j\right)$$

(1)

Correlation is a measure of the linear dependence of gray levels between individual pixels in an image. The higher the correlation is, the more uniform the image is, while correspondingly low correlation values indicate non-uniformity [25,60]. It can be computed by the equation:

$$Correlation={\displaystyle \sum }_{i,j}\frac{\left(i-\mu i\right)\left(j-\mu j\right)p\left(i,j\right)}{{\sigma }_i{\sigma }_j}$$

(2)

Energy is the sum of the squares of all elements of the GLCM. It measures the uniformity of texture in an image and is considered the appropriate measure for detecting its anomaly. The energy is given by the equation:

$$Energy={\displaystyle \sum }_{i,j}p{\left(i,j\right)}^2$$

(3)

Entropy is a measure of randomness in the distribution of pixels in an image. Specifically, the denser the texture of an image, the more dispersed the grayscale distribution and the higher the entropy value. Conversely, small entropy values reveal smoother areas in an image. Entropy is directly dependent on the range of the data used for the analysis [61]. It can be computed by applying the equation:

$$Entropy=-{\displaystyle \sum }_{i,j}p\left(i,j\right){\log }_{2}p\left(i,j\right)$$

(4)

Homogeneity calculates the uniformity of pixels in the image and shows a comparative arrangement of their equal values. In addition, this index is similar to energy and is given by the equation:

$$Homogeneity={\displaystyle \sum }_{i,j}\frac{p\left(i,j\right)}{1+\left|i-j\right|}$$

(5)

where i and j, respectively, are the row and column numbers of the image, and p is the number of gray-level co-occurrence matrices in the GLCMPl.

In order to create the input variables for the ANN model, three main steps were carried out. In a first stage, conversion of the 12 orthophotomaps from the RGB (Red, Green, Blue) color model to grayscale was performed in order to reduce the complexity of the representation of the texture features. By converting the orthophotomaps to grayscale, i.e., by calculating the weighted sum of the three bands, a pixel with only one band is obtained, thus reducing its information. The next step was the calculation of the GLCM texture features for each building footprint of the 12 time steps in the Matlab environment. For achieving this, both the orthophotomaps (raster files) and the shapefiles of the building footprints were used. At first, five GLCM features, i.e., contrast, correlation, energy, entropy and homogeneity, were selected to be calculated based on the literature review, since they are the most commonly used in change detection studies [62,63]. To reduce the ANN model’s complexity, three of the GLCM features were finally selected as inputs for the model. For this task, correlation analysis of all calculated GLCM features was conducted and the correlation coefficients, expressed by the Pearson correlation coefficient (r), were computed. The correlation coefficients were used to investigate the relationship between the five examined GLCM features and assist in selecting the most non-correlated of them. The rationale behind this choice lies in the fact that, in the case of selecting features that are more correlated to each other, the training of the ANN generally becomes more complicated, leading to unreliable results.

2.3.3. Artificial Neural Networks (ANNs)

As ANNs learn from experience, they can perform complicated operations and recognize complex patterns, even if these patterns are noisy and ambiguous. An ANN model consists of a set of nonlinear and densely interconnected elements or neurons, which are inspired by biological neural systems. The fundamental idea of an ANN is to connect a set of inputs to a set of outputs. Hence, the main architecture of an ANN model usually contains three distinct layers: (1) the input layer; (2) the hidden layer(s); and (3) the output layer, where the results of the ANN are generated. Moreover, the performance of ANNs generally relies on several factors, such as the number of hidden layers, the number of hidden nodes, the learning algorithm and the activation function of each node. In this study, a multi-layer perceptron (MLP) Feed-Forward Neural Network (FFNN) was chosen to generate the classification model. The FFNN in general is the fastest neural network and broadly consists of a set of layers representing the input layer, one or more hidden processing neuron layers and an output layer of processing neurons [64,65,66].

As already stated, the ANN model developed in the present study aims to estimate the status of buildings, i.e., whether any changes are detected or not, based on the changes in their textural characteristics determined by applying the GLCM functions. More specifically, the main purpose was to train ANNs to detect any potential building changes (dependent variable) between two sequential time steps using the texture characteristics of the buildings (independent variables) computed for these time steps as an input.

Various structures of the ANN were developed, trained and tested based on the generic scheme of one input layer, one hidden layer and one output layer (Figure 2). The GLCM texture features were defined as the independent variables (input layer), while the existence of building changes or not was defined as the dependent variable by setting the variable value to 1 or 0, respectively. With regard to the input layer, the values of three GLCM features of two different time steps, i.e., six input nodes, were considered. On the other hand, the output layer contains one neuron, since buildings are classified into 1 or 0, depending on whether changes were detected or not.

Based on the binary building classification described in Section 2.3.1, a total of 11,660 records were documented (1060 buildings in 11 time intervals). As expected, most of the recorded values are equal to zero (0) due to the fact that most of the buildings in the study area did not undergo any changes during the entire study period. In fact, only 678 out of 11,660 records were set equal to one (1), indicating building changes between two time steps. By taking into consideration the number of records equal to one (1) and in order to create a balanced training dataset for the ANN model, the same number of records equal to zero (0) were randomly selected, i.e., 678. As a result, a total number of 1356 records are included in the training dataset.

Another important step in the ANN model development process is the selection of the activation (or transfer) function used for controlling the output of the ANN across different domains, since activation functions can significantly affect the complexity and performance of the ANN. In the present study, hyperbolic tangent sigmoid (Tansig) was selected for the hidden layer, and log-sigmoid (Logsig) for the output layer. In general, the sigmoid function, which is one of the most commonly used activation functions in ANN models, transforms the input of any value into a value in the range between 0 and 1.

After defining the ANN architecture, the training of the ANN model was based on the Levenberg–Marquardt (LM) algorithm [67]. The role of the training algorithm is to repetitively correct the weights and biases of neural network throughout the training process and thus to maximize its performance. During the ANN training, a wide number of nodes in the hidden layer were tested while each structure was trained with 10 random weight initializations. At the end, the model with the best performance was saved for the final implementation. Finally, to control the training process and test the performance of ANN models, the cross-validation technique was applied. According to this method, the available data, i.e., 1356 patterns, were divided in three separate datasets: training, validation and test sets with a ratio of 70%, 15% and 15%, respectively. The training dataset was used to train the model, the validation dataset to control the training process, i.e., stop the training before overfitting, and the test dataset for the final performance evaluation of the trained model. Each of these sets contains totally different patterns and, therefore, the ANN model sees completely different data in order to perform each of the three processes, i.e., training, validation and testing. The evaluation of the applied method was based on the following metrics that are widely used for binary classification problems [68]:

$$Overall Accuracy \left(OA\right)=\frac{TP+TN}{TP+FP+TN+FN}$$

(6)

$$User\prime s accuracy \left(UA\right)=\frac{TP}{TP+FN}$$

(7)

$$Producer\prime s accuracy \left(PA\right)=\frac{TP}{TP+FP}$$

(8)

$$F1-score=\frac{2\times TP}{2\times TP+FP+FN}$$

(9)

where TP: true positive, TN: true negative, FP: false positive, and FN: false negative. Figure 3 presents the overall workflow of the current research.

3. Results and Discussion

3.1. Photo Interpretation and Building Classification

Figure 4 shows some indicative examples of buildings with and without structural changes, as they have been categorized during visual interpretation. Changes were considered not only in buildings that have collapsed, but also in those where significant changes (cracks, partial collapses) and other visible damage were observed. Moreover, the removal of debris from a building site was marked as a building change; otherwise, there was a high risk of increasing the commission error due to the large difference in orthophotomaps between two time steps (Figure 4a). On the other hand, in the case that buildings had undergone no changes or debris was not removed from a site, no building changes were considered (Figure 4b).

After the image interpretation of the status of the buildings’ footprints into 0 or 1, a map showing the spatial distribution of all buildings in the settlement of Vrissa that had undergone at least one change within their footprints over the entire study period (13 June 2017–23 October 2021) was created (Figure 5). As can be observed, most building changes are located in the north and western part of the settlement. Since this part of the settlement was the most affected by the earthquake, it is reasonable that most changes over time are observed there. These changes may refer to either the collapse of crumbling buildings that did not collapse during the seismic activity or the construction/repair of buildings. At this point, what is worth mentioning is that buildings with changes do not occupy only 33% (370 buildings) of the total building stock, as shown in the pie chart in Figure 5, but even more; this is because some of the buildings were being continuously reconstructed, thus reaching a total number of 650 buildings with changes. However, in the case of Figure 5, considering each building as one change, the final percentage of buildings with changes is equal to 33% of the total building stock.

3.2. GLCM Texture Features

3.2.1. Correlation Analysis

As already mentioned in Section 2.3.2, the five calculated GLCM texture features, i.e., contrast, correlation, energy, entropy and homogeneity, may contain statistically correlated information, thus making it difficult to estimate the buildings’ conditions through machine learning, that is, to develop a reliable ANN model. Hence, for the purpose of reducing the data, it was considered necessary to use three out of five GLCM features. For this task, a correlation analysis of the five calculated parameters was conducted and the three less correlated parameters were selected as the input for the ANN model. Figure 6 shows the results of the correlation analysis in terms of the correlation matrix, including the scatterplots for all pairs of variables, along with the value of the Pearson correlation coefficients (r) for each pair. After calculating the sum of the correlation coefficient values in each row (or column) for all five variables and considering that the correlation of every variable with itself is 1, it was found that correlation, energy and entropy show the lowest sum values, i.e., 3.30, 3.19, and 3.17, respectively, and, therefore, they were selected as the input for the ANN model. On the contrary, homogeneity shows the highest sum (3.76) and, hence, the highest correlation among the other variables.

3.2.2. Visual Interpretation of Selected GLCM Features

A visual inspection of the three selected GLCM features, i.e., correlation, energy and entropy, was performed to examine how these features differentiate between two time steps in the case of building changes. For better visualization results and in order to draw clearer conclusions, examples with a high difference in the calculated values of the three GLCM features were taken into consideration. At first, Figure 7 depicts the visual differentiation of correlation between two images in the case of a collapsed building. As is obvious, before a building collapse (Figure 7a) the neighboring pixels of the corresponding image are more correlated with each other, thus creating a uniform image; on the contrary, after a building collapse (Figure 7b), the pixel distribution contains various gradations of the grayscale, meaning that the pixels are less related to their neighbors.

Next, Figure 8 illustrates the visual differentiation of energy between two images, again in the case of a collapsed building. It can be clearly seen that, in the image before the building collapse (Figure 8a), the energy is higher, since no texture disorder is detected in comparison with the corresponding image after the collapse (Figure 8b). In other words, there are more repetitions of pairs of pixels in the first image, thus resulting in a higher value of energy.

Finally, as far as the entropy is concerned, it is higher in the second image where the debris was removed from the site due to the fact that the randomness of the pixel distribution is higher, which increases the texture density (Figure 9).

3.3. Development of ANN Models

3.3.1. ANN Model Training and Testing

Different structures of ANNs were tested according to the trial-and-error procedure described above. The optimal number of neurons in the hidden layer was found equal to eight, thus resulting in an ANN architecture composed of six input neurons, eight hidden neurons and one output neuron (6:8:1). Figure 10 shows the learning performance of the ANN model based on the MSE values. It should be noted that, hereinafter, the results of all three independent datasets are presented; however, the accuracy metrics of the testing dataset (the unseen during the training procedure) are the metrics for the final evaluation of the presented approach. Learning curves are a widely used diagnostic tool for evaluating model learning performance over experience, i.e., as the model is exposed to more and more training data, as well as for identifying problems, such as model underfitting or overfitting. According to this figure, the developed ANN model shows an MSE value equal to 0.078 at the convergence point, which is lower than Zahraee and Rastiveis [50], even though it reached the best validation performance after almost the same number of training epochs, i.e., at epoch 18, as in the aforementioned study.

Table 1. Accuracy metrics for all datasets.

Dataset	User’s Accuracy	Producer’s Accuracy	Overall Accuracy	F1-Score
Training	0.896	0.907	0.904	0.902
Validation	0.886	0.895	0.887	0.890
Test	0.906	0.950	0.926	0.927

presents the evaluation results of the ANN model for all datasets (training, validation and testing). It is evident that, according to the overall accuracy (OA) of the training, validation and testing datasets, the ANN approach on GLCM texture features performed quite well, providing good results. More specifically, the OA for training, validation and testing datasets is 90.4%, 88.7% and 92.6%, respectively, the latter of which is slightly higher than Liu and Lathrop [17], who developed an ANN model to detect newly urbanized areas in New Jersey, USA, and significantly higher than Mansouri et al. [69], who used an ANN classifier on GLCM features to identify building damage after an earthquake event.

Furthermore, with regard to the errors of the ANN model and, in particular, those of the testing data set, the results reveal that 15 instances of building changes were wrongly classified. All these errors were detected, while some of them are visualized and discussed in the next section.

Next, the ROC curves for the three datasets are presented (Figure 11).

The ROC curve is an effective platform for visualizing and evaluating classifiers, including binary classifier systems such as the one developed in the present study. This is usually achieved by plotting the true positive rate (TPR), also known as the sensitivity rate, against the false positive rate (FPR), also known as the specificity rate [70,71]. As an important tool for evaluating the performance of a machine learning model, a perfect classifier has a TPR equal to 1 and an FPR equal to 0. In general, the performance of the ROC curve is indicated by the Area Under the ROC curve (AUC), which provides a scalar measurement of the classification performance and varies between 0 and 1. The higher the value of the AUC, the better the performance of the model. From the acquired curves shown in Figure 11, it is concluded that the training performed quite well, since for all data sets, i.e., training, validation and testing, the curves are far from the diagonal line, with an AUC equal to 0.968, 0.961 and 0.969 for the three datasets, respectively.

Finally, the adequate overall performance of the ANN model is also reflected in the error histograms for the training, validation and testing steps shown in Figure 12. The error histogram visualizes the errors between the target values and predicted/output values after training a neural network, while the more distributed to zero the error values are, the better the model performance is [72]. From this figure, it is clear that the model errors have spread around the zero error value, revealing that the ANN model has a high degree of reliability.

3.3.2. Model Error Visualization

After the ANN model training, the omission and commission errors, i.e., buildings that were wrongly identified within the testing dataset, were visualized and discussed. Out of the 203 patterns used in the testing step (15% of the total dataset), the developed ANN models resulted in 15 errors. The detection and analysis of the omission errors revealed that the model had more difficulty in recognizing changes in the building structure related to building collapses (total or partial), rather than other types of changes, such as debris removal from a site. Figure 13 depicts two examples of omission errors, which are associated with the weakness of the model in detecting changes in building structure. Ιn these examples, the way in which the six GLCM features relate to each other may be very close to the corresponding one that leads the neural network to classify the building as not having undergone any change; hence, the ANN cannot recognize the change, thus leading to an omission error.

On the other hand, the detection and interpretation of the commission errors revealed that the model affected either by image resolution or by image shading resulted in detecting changes in cases where they were not actually observed (Figure 14).

4. Conclusions

The present study attempted to develop a methodology that is based on the GLCM texture features extracted from UAV orthophotomaps and an ANN classifier in order to detect building changes during the recovery phase following an earthquake event. As indicated by the high overall accuracy and the relatively fast execution time, the ANN-based scheme of building change classification works quite properly in estimating the status of buildings in the study area. The error analysis showed that the ANN model was mainly affected by either image resolution or image shading. In general, as the present study concluded, the joint utilization of remote sensing image texture analysis through the GLCM method and neural network algorithm is of great efficiency for detecting changes after a natural disaster such as an earthquake. In future research, it would be of great interest to evaluate how the modern and more powerful CNNs can individually recognize the various types of building changes, i.e., building collapse (partial or total), new building construction and debris removal, to achieve an even more detailed monitoring of the area under study.