In this section, we begin by outlining the datasets and evaluation criteria utilized in our experiments, followed by a presentation of our method’s performance.
3.2. Experimental Results and Analysis
To assess the performance of the proposed method, it was compared against seven established methods: principal component analysis and K-means (PCAKM) [
42], Bayesian [
43], Gabor and two-level clustering (GaborTLC) [
29], logarithmic mean-based thresholding (LMT) [
44], neighborhood-based ratio and collaborative representation (NRCR) [
45], the adaptive generalized likelihood ratio test and maximum entropy principle for change detection in SAR images (AGLRTM) [
25], and the improved nonlocal patch-based graph and principal component analysis network (INPCANet) [
10]. For the parameter settings, the following are used: the
and
in the PCAKM method; in the Bayesian method, the low-pass filtering is carried out by averaging the complex values in a
sliding window; a median filter with a size of
is used in the LMT method; in the INPCANet method,
and neighborhood of size is
; in the NRCR and AGLRTM methods, the parameters are set to the default values set by the authors of the papers, respectively; and in the proposed method and the GaborTLC method,
,
,
,
, and
are defined.
In this subsection, we examine the difference images produced by various methods and discuss the change detection outcomes achieved using the two-level clustering model.
Figure 10 illustrates the difference images derived from the log-ratio operator (LR), mean-ratio operator (MR), and PCA fusion technique on the Village of Feltwell dataset, respectively.
To evaluate the efficacy of the difference images (DIs) calculated by the LR, MR, and PCA-fusion approaches, empirical receiver operating characteristic (ROC) curves are employed, as depicted in
Figure 11. These curves plot the true positive rate (TPR) against the false positive rate (FPR). Additionally, two quantitative criteria based on the ROC curve—the area under the curve (AUC) and the diagonal distance (Ddist)—along with their metrics [
46], are provided in
Table 3. For these criteria, larger values indicate superior detection performance. According to the data in
Table 3, the PCA fusion model outperforms both the LR and MR operators, showcasing its effectiveness in change detection scenarios.
Figure 12 presents the change detection outcomes for the Village of Feltwell dataset using the difference images processed with the two-level clustering model, with the associated performance metrics detailed in
Table 4. From these visuals and data, it is observed that the log-ratio with two-level clustering (LRTLC) method exhibits the fewest false positives, with an FP value of 77. Meanwhile, the mean-ratio with two-level clustering (MRTLC) approach shows the least number of missed detections, evidenced by the FN value of 124. The PCA with two-level clustering (PCATLC) method ranks second in terms of both FN and FP, but it achieves the best performance in overall error (OE), percentage of correct classifications (PCC), Kappa coefficient (KC), and F1-score, with values of 360, 99.77%, 95.54%, and 95.66%, respectively. These results demonstrate the superior change detection capability of the fused difference image approach.
- (1)
Results for the Village of Feltwell dataset:
The experimental outcomes for the Village of Feltwell dataset are displayed in
Figure 13, with the corresponding performance metrics provided in
Table 5. This figure reveals that the PCAKM, Bayesian, and LMT methods report high false positives, with values of 340, 2731, and 1863, respectively. Conversely, the NRCR, AGLRTM, and INPCANet approaches are characterized by high false negatives, with values of 1941, 2715, and 1337, respectively. The GaborTLC method has also achieved relatively good change detection effects, with the values of FN and FP being relatively balanced. Notably, the proposed method outperforms the others, showcasing the best overall performance, with an OE of 360, a PCC of 99.77%, a KC of 95.54%, an F1-score of 95.66%. The FN (269) and FP (91) both rank within the top three, respectively. These findings underscore the effectiveness of the proposed method in achieving accurate change detection.
- (2)
Results for the Ottawa dataset:
Figure 14 displays the experimental outcomes for the Ottawa dataset, with the related performance metrics detailed in
Table 6. From this figure, it is evident that the PCAKM and Bayesian methods result in a high number of false positives, with values of 583 and 3924, respectively. Meanwhile, the LMT, NRCR, and INPCANet approaches record high false negatives, with values of 5266, 4971, and 8709, respectively. The results of the GaborTLC and AGLRTM methods are similar. It is clear from the analysis that the overall performance of the proposed method surpasses that of the comparative methods, achieving the best results in overall error (OE) of 2316, the percentage of correct classifications (PCC) of 97.72%, a Kappa coefficient (KC) of 90.92%, and an F1-score of 92.25%. Additionally, FN (2258) is one of the top three, and FP (58) is one of the top four. These results highlight the superior accuracy and reliability of the proposed method in detecting changes within the Ottawa dataset.
- (3)
Results for the San Francisco dataset:
Figure 15 illustrates the experimental results for the San Francisco dataset, with the performance metrics detailed in
Table 7. The figure highlights the fact that the PCAKM, Bayesian, GaborTLC, and AGLRTM methods exhibit high rates of false detections, with false positive (FP) values of 1023, 923, 803, and 1204, respectively. On the other hand, the LMT, NRCR, and INPCANet methods demonstrate significant rates of missed detections, with false negative (FN) values of 1046, 598, and 1120, respectively. It is apparent from the analysis that the overall performance of the proposed method outshines the comparative methods, achieving the highest scores in overall error (OE) of 840, a percentage of correct classifications (PCC) of 98.72%, a Kappa coefficient (KC) of 90.16%, and an F1-score of 90.85%. Moreover, it maintains moderate values for both FN (513) and FP (327), further evidencing its superior ability to accurately detect changes within the San Francisco dataset.
- (4)
Results for the Yellow River dataset:
Figure 16 presents the experimental outcomes for the Yellow River dataset, with performance metrics detailed in
Table 8. This visualization reveals that the PCAKM, GaborTLC, NRCR, and INPCANet methods exhibit high rates of false detections, with false positive (FP) values of 4224, 1656, 2501, and 955, respectively. Conversely, the Bayesian, LMT, and AGLRTM methods show significant missed detection rates, with false negative (FN) values of 4422, 13404, and 6574, respectively. The overall performance of the proposed method stands out as the best among those evaluated, achieving the lowest overall error (OE) of 3635, the highest percentage of correct classifications (PCC) of 95.11%, the highest Kappa coefficient (KC) of 82.20%, and the highest F1-score of 85.09%. Additionally, it maintains moderate levels for both FN (3061) and FP (574), highlighting its effective balance in accurately detecting changes within the Yellow River dataset.
- (5)
Results for the Sulzberger Ice Shelf dataset:
Figure 17 displays the experimental results for the Sulzberger Ice Shelf dataset, with performance metrics outlined in
Table 9. This visual representation shows that the PCAKM, Bayesian, and INPCANet methods have considerable false detection issues, with false positive (FP) values of 733, 7734, and 1621, respectively. On the other hand, the LMT, NRCR, AGLRTM, and INPCANet methods exhibit notable missed detection issues, with false negative (FN) values of 5322, 466, 1173, and 2330, respectively. The GaborTLC method excels, achieving the lowest overall error (OE) of 512, the highest percentage of correct classifications (PCC) of 99.22%, the highest Kappa coefficient (KC) of 97.48%, and the highest F1-score of 97.96%. Our proposed algorithm is ranked second in overall change detection performance, with metrics of OE (747), PCC (98.86%), KC (96.34%), and F1-score (97.05%), which underscores the robustness of our approach in detecting changes.
3.3. Discussion
Based on the aforementioned results, we can summarize the performance of the proposed change detection method as follows: the proposed method was rigorously evaluated across five distinct SAR datasets: Village of Feltwell, Ottawa, San Francisco, Yellow River, and Sulzberger Ice Shelf, demonstrating its robustness and effectiveness in detecting changes. For the Yellow River dataset, although the range of variation in this dataset is relatively regular, the presence of significant noise and multiple targets results in comparatively low detection accuracy. By employing a combination of Gabor feature extraction and the two-level clustering model, including the fuzzy C-means (FCM) algorithm and PCA fusion, this method outperformed several established approaches such as PCAKM, Bayesian, GaborTLC, LMT, NRCR, AGLRTM, and INPCANet in various metrics.
The proposed method consistently achieved high performance metrics across all datasets, with notable scores in the overall error (OE), percentage of correct classifications (PCC), Kappa coefficient (KC), and F1-score, indicating a superior ability to accurately detect both changes and non-changes in complex SAR imagery. False positives (FPs) and false negatives (FNs) maintained moderate levels, demonstrating the method’s balanced sensitivity and specificity in identifying changed and unchanged areas, respectively. The PCA fusion model, in particular, was highlighted for its effectiveness in enhancing change detection performance by leveraging the complementary strengths of the log-ratio and mean-ratio operators. In our subsequent work, we will attempt to use algorithms such as edge-preserving filtering [
47], contourlet [
48], shearlet [
49,
50,
51], etc., for processing LR and MR fusion.
These results underscore the proposed method’s advanced capability in SAR image change detection, combining effective feature extraction with sophisticated clustering techniques to deliver accurate and reliable change maps. This method stands as a significant contribution to the field of remote sensing, especially for applications requiring precise monitoring and analysis of environmental changes, urban development, and disaster assessment.