Change Detection Based on Existing Vector Polygons and Up-to-Date Images Using an Attention-Based Multi-Scale ConvTransformer Network

Abstract

Vector polygons represent crucial survey data, serving as a cornerstone of national geographic censuses and forming essential data sources for detecting geographical changes. The timely update of these polygons is vital for governmental decision making and various industrial applications. However, the manual intervention required to update existing vector polygons using up-to-date high-resolution remote sensing (RS) images poses significant challenges and incurs substantial costs. To address this, we propose a novel change detection (CD) method for land cover vector polygons leveraging high-resolution RS images and deep learning techniques. Our approach begins by employing the boundary-preserved masking Simple Linear Iterative Clustering (SLIC) algorithm to segment RS images. Subsequently, an adaptive cropping approach automatically generates an initial sample set, followed by denoising using the efficient Visual Transformer and Class-Constrained Density Peak-Based (EViTCC-DP) method, resulting in a refined training set. Finally, an enhanced attention-based multi-scale ConvTransformer network (AMCT-Net) conducts fine-grained scene classification, integrating change rules and post-processing methods to identify changed vector polygons. Notably, our method stands out by employing an unsupervised approach to denoise the sample set, effectively transforming noisy samples into representative ones without requiring manual labeling, thus ensuring high automation. Experimental results on real datasets demonstrate significant improvements in model accuracy, with accuracy and recall rates reaching 92.08% and 91.34%, respectively, for the Nantong dataset, and 93.51% and 92.92%, respectively, for the Guantan dataset. Moreover, our approach shows great potential in updating existing vector data while effectively mitigating the high costs associated with acquiring training samples.

1. Introduction

Land cover change detection plays a crucial role in understanding dynamics on the Earth’s surface, with it being indispensable for applications such as land use analysis, environmental assessment, monitoring of human development, and disaster response [1,2,3,4,5]. The increasing availability of high-resolution remote sensing (RS) images has revolutionized the field of change detection, enabling detailed long-term monitoring of land cover dynamics [6]. The combination of these images with advancements in deep learning techniques offers unprecedented opportunities for automating the change detection process, thereby facilitating the timely updating of critical geographic datasets. However, despite these advancements, current methods used for change detection still face several challenges. The processing involved in updating existing vector polygons with new high-resolution RS images is time-consuming and labor-intensive, with manual intervention remaining a bottleneck, leading to inefficiencies and high costs. Furthermore, traditional methods for change detection through image differencing often struggle to handle noisy data and require large quantities of high-quality training samples, limiting their scalability and applicability in real-world scenarios [7,8,9,10]. Therefore, there is an urgent need for innovative approaches to overcome these limitations and enhance the efficiency and accuracy of change detection in high-resolution RS images.

To address the challenge of difficult sample acquisition, Zhao et al. [11] utilized linear spectrum hybrid analysis and spectral index method to extract ground object samples in an attempt to tackle sample selection difficulties in deep learning classification. Cui et al. [12] determined the optimal focus radius for different land types based on focus statistics and unique phenological characteristics, and subsequently proposed an approach for the automatic generation of training samples using an enhanced distance measure. Cao et al. [13] introduced a comprehensive fused cross-task transfer learning method (FFCTL) that effectively utilizes crowdsourced building data and high-resolution satellite images to address the issue of expensive real samples in building change detection. Lv et al. [7] presented an iterative training sample augmentation (ITSA) strategy combined with a deep learning neural network to enhance change detection performance. Li et al. [14] developed a label-noise active learning sample collection method for multi-temporal land cover classification. However, these studies did not consider the uncertainty inherent in the training data, which may introduce errors into the final results. Therefore, addressing the denoising problem associated with training samples is crucial for accurate change detection [15].

In addition, feature extraction plays a crucial role in object-based change detection. Deep learning is widely employed in RS image change detection due to its robust feature extraction and modeling capabilities. For instance, Zhang et al. [16] proposed a deep learning change detection framework to detect newly built buildings by paying more attention to the overall features and contextual associations of the change object instances. Gu et al. [17] introduced a multi-scale convolutional layer feature fusion network to achieve high-precision image change detection by addressing pseudo-changes and reducing the loss of details in the detection process. Despite the excellent performance of Convolutional Neural Networks (CNNs) in extracting relevant multi-scale features from images, they have limitations in establishing long-range dependencies of self-attention within images. The Transformer model has made significant advancements in image recognition and computer vision due to its efficient processing of contextual information and global modeling ability [18,19,20,21,22]. Recent studies have applied Transformer-based architectures to RS tasks, such as multi-modal image processing [23] and scene classification, which is carried out utilizing a Vision Transformer (ViT) [24].

Despite the enhanced perspectives ViT offers for image modeling, it frequently faces challenges related to high computational complexity and memory usage. Addressing this concern, Chen et al. [20] devised an efficient dual-path Transformer architecture. This innovative approach has led to achieving state-of-the-art accuracy on benchmark building extraction datasets. Furthermore, integrating the strengths of CNNs and Transformers to enhance change detection capability has become a current research focus and has yielded significant results [25,26,27,28]. While these advanced deep learning methods hold significant promise in image analysis, their application in change detection encounters challenges due to the inherent complexity of high-resolution RS images. These challenges include, but are not limited to, diverse ground objects, image noise, and seasonal changes. Specifically, difficulties arise in high label cost, accurate detection of multi-scale ground objects, data imbalances, and the substantial demand for computing resources. Consequently, many current methods for multi-temporal image change detection, which are based on deep learning, encounter difficulties in direct applicability to practical tasks. While recent developments in RS foundation models, such as SAM-enhanced [29], RingMo [30], and RSPrompter [31], have exhibited excellent generalization ability and zero-shot learning capability, they still confront challenges related to reducing human intervention, decreasing computing resource requirements, and accommodating diverse downstream task applications [21,32].

The effective utilization of prior knowledge, such as vector data, is crucial for change detection. However, currently, few studies integrate vector and image data for change detection methods. Vector data contain boundary and categorical information of ground objects, with a large number of valuable vector polygons amassed by land survey projects and other endeavors. Unlike image data, it offers essential support for image segmentation, sample annotation, and classification tasks. Zhang et al. [33] proposed a change detection method based on vector data and the isolated forest algorithm. This method utilizes vector constraints with category information from an old time phase to finely segment a new time-phase RS image, obtaining the plot, and applies the isolated forest to calculate the change index of the plot. However, its efficacy hinges on the assumption that the proportion of various ground object change plots is small, and it imposes high computational demands. Wei et al. [34] replaced the historical RS image with vector data, and introduced a new method to detect changes from single-phase RS image and vector data using the outlier index of texture feature space. However, the texture-based approach limits the widespread adoption of the method.

The automatic generation of labeled samples based on vector boundary constraints has garnered significant attention from scholars in recent years, holding considerable promise for application in vector data-aided change detection within the field [35,36,37]. However, due to registration errors, semantic gaps among ground objects, land cover changes, variations in annotation personnel, and other factors, label noise inevitably exists when directly utilizing vector attributes for sample annotation within RS datasets. Consequently, this leads to an incomplete capture of the image’s semantic content by the training model and a subsequent decline in generated feature discrimination ability [35,38,39,40,41,42]. Currently, denoising methods for samples have been initially developed. Li et al. [43] employed superpixel segmentation on the image to delineate image objects and extracted spectral and texture features. They then analyzed the distribution of homogeneity characteristic values within cultivated land and applied a box plot anomaly detection method to eliminate noisy samples, which was employed in monitoring the non-agriculturalization of cultivated land. Kang et al. [44] utilized an energy constraint minimization criterion for hyperspectral image target detection and effectively corrected training samples. However, these studies only considered abnormal removal from pixel values and texture features and did not integrate the actual needs of change detection. In summary, the theory and technology behind the change detection methods integrating vector data and images are still immature, and the automatic generation of samples and efficient unsupervised denoising methods are lacking, requiring further research.

To address the aforementioned issues, this paper proposes a novel change detection method for detecting changed land cover vector polygons using high-resolution RS images and deep learning. Figure 1 shows the differences between our method and conventional change detection methods.

The key contributions of this work are delineated as follows:

• We introduce a framework for detecting changes in vector polygons utilizing single-temporal high-resolution RS images and deep learning. This framework enables end-to-end application, encompassing image preprocessing through change detection, requiring solely up-to-date images and corresponding land cover vector data from the previous time image. This method offers a comprehensive bottom-up solution.
• For sample construction, we propose boundary-preserved masking Simple Linear Iterative Clustering (SLIC) for generating superpixels. These are then combined with land cover vector data to create an adaptive sample cropping scheme. To address noise, we introduce an efficient Visual Transformer and class-constrained Density Peak-based (EViTCC-DP) method for noisy label removal, followed by the transformation of noisy samples into representative ones using k-means clustering, resulting in the automatic generation of a high-quality multi-scale sample set.
• To enhance fine-grained scene classification precision, we employ an improved attention-based multi-scale ConvTransformer network (AMCT-Net) for superpixel cropping unit classification. By integrating a CNN structure and Transformer, along with the attention mechanism module, we achieve a more discriminative feature representation, enhancing the model’s classification accuracy. Additionally, we introduce a change decisionmaker with various rules, which synergistically combines and post-processes sample predictions with land cover vector data to effectively extract changed vector polygons.

The work is organized as follows. Section 2 introduces the proposed methodology in detail, consisting of five main parts: Density Peak (DP) clustering, automated generation of initial samples with vector boundary constraints, initial samples denoising based on DP clustering algorithm, AMCT-Net, and vector polygons change detection analysis based on confidence rules. Section 3 provides details of the datasets, describes the experiments, and analyzes the results. Section 4 presents the discussion. Finally, Section 5 concludes this paper.

2. Methodology

Most existing change detection algorithms rely heavily on image contrast and require high-quality training samples, which are often challenging to acquire. This limitation impedes their effectiveness in large-scale practical change detection scenarios. In this work, we propose a novel framework for detecting changes in vector polygons using high-resolution RS images, as illustrated in Figure 2. Our framework comprises an end-to-end processing architecture consisting of four primary stages. Initially, we employ a boundary-preserved masking SLIC [45] algorithm for image superpixel segmentation, leveraging high-resolution RS images and land cover vector data as inputs. Subsequently, adaptive cropping is performed on each superpixel unit to generate initial samples, which undergo denoising via the EViTCC-DP method, and the inclusion of representative training samples (RTS) is introduced to construct the final high-quality training sample set. These refined samples are then subjected to fine-grained image classification through an improved attention-based multi-scale ConvTransformer network (AMCT-Net). Finally, a change decisionmaker applies various rules to detect and locate changes within vector polygons. A detailed elaboration of each module will be provided subsequently.

2.1. Density Peak Clustering

In contrast to the k-means algorithm [46,47], which necessitates the specification of the cluster number, the Density Peak (DP) algorithm [48] can identify cluster centers without iterative procedures. Operating non-iteratively, it adheres to a straightforward principle and has showcased outstanding clustering efficacy in handling both spherical clusters and non-convex datasets. The fundamental principles are as follows:

• The density around the cluster center should be relatively high;
• The cluster center should be situated at a considerable distance from points with higher surrounding density.

The DP clustering algorithm is based on the aforementioned concept to achieve data clustering, where local density ${\mathit{\rho }}_i$ and relative distance ${\mathit{\delta }}_i$ are two crucial factors that significantly influence the outcomes of the DP algorithm.

Figure 3 provides an example that illustrates the fundamental concept of the DP clustering algorithm. The cluster centers (designated as 1 and 10) are encompassed by samples from the same class, which exhibit lower local densities compared to the cluster centers (see Figure 3). Additionally, samples located far from the class centers typically exhibit very low local densities.

The main steps of the DP clustering algorithm are outlined as follows. Given a dataset $\mathbf{X}\in {\mathbb{R}}^M$, where $M$ represents the number of samples, the Euclidean distance ${\mathit{d}}_{ij}$ between the samples ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$ can be calculated as follows:

$${\mathit{d}}_{ij}={‖{\mathit{X}}_i-{\mathit{X}}_j‖}_2^2.$$

(1)

The local density ${\mathit{\rho }}_i$ of each sample can be obtained as follows:

$${\mathit{\rho }}_i={\sum }_{j\ne i}\chi ({\mathit{d}}_{ij}-d_c)$$

(2)

where $d_c$ is the cut-off distance and is constrained by a parameter P, which takes a value between 10 and 30 [48]. In Equation (2), $\mathrm{\chi }(u)=\left\{\begin{array}{c}1,u<0\\ 0,u\ge 0\end{array}\right.$.

After computing ${\mathit{\rho }}_i$, ${\mathit{\delta }}_i$ is defined as follows:

$${\mathit{\delta }}_i=\left\{\begin{array}{c}\underset{j}{\mathit{max}}\left({\mathit{d}}_{ij}\right),\hspace{1em}\hspace{1em}if{\mathit{\rho }}_i=\mathit{max}\left(\rho \right)\\ \underset{j:{\mathit{\rho }}_j>{\mathit{\rho }}_i}{\mathit{min}}\left({\mathit{d}}_{ij}\right),\hspace{1em}\hspace{1em}Otherwise.\end{array}\right.$$

(3)

Generally, ${\mathit{\delta }}_i$ refers to the minimum distance between sample i and other samples with a higher density than sample $i$. In special cases, ${\mathit{\delta }}_i$ refers to the maximum distance between sample $i$ and samples other than sample $i$. Samples with relatively higher ${\mathit{\rho }}_i$ and ${\mathit{\delta }}_i$ values will be identified as clustering centers. Therefore, an index ${\mathit{\gamma }}_i$ is defined as follows:

$${\mathit{\gamma }}_i={\mathit{\rho }}_i\times {\mathit{\delta }}_i.$$

(4)

The samples with higher ${\mathit{\gamma }}_i$ values are more likely to be clustering centers. Therefore, the clustering center in the dataset can be easily found using a sorting algorithm. In Equation (2), outlier samples are detected by a hard index constraint, such as $d_c$. In accordance with previous studies [39,40], this paper adopts a soft Gaussian kernel function to detect the outlier probability of samples, which is defined as follows:

$${\mathit{\rho }}_i={\sum }_j{e}^{-{(\frac{{\mathit{d}}_{ij}}{d_c})}^2}.$$

(5)

The advantage of the soft Gaussian function is that it can reduce the negative impact of statistical errors caused by fewer samples in some categories.

2.2. Automated Generation of Initial Samples with Vector Boundary Constraints

2.2.1. Automatic Generation of Initial Samples

The quality of sample generation and the accuracy of vector polygons change detection in subsequent stages are directly influenced by the segmentation results of images. While the SLIC algorithm can rapidly generate uniform and compact superpixels, it does not guarantee a perfect fit between superpixels and ground object boundaries, potentially leading to boundary exceedance situations [49].

To address this issue, we propose a novel approach termed boundary-preserved masking SLIC. By incorporating prior knowledge of vector boundaries, our approach significantly enhances the accuracy of image segmentation, as illustrated in Figure 4.

The details of the algorithm are as follows:

1. Given the number of seed points (N), to distribute them uniformly across the region, each seed point is positioned furthest from the boundary and from any other seed point. The seed point ($P$) is calculated as follows:

$$P=\underset{x}{\mathrm{argmax}}\left({‖x-y‖}_2\right),$$

(6)

where $y$ represents a point on the boundary or a seed point and $x$ represents another point within the region. The boundary of the region is derived from a polygon. To ensure a rational spatial distribution of seed points, $N$ is calculated using a ratio, as depicted in (7), where K denotes the total number of pixels in the mask and $C_0$ denotes the preset cropping size for the sample. Assuming each superpixel is uniformly square, the edge length is defined as $S$, where $S={m\times C}_0$. The parameter $m$ is introduced to preserve specific proximity information while ensuring the integrity of the features in the generated data. Typically, $0<m\le 1$.

$$N=K|({{m\times C}_0)}^2$$

(7)

2. The image is converted from RGB color space to CIELAB color space to facilitate the measurement of color differences. The weighted distance $d$ between color and space, used in clustering, can assess the similarity between pixels, as expressed in Equation (9), where $w$ is the weight balancing the color distance ($d_c$) and spatial distance ($d_s$). To minimize repeated searches, a circular area with the seed point as the center and a radius of $2S$ is defined as the search range during iterative clustering. Subsequently, any isolated small clusters are merged with adjacent larger clusters based on lightness similarity $d_l={(\mu -{\mu }_m)}^2$, where $\mu$ and ${\mu }_m$ are the average lightness of the small cluster and the neighboring cluster, respectively.

$$d_c=\sqrt{{\sum }_{n\in l,a,b}{(n_j-n_i)}^2}$$

(8)

$$d=\sqrt{{\left(d_c\right)}^2+{\left(d_s/w\right)}^2}$$

(9)

The segmentation accuracy at the image boundary is effectively enhanced by this method, as demonstrated in Figure 5. Following image segmentation, we employed an adaptive cropping method to automatically generate initial samples. Figure 6 illustrates the process of adaptive image cropping. With the initial cropping size $C$ of the sample, superpixels are utilized as processing units for adaptive cropping, relying on the results of image superpixel segmentation to generate the sample. Nevertheless, increasing the cropping scale raises the likelihood of generating samples that contain features from other land cover types, resulting in a decrease in sample quality. Additionally, when cropping samples along the edges of imagery, areas extending beyond the image boundaries may result in null values (Nodata), as shown in Figure 6a. To address these issues, we propose a sample-adaptive cropping method with a formula for determining the cropping size $C$ as follows:

$$C=\left\{\begin{array}{c}{C}_0,d_b>C_0\\ d_b,d_b\le C_0\end{array}\right.,$$

(10)

where $C_0$ is the preset size, and $d_b$ is the shortest distance between the center and boundary. Finally, pure multi-scale samples were generated based on the constraints of vector polygons boundaries.

2.2.2. Source of Noise Samples

The sources of noise samples primarily encompass the following aspects. Firstly, they encompass discrepancies between vector boundaries and actual ground object boundaries due to variations in operating personnel and standards contribute to noise generation, as shown in Figure 7. Secondly, the formation of vector survey results often takes a certain time period, leading to disparities between land class attributes derived from vector polygons and those obtained from high-resolution RS images (see Figure 8). Consequently, it is not feasible to directly extract labeled samples for training solely based on vector polygons depicted on high-resolution RS images. In view of this, DP clustering is introduced for initial samples denoising with the aim of addressing the issue of automatic sample set construction.

2.3. Initial Samples Denoising Based on DP Clustering Algorithm

Sample denoising plays a crucial role in acquiring high-quality training samples, thereby enhancing the classification accuracy of the model [50,51]. Consequently, this paper introduces the EViTCC-DP approach, as illustrated in Figure 9. The approach comprises the following steps:

• Employing boundary constraints of vector polygons and adaptive cropping of RS images to automatically generate initial samples and train ViT models.
• Utilizing the pre-trained ViT to extract features from scene samples and inputting them into the DP clustering algorithm according to class constraints to achieve the purpose of denoising.

The approach uses the vector polygon attribute to classify the samples, thereby eliminating the need to define the cluster center in the DP clustering algorithm, and can construct a high-quality multi-scale sample set with minimal manual intervention. In this paper, we introduce the DP clustering algorithm for removing noise samples in RS image scene classification for the first time. Additionally, this paper improves classification accuracy in subsequent stages by transforming noise samples into RTS under the supervision of k-means clustering. The k-means algorithm surpasses random selection in terms of selecting optimal clustering centers while ensuring consistency between cluster numbers and label category data, thus obviating the requirement for manual definition.

The implementation steps are described as follows:

1. Calculating the distance of training samples

Firstly, image feature embeddings are extracted for each sample using a pre-trained ViT model. Then, let $\mathbf{X}=\left\{{\mathbf{X}}^1,{\mathbf{X}}^2,\cdots ,{\mathbf{X}}^M\right\}$ refer to the feature embeddings of the initial samples, where $\mathrm{M}$ denotes the number of classes and $X^m$ refers to the training samples in the mth class. For two training samples belonging to the mth class, i.e., ${\mathbf{X}}_a^j$ and ${\mathbf{X}}_b^j$, the distance $d_{ab}^m$ between two samples can be measured. In this paper, Euclidean distance [39] is used as the distance measurement between training samples, with the distance ${\mathit{d}}_{ab}^j$ between ${\mathbf{X}}_a^j$ and ${\mathbf{X}}_b^j$ being able to be calculated as follows:

$${\mathit{d}}_{ab}^j={‖{\mathbf{X}}_a^j-{\mathbf{X}}_b^j‖}_2^2.$$

(11)

Through calculating the distances among the nth scene sample and the scene samples in the mth class, a distance array ${\mathit{d}}_n^m$ of the nth scene sample can be constructed as follows:

$${\mathit{d}}_n^m={\left[{\mathit{d}}_{n1}^m,{\mathit{d}}_{n2}^m,\cdots ,{\mathit{d}}_{n{N}_m}^m\right]}^T$$

(12)

where $N_m$ refers to the number of samples in the mth class. In this way, a distance matrix ${\mathit{D}}^m$ can be constructed as ${\mathit{D}}^m=\left\{{\mathit{d}}_1^m,{\mathit{d}}_2^m,\cdots ,{\mathit{d}}_{N_m}^m\right\}$.

2. Calculating the local density of the training samples

The cut-off distance $d_c^m$ can be calculated as follows:

$$d_c^m={\mathbf{S}}^m(t)\mathrm{s}.\mathrm{t}.t=\langle \frac{N_m·(N_m-1)}{100}·P\rangle$$

(13)

where ${\mathbf{S}}^m$ is a matrix that sorts the non-zero elements in the upper triangular matrix of ${\mathit{D}}^m$ from the smallest to the largest elements, $P$ is a free parameter that will be analyzed in Section 4.1, and < · > refers to the round operation.

With the above-obtained $d_c^m$, the local densities ${\mathit{\rho }}^m=\left\{{\mathit{\rho }}_1^m,{\mathit{\rho }}_2^m,\cdots ,{\mathit{\rho }}_{N_m}^m\right\}$ of the samples in the mth class can be calculated as follows:

$${\mathit{\rho }}^m=\sum e^{-{\left(\frac{{\mathit{D}}^m}{d_c^m}\right)}^2}.$$

(14)

3. Detecting the mislabeled samples in each class

Once the local densities of the training samples in different classes are obtained, mislabeled samples can be easily detected and removed as follows:

$${\mathbf{Y}}_i^m=\left\{\begin{array}{l}{\mathbf{X}}_i^m\hspace{1em}if{\rho }_{N_m}^m\ge \lambda ·{\overline{\mathit{\rho }}}^m\\ \varnothing \hspace{1em} \mathrm{Otherwise}\end{array}\right.$$

(15)

where $\mathbf{Y}=\left\{{\mathbf{Y}}^1,{\mathbf{Y}}^2,\cdots ,{\mathbf{Y}}^M\right\}$ refers to the resulting training set, in which the noisy labels are detected and removed. $\lambda$ is a free parameter.

4. Converting the mislabeled samples to representative samples

The scene sample ${\mathbf{X}}_a^j$ represents noise for the mth class; however, it may be a valuable representative sample for the nth class. In this paper, the final high-quality sample set can be automatically constructed as follows:

$${\widehat{\mathbf{Y}}}_i^m={\mathbf{Y}}_i^m+\mathrm{k}{\mathrm{\varnothing }}_{class}^j$$

(16)

where $\widehat{\mathbf{Y}}=\left\{{\widehat{\mathbf{Y}}}^1,{\widehat{\mathbf{Y}}}^2,\cdots ,{\widehat{\mathbf{Y}}}^M\right\}$ refers to the final high-quality training scene sample set in which the noisy labels are supervised and placed into the correct class using the k-means clustering algorithm, and $\mathrm{k}{\mathrm{\varnothing }}_{class}^j$ represents the k-means clustering algorithm.

2.4. Attention-Based Multi-Scale ConvTransformer Network, AMCT-Net

2.4.1. Overview of the Proposed AMCT-Net

The samples are generated using vector polygons and high-resolution RS images, where each element requires fine-grained image classification [52]. Traditional methods for image classification encounter challenges in capturing the intricate features of an image. However, the fusion of the Convolutional Neural Network (CNN) and Transformer enables the extraction of both global and local features, thereby enhancing classification accuracy [17,23,27,53]. In this article, we introduce an attention-based multi-scale ConvTransformer network (AMCT-Net), depicted in Figure 10. AMCT-Net integrates a spatial attention mechanism and a multi-scale feature extraction module into the Transformer architecture to enhance fine-grained classification accuracy.

The proposed AMCT-Net adopts a Siamese structure for local feature extraction, comprising four Encoder Residual Modules (ERMs) that generate feature maps of varying sizes with channel dimensions of 16, 32, 64, and 128, respectively. Subsequently, to enhance model performance, we incorporate an attention mechanism known as the Efficient Convolution Spatial Attention Module (ECSAM) at the backend of ERM. ECSAM prioritizes channel features and effectively leverages low- and high-level features [54]. Finally, ECSAM feature maps of different sizes are transformed into flattened feature patch tokens with uniform shapes. Similar to the original Transformer [55], we flatten the patches and map them to D dimensions using a trainable linear projection. We refer to the output of this projection as the patch embeddings. Position embeddings are added to the patch embeddings to retain positional information. The resulting sequence of embedding vectors is utilized as input to the Adapter-Scale Transformer Encoder for classification, facilitating the extraction of global features. Following an overview of the general motivation and architecture of our proposed method, we proceed to describe each main improved module in detail.

2.4.2. Module Details

1. Efficient Convolution Spatial Attention Module

Generally, spatial attention mechanisms are employed to assist models in focusing on crucial image regions. However, computing weights between features across all positions entails significant computational effort. In this study, drawing inspiration from the concept of “coordinate separation” [56], we propose the ECSAM to effectively capture cross-channel relationships and long-range dependencies while incorporating specific positional information.

The detailed architecture of ECSAM is illustrated in Figure 11. First, the input feature map, denoted as $F\in {\mathbb{R}}^{H\times W\times C}$, undergoes horizontal avg-pooling of size $(\mathrm{H},1)$ and vertical avg-pooling of size $(1,\mathrm{W})$ for each channel of the feature map, respectively. These operations are defined as follows:

$$F_c^H\left(h\right)=\frac{1}{W}\sum _{0\le x<W}{F}_c(h,x)$$

(17)

$$F_c^W\left(w\right)=\frac{1}{H}\sum _{0\le y<H}{F}_c(y,w)$$

(18)

where $F_c^H\left(h\right)$ represents the horizontal avg-pooled feature of the c-th channel at height $\mathrm{h}$, and $F_c^W\left(w\right)$ represents the vertical avg-pooled feature of the c-th channel at width $\mathrm{w}$.

To prevent reduction in channel dimensionality during cross-channel interactions, ECSAM employs 1D convolution extended to 2D with an adaptive kernel size for generating attention weights along two spatial directions, respectively. These operations are formulated as

$${\widehat{F}}_c^H\left(h\right)=ReLU(C2D_k(F_c^H(h)))$$

(19)

$${\widehat{F}}_c^W\left(w\right)=ReLU(C2D_k(F_c^W(w)))$$

(20)

where ${\widehat{F}}_c^H\left(h\right)$ denotes the horizontal feature vector of the c-th channel at height h, ${\widehat{F}}_c^W\left(w\right)$ denotes the vertical feature vector of the c-th channel at width w, and $C2D_k$ represents 2D convolution with kernel size of k, where k can be calculated as

$$\mathrm{k}={\left|\frac{{\log }_2\left(C\right)+1}{2}\right|}_{odd}$$

(21)

where ${\left|\mathrm{t}\right|}_{odd}$ represents the nearest odd number of $\mathrm{t}$, and $C$ denotes the number of channels in $F$.

2. Encoder Residual Module

The residual module structure of the encoder is shown in Figure 12a. Initially, the input features traverse a convolutional layer (Conv), followed by batch normalization (BN) and rectified linear unit (ReLU) activation. Subsequently, another combination of Conv and BN is introduced. Ultimately, the output is derived by adding the outcome of the second BN to the outcome of the initial Conv, followed by another ReLU. The Conv kernels are sized at 3 × 3. Employing residual connections and activation function layers serves to accelerate network learning and mitigate the problem of gradient vanishing [25].

3. Adapter-Scale Module

The architecture of the Adapter-Scale Module (ASM) is depicted in Figure 12b. It comprises three components: Downscale, ReLU, and Upscale. The Downscale segment utilizes a single Multi-Layer Perceptron (MLP) layer to reduce the dimensionality of the embedding. Subsequently, the ReLU activation function is applied, and the embedding is restored to its original dimensionality through another MLP layer in the Upscale segment. Two ASMs are integrated into the ViT block. The first is positioned before the multi-head attention blocks and residual connections, while the second is embedded within the residual structure of the MLP. Additionally, a scale factor of 0.5 is applied to each adapter.

2.5. Vector Polygons Change Detection Analysis Based on Confidence Rules

The set of image polygons generated by the post-processing unit $R_i$ after interpretation is denoted as $\left\{P_N^i\right\}$, where N represents the total number of image polygons. Let $y_N^i$ refer to the classification attribute confidence of $P_N^i$, with $y_N^i$ being able to be calculated by counting the occurrences of $\left\{x\right\}$, where $\left\{x\right\}$ is the set of class attribute values. The attribute $X_{P_N^i}$ of $P_N^i$ is subsequently computed based on the maximum confidence rule. The operation is formulated as follows:

$$X_{P_N^i}=argmax\left\{y_N^i\right\}.$$

(22)

Finally, the vector polygon attribute value $Y$ can be obtained with

$$Y=argmax\left\{S_x\right\}$$

(23)

where $x$ is the attribute of different types of processing units after interpretation, and $S_x$ is the area corresponding to $x$.

The threshold $k$ for change detection can be calculated with

$$k=\frac{S_Y}{S_{total}}\times 100\mathrm{\%}$$

(24)

where $S_Y$ is the area corresponding to $Y$, and $S_{total}$ is the total area of the vector polygon.

In this experiment, the threshold $k$ was set at 0.9. The changed vector polygons were identified based on the “prediction category <> initial category” and a developed change decisionmaker, as illustrated in Figure 13. A detailed explanation of the change decisionmaker will be provided in Section 3.2.1.

3. Experiments and Results

3.1. Description of Data Sources and Research Scheme

The RS images and vector data used in this study were provided by the land change survey project conducted by the Jiangsu Institute of Geology and Surveying. The Nantong dataset is situated in Nantong Development Zone, Nantong City, Jiangsu Province, China (see Figure 14). The existing vector data depicted in Figure 14a were collected during the third land survey conducted in November 2021. The images presented in Figure 14b,c were obtained from the Beijing-2 satellite with a spatial resolution of 0.8 m in October 2021 and October 2022. Similarly, the Guantan dataset is located in Guantan Town, Xuyi County, Huai’an City, Jiangsu Province (refer to Figure 15). Moreover, the existing vector data shown in Figure 15a were also collected during the third land survey conducted in November 2021. The images presented in Figure 15b,c were acquired from the GF-2 satellite with a spatial resolution of 1m, with images also being taken in October 2021 and October 2022. Temporal image details are illustrated in Figure 14d and Figure 15d, with land use types within these areas primarily comprising buildings, cropland, forest land, industrial land, paddy fields, roads, and water bodies. Considering the spatial resolution of the images, the standard sample size for both datasets in the experiment was set at 96 pixels, with a minimum size of 32 pixels. The final multi-scale sample sizes range from 32 × 32 to 96 × 96 pixels, as depicted in Figure 14e and Figure 15e.

After automatically generating the initial sample set, we obtained a high-quality training sample set using the EViTCC-DP method. Specifically, the sample categories and quantities for the two datasets are as shown in

Table 1. Sample Information of the Nantong dataset and the Guantan dataset.

Number	Land Use Types	Nantong Dataset	Guantan Dataset
C1	buildings	5243	4849
C2	cropland	2647	1744
C3	forest	3201	3768
C4	industrial	5844	4160
C5	paddy field	4848	5115
C6	road	2738	1240
C7	water	5690	5139

.

The experiments were conducted on a computer equipped with 64GB of RAM and an Intel Xeon CPU E-2186M @ 2.9GHz processor. The training process utilized the NVIDIA GeForce GTX 1080 Ti with 11GB of memory. The source code for the proposed ConvTransformer was implemented using PyTorch 1.5.1+cu92 and Python 3.7. During training, the stochastic gradient descent algorithm was employed, incorporating a momentum value of 0.9 and a weight decay penalty coefficient of 10⁻⁵. The initial learning rate was set to 0.001 and decayed following a cosine annealing schedule. A batch size of 64 was utilized.

3.2. Results

3.2.1. Change Detection and Post-Processing

There exists a significant semantic disparity between RS images and existing vector polygons, resulting in the potential inclusion of non-subject objects within the subject object, thereby leading to impure land classification within vector polygons. To achieve reliable change detection results in accordance with task requirements, we have developed a “change decisionmaker” (see Figure 16) that effectively mitigates the influence of non-subject objects through the implementation of distinct thresholds. As depicted in Figure 16, vector polygons are categorized into two types, namely ‘sensitive’ and ‘non-sensitive’. The ‘sensitive’ type [see Figure 16a] represents the most stringent change scenario, involving the presence of new buildings in croplands, which is prohibited by law in certain countries. On the other hand, other scenarios can be classified as the ‘non-sensitive’ type [see Figure 16b], which do not violate legal regulations or relevant rules and receive comparatively less attention than the ‘sensitive’ type.

3.2.2. Evaluation Metrics

To compare the performance of our method with that of traditional models, we employed overall accuracy (OA), Precision, Recall, F1 score, and specificity as the primary quantitative metrics. A higher value for each metric indicates superior model performance. The F1 score considers both precision and recall of the classification model on a scale from 0 to 1. Furthermore, tests with high specificity indicate a lower Class I error rate [57].

The definitions of these metrics are outlined as follows:

$$\mathrm{O}\mathrm{A}=\frac{TP+TN}{TP+FP+TN+FN}$$

(25)

$$Precision=\frac{TP}{TP+FP}$$

(26)

$$Recall=\frac{TP}{TP+FN}$$

(27)

$$F1=2·\frac{Precision·Recall}{Precision+Recall}$$

(28)

$$\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TN}{TN+FP}$$

(29)

where TP denotes true positives, FP denotes false positives, TN denotes true negatives, and FN denotes false negatives.

3.2.3. Vector Polygons Change Detection Results

The accuracy of subsequent vector polygons’ change detection is directly influenced by the classification accuracy of the proposed method [58], which adopts a post-classification comparison approach. To assess the classification performance of the proposed AMCT-Net model, we conducted comparisons with several other state-of-the-art models, including ViT [59], MTC-Net [53], and HCTM [25]. Specifically, the ViT model utilizes the encoder module of the Visual Change Transformer (VcT), MTC-Net combines the advantages of the multi-scale Transformer with the convolutional block attention module (CBAM), and HCTM represents a hybrid CNN-Transformer model. Before training the model, 80% of samples from each dataset are allocated for training purposes, while the remaining 20% are reserved for accuracy validation.

To validate the effectiveness and robustness of the proposed method in denoising samples, we conducted comparative experiments using consistent training and test sets. All models were evaluated under the same baseline conditions, and the classification results on the two datasets are presented in

Table 2. Comparison of the classification metrics of different models for the two datasets.

Model	Metrics	Nantong Dataset	Guantan Dataset
	Accuracy	0.9068	0.9165
	Precision	0.8938	0.8914
ViT	Recall	0.8903	0.9072
	Specificity	0.9816	0.9811
	F1 Score	0.8921	0.8992
MTC-Net	Accuracy	0.9117	0.9164
	Precision	0.9103	0.8969
	Recall	0.8904	0.9203
	Specificity	0.9824	0.9811
	F1 Score	0.8985	0.8996
HCTM	Accuracy	0.9135	0.9292
	Precision	0.8975	0.9230
	Recall	0.9109	0.9100
	Specificity	0.9817	0.9859
	F1 Score	0.9182	0.9215
AMCT-Net (ours)	Accuracy	0.9134	0.9351
	Precision	0.9179	0.9228
	Recall	0.9134	0.9292
	Specificity	0.9839	0.9898
	F1 Score	0.9201	0.9306

Color convention: best in red; second best in blue.

.

We can see the following from Table 2:

• Across both datasets, the baseline model (ViT) exhibits unsatisfactory performance across the five evaluation metrics, while the enhanced model incorporating attention mechanisms and a multi-scale convolution module demonstrates a notable improvement in accuracy. Notably, the AMCT-Net model outperforms other architectures in terms of Recall, specificity, and F1 score. Specifically, on the Nantong dataset, AMCT-Net achieves a Recall of 0.9134, specificity of 0.9839, and F1 score of 0.9201, representing a 0.25% increase in Recall compared to the sub-optimal HCTM model (Recall = 0.9109). On the Guantan dataset, AMCT-Net’s Recall reaches 0.9292, specificity stands at 0.9898, and F1 score is 0.9306, with a significant 1.92% increase in Recall compared to the sub-optimal HCTM model (Recall = 0.9100). This underscores the substantial advancement in classification accuracy achieved by AMCT-Net.
• It is noteworthy that the performance of the model differs between the two datasets. For instance, the proposed AMCT-Net model only marginally improves accuracy by 0.66% compared to the baseline model on the Nantong dataset. Conversely, the model exhibits a much more substantial improvement in accuracy on the Guantan dataset, with an increase of 1.86% compared to the baseline model. The variation in performance may be attributed to the urban development context of the Nantong dataset, where change types are inherently more complex compared to the Guantan dataset. However, as AMCT-Net integrates the local feature extraction capabilities of CNN structures with the global information processing characteristics of Transformer architecture, supplemented by the introduction of a multi-scale module, these enhancements prove particularly advantageous for processing the multi-scale sample set in this study, underscoring its adaptability to diverse dataset features.

Following image classification, the detection of change vector polygons is achieved through a change decisionmaker and post-processing rules. Figure 17a,c illustrate the change detection outcomes for the two datasets, while the confusion matrix generated by the proposed AMCT-Net model in this study is presented in Figure 18. Apart from confusion between roads and buildings, there are instances of misclassification in industrial areas. This is due to the spatial distribution, color form, and density similarities between industrial areas, buildings, and roads, posing a challenge in scene classification. Moreover, croplands share local semantic characteristics with paddy fields, leading to potential misclassifications. Taking Figure 18 as an example, only croplands exhibit a classification accuracy below 90%. Nonetheless, the experimental model effectively identifies other scene categories with high recognition accuracy. Notably, some forest test samples contain mixed category information, such as croplands and paddy fields surrounding forest farms, resulting in a lower classification accuracy for this category. Additionally, for the Guantan dataset, the classification accuracy of water bodies is not 100% due to aquaculture activities in the research area, including the phenomenon of digging breeding pits in some croplands, leading to confusion between paddy fields and breeding pits. Overall, the AMCT-Net in this study achieves a high classification accuracy, demonstrating satisfactory performance.

The visual interpretation results of the proposed method are illustrated in Figure 19, showcasing typical examples. Part (a) demonstrates the conversion of cropland into buildings, part (b) exhibits the construction of a road within the cropland, part (c) displays the transformation of part of the cropland into an industrial and mining area, and part (d) reveals traces of earthwork in the region. These aforementioned examples demonstrate that our proposed method is capable of effectively detecting changes in vector polygons with support from high-resolution RS images.

4. Analysis and Discussion

4.1. Analysis of the DP Algorithm Parameters Selections

The influence of the parameters p and λ on the performance of the proposed method is analyzed in this section. To facilitate this analysis, a simple accuracy evaluation index called denoising accuracy (DA) was devised in this article.

The DA is calculated as follows:

$$\mathrm{D}\mathrm{A}=\frac{N_{DP}}{N_{total}+N_{normal}}$$

(30)

where $N_{DP}$ is the selected correct noise samples, $N_{total}$ represents all noise samples in a category, and $N_{normal}$ is the selected erroneous noise samples. The range of DA is between 0 and 1, and the larger its value, the better the denoising performance of the proposed method. Experimental results obtained from the two datasets are presented in Figure 20, using cropland as an illustrative example. The values of p and λ are selected from the intervals 10∼30 and 0.03∼0.07, respectively. Based on the experimental results presented in Figure 20, it can be found that p is actually not related to the number of samples in the training set N. For instance, despite different sample sizes for the Nantong dataset and Guantan dataset, a fixed $P=20$ consistently achieves optimal denoising performance (see Figure 20). Moreover, $\lambda$ emerges as a crucial parameter influencing denoising performance. Taking Figure 20a as an example, DA exhibits significant variations in response to changes in parameter $\lambda$. It is noted that $\lambda =0.05$ and $P=20$ consistently yield relatively optimal denoising accuracies. Furthermore, the value of $\lambda$ is associated with the intensity of changes within the study area contextually considered at this time node (2021–2022), where China experienced a COVID-19 epidemic, which lead to restricted human activities; hence, there is no need for excessively large values of $\lambda$ under such circumstances. Therefore, given a new dataset, $\lambda =0.05$ and $P=20$ are suggested to be used as the default parameters in the proposed method.

4.2. Influence of Sample Set Denoising

In contrast to conventional sample denoising methods that heavily rely on manual intervention, the proposed EViTCC-DP approach significantly enhances automation. To evaluate the impact of mislabeled samples on model training, we compared the accuracy of the Two-Classifier Cross-Validation (TCCV) [35] and EViTCC-DP before and after denoising the training samples. As shown in

Table 3. Comparison of the classification accuracy of the Nantong dataset before and after sample denoising (no improvement observed after epoch 82).

Training Set					Epoch
		10	20	30	40	50	82
Initial	OA	0.8153	0.8459	0.8644	0.8798	0.8984	0.9068
Denoised by TCCV	OA	0.8342	0.8709	0.8804	0.8979	0.9079	0.9288
Denoised by EViTCC-DP	OA	0.8420	0.8727	0.8875	0.9052	0.9129	0.9348

and

Table 4. Comparison of the classification accuracy of the Guantan dataset before and after sample denoising (no improvement observed after epoch 84).

Training Set					Epoch
		10	20	30	40	60	84
Initial	OA	0.8063	0.8379	0.8595	0.8752	0.8906	0.9165
Denoised by TCCV	OA	0.8389	0.8711	0.8888	0.9010	0.9093	0.9333
Denoised by EViTCC-DP	OA	0.8441	0.8771	0.8953	0.9089	0.9197	0.9421

, the OA of the model improved by 2.80% and 2.56% on the Nantong dataset and Guantan dataset, respectively, after denoising with EViTCC-DP.

The comparison results indicate that EViTCC-DP achieves a higher accuracy with fewer iterations, highlighting its advantage in mitigating noise interference. Although EViTCC-DP only slightly outperforms TCCV in accuracy (by 0.6% for the Nantong dataset and 0.88% for the Guantan dataset), TCCV removes RTS, which is crucial for enhancing model performance. This emphasizes the superior ability of our method in identifying relevant instances within the dataset and in effectively removing noise.

In order to enhance the visualization of the denoising effect, t-SNE [60] analysis was employed to assess the discriminability of high-dimensional sample features before and after denoising, as depicted in Figure 21. Each dot represents a distinct class, with increased clustering among dots of the same class indicating reduced noise levels. As illustrated in Figure 21, samples exhibit a more compact distribution within each class after denoising, leading to improved homogeneity and facilitating model training.

4.3. Introducing Representative Training Samples

The crucial role of RTS has been widely acknowledged by researchers [61,62,63,64,65]. The proposed transformation procedure of RTS is illustrated in Figure 22. Firstly, the detected mislabeled samples were fed into the ViT model to extract depth features. Subsequently, using the k-means clustering algorithm, these noisy samples were supervised and transformed into RTS.

We utilized artificially generated reference data [see Figure 17b,d] to further visually assess the model’s detection accuracy.

Table 5. Comparison of detection accuracy results with and without the inclusion of representative training samples.

Training Set	%	Nantong Dataset	Guantan Dataset
Denoised (excluding RTS)	Precision	88.22	89.97
	Recall	91.26	92.13
Denoised (including RTS)	Precision	90.33	91.06
	Recall	91.41	92.38

presents the detection outcomes of the model on the two datasets. Excluding RTS, the model achieved precision and recall rates of 88.22% and 91.41%, respectively, in change detection for the Nantong dataset, as well as achieving precision and recall rates of 89.97% and 92.13%, respectively, for the Guantan dataset. Interestingly, including RTS improved precision by 2.11% and 1.09% on the Nantong dataset and Guantan dataset, respectively, while maintaining consistent recall due to its high-value information, which is conducive to change detection.

5. Conclusions

The integration of RS images and deep learning in vector polygon change detection presents a promising solution to streamline data collection and processing, thereby mitigating challenges associated with spatial data updating. Despite notable progress in leveraging vector data for RS image change detection driven by advancements in artificial intelligence, geospatial big data, and RS interpretation, persistent challenges remain. This paper introduces a novel change detection method for land cover vector polygons using high-resolution RS images and deep learning techniques. The method is tailored to update vector polygons using up-to-date RS images and deep learning, distinguishing itself from previous approaches by incorporating vector polygons as prior knowledge into the detection process, thus automating sample construction. The change detection process involves four phases: segmentation, denoising, classification, and detection. First, RS images are combined with land cover vector polygons to automatically generate an initial sample set using boundary constraints. Subsequently, the class-constrained DP clustering algorithm denoises the initial set while converting filtered samples into RTS using the k-mean algorithm to construct a high-quality multi-scale sample set. Finally, the improved AMCT-Net model classifies this sample set, which is followed by detecting changed vector polygons by combining change rules. The effectiveness of our proposed method was validated and analyzed using real data from two typical regions in Jiangsu Province.

The main conclusions of this paper are as follows:

• The boundary constraint segmentation method utilized in this study accurately segments the boundaries of ground objects, while the adaptive cropping strategy facilitates comprehensive sampling within vector polygons, minimizing confusion among ground objects in the generated samples. The proposed sample denoising method, EViTCC-DP, significantly enhances model accuracy, leading to a 2.80% and 2.56% improvement in OA on the Nantong and Guantan datasets, respectively.
• To enhance classification performance, we introduced multi-scale modules and attention mechanisms to construct a novel model, AMCT-Net. This network combines the advantages of CNNs and Transformers, enabling the extraction of more discriminative features. Experimental results on the two datasets demonstrate the effectiveness of the proposed method, with the accuracy of the AMCT-Net model reaching 91.34% and 93.51%, respectively, surpassing that of other advanced models.
• Visual interpretation results demonstrate the significance of RTS in enhancing detection accuracy. The introduction of RTS yields a 2.11% and 1.09% increase in change detection accuracy for the Nantong and Guantan datasets, respectively. Our approach enables the swift construction of a high-quality multi-scale scene sample set incorporating RTS, requiring minimal manual intervention. Furthermore, in conjunction with designed change decision rules featuring adjustable parameters and improved applicability, the change detection method outlined in this paper effectively identifies changed vector polygons, offering clear advantages over traditional manual vector polygons updating methods.

Our future research will focus on incorporating prompt learning into our innovative change detection method to develop a comprehensive methodology for detecting unauthorized land encroachment, evaluating spatial database quality, and monitoring urban development. Moreover, we will carry out an additional investigation that will explore the incorporation of sample transfer learning into our proposed method to meet the increased demands for high-resolution RS image change detection across various sensors, time periods, and resolutions.