Advances in Deep Learning Recognition of Landslides Based on Remote Sensing Images

Abstract

Against the backdrop of global warming and increased rainfall, the hazards and potential risks of landslides are increasing. The rapid generation of a landslide inventory is of great significance for landslide disaster prevention and reduction. Deep learning has been widely applied in landslide identification due to its advantages in terms of its deeper model structure, high efficiency, and high accuracy. This article first provides an overview of deep learning technology and its basic principles, as well as the current status of landslide remote sensing databases. Then, classic landslide deep learning recognition models such as AlexNet, ResNet, YOLO, Mask R-CNN, U-Net, Transformer, EfficientNet, DeeplabV3+ and PSPNet were introduced, and the advantages and limitations of each model were extensively analyzed. Finally, the current constraints of deep learning in landslide identification were summarized, and the development direction of deep learning in landslide identification was analyzed. The purpose of this article is to promote the in-depth development of landslide identification research in order to provide academic references for the prevention and mitigation of landslide disasters and post-disaster rescue work. The research results indicate that deep learning methods have the characteristics of high efficiency and accuracy in automatic landslide recognition, and more attention should be paid to the development of emerging deep learning models in landslide recognition in the future.

1. Introduction

The natural environment is being increasingly harmed by economic development and human activity. Moreover, landslide disasters are happening more frequently, endangering people’s lives, property, and socioeconomic status [1]. According to the statistics taken from NASA Landslide Viewer, the global distribution location map of landslide sites from 1915 to 2021 is shown in Figure 1 [2]. The majority of landslides occur in highlands, mountains, and other areas with more challenging terrains, and their concentration is highly consistent with the Earth’s plate-connecting activity zones [3]. Currently, we are facing an increasing demand for detailed and accurate landslide maps and inventories from around the world [4,5,6]. Especially in areas prone to landslides or earthquake disasters, it is of great significance to quickly and accurately obtain an inventory of landslides over a large area for disaster prevention and relief [7,8,9,10,11,12,13,14]. It is also important to promptly determine the location of landslides and their area of influence and to take targeted action to minimize the damage caused by landslide disasters [15].

The most traditional and basic landslide detection methods are field detection [16] and remote sensing visual interpretation [17], which rely on the professional knowledge of researchers to detect landslide areas from remote sensing images by interpreting various image features such as shape, color tone, texture, and layout through inference analysis. Although this method is accurate, it requires a lot of time and labor investment and is not suitable for the rapid identification of large-scale landslides [18]. The change detection method quantitatively analyzes the process of landslide occurrence and determines the distribution range of landslides based on the threshold changes of image spectra before and after landslides, breaking the limitations of traditional methods and improving the accuracy of detection. It is more effective for recently occurring landslides but is easily affected by background terrain interference [19]. Image segmentation and classification techniques can fully utilize image spectral and texture information to quickly identify landslide objects [20].

With the development of computer science, image detection, and other technologies, computers have shown great advantages in processing remote sensing images [21]. The deep learning model has a deep network structure, strong generalization ability, as well as strong robustness to identify landslides quickly and accurately, which enables the model to extract higher-level remote sensing image information and, ultimately, differentiate landslides from non-landslide areas [22,23]. For the successful identification of landslides, deep learning applied to system quantitative simulations of the process of visual interpretation by researchers can be used to complete landslide detection work. It plays an increasingly important role in disaster prevention and mitigation applications. After reviewing the currently available literature on landslide recognition in optical remote sensing images, we found that several classic models such as ResNet [24,25], YOLO [26,27,28,29,30], Mask R-CNN [31,32,33,34], U-Net [28,35,36,37], DeeplabV3+ [38,39,40], Transformer [41,42,43], and EfficientNet [44,45] and several open landslide datasets such as Bijie landslide dataset [24], HR-GLDD dataset [46], CAS Landslide Dataset [47], and so on, have been popularly used for landslide recognition. In this paper, we will first introduce the fundamentals of landslide recognition based on deep learning and then discuss and analyze the current development status of each type of model; finally, we will compare the advantages and disadvantages of each model and analyze the development trends of landslide identification.

2. Fundamentals of Deep Learning to Recognize Landslides

Deep learning is a very powerful machine learning technique. It is an expansion of artificial neural networks (ANNs) but has a deeper, more complicated structure than ANNs [48] and has a wide range of applications in many fields. One of the most successful applications of deep learning methods, like convolutional neural networks (CNNs), is landslide recognition. The process of automatic landslide recognition using deep learning involves several steps: data preprocessing, creating sample sets, training models, and identifying landslides, as illustrated in Figure 2.

The task of landslide recognition consists of target detection and semantic segmentation. The output result of target detection is landslide border information, which can only represent where the landslide is located within the border but cannot provide landslide boundary information. Semantic segmentation is first used to identify the target detection frame to locate the position where the landslide occurred and then to segment the landslide boundary in the local area. In addition, landslide deep learning recognition is related to the quantity and quality of samples; therefore, multiple research teams have established their landslide remote sensing databases [24,46,47].

2.1. Targeted Detection

As early as 1997, Shikada [49] proposed applying remote sensing data to landslide disaster detection. The earliest landslide detection of remotely sensed images was performed via manual visual interpretation [50]. With the continuous progress of computer technology, it has gradually replaced some of the manual operations, such as in the case of automatic landslide disaster detection of multi-temporal remote sensing images through the use of a landslide interpretation key [51], the extraction of landslide areas in remote sensing images via human–computer interaction interpretations, etc. [52]. However, the regional division of landslide detection still requires the participation of researchers, partly because the deep network model is not yet mature and partly because the complexity and variety of landslide-generating environments make it difficult for computers to perform detection tasks independently.

While earlier landslide detection methods were mainly based on human-designed feature extractors and classifiers, most of them are now turning to CNNs for end-to-end training and prediction. Region-based convolutional neural networks (R-CNNs) are a neural network architecture used for target detection that can be used to generate several candidate regions in the input image and the use a CNN to extract and classify the features of each candidate region [53]. R-CNNs select the candidate frames of the region of interest of the landslide image as the input samples, then compose a vector of the landslide samples from the candidate regions and the calibrated regions of the image, and then classify the landslide feature vectors via SVMs to locate the landslide region [54]. The performance of R-CNNs has greatly improved, but there are limitations in the speed of calculating candidate frames, and repeated calculations lead to low efficiency. Faster R-CNNs use the candidate areas of RPN landslide images for judgment and then determine the landslide type, which avoids the problem of low accuracy caused by the extraction of too many candidate areas [55]. R-CNNs and their variants (e.g., Fast R-CNN and Faster R-CNN) have achieved better results in the field of landslide detection. Many scholars in China have conducted related research in this field, and Li et al. [56] combined the BP network model with different bootstrap methods to predict landslide displacement intervals. Guo et al. [57] used migration learning for the automatic interpretation of landslides from high-resolution images. Other countries started landslide susceptibility assessments earlier. For example, Tien Bui et al. [58] compared the effectiveness of SVMs, artificial neural networks, kernel logistic regression, and logistic model trees in spatial prediction models of landslide disasters. Furthermore, with a large number of scholars emerging in this field, China has become the country with the most published papers on landslide identification in the past twenty years [59] and the leader in deep learning recognition of landslides [43]. The amount of research on CNNs in the field of target detection has been increasing year by year, and the most widely used is the Faster R-CNN network model (Figure 3). Jian et al. [53] used Faster R-CNN to automatically identify landslides in Fugong County, with an accuracy of 91.19%; however, it is difficult to successfully identify landslides with insufficiently distinctive features. To make up for this shortcoming, it can be combined with InSAR and LiDAR technology.

These network models applied in the field of target detection make computer target recognition of images more intelligent and achieve better results than traditional target detection using image processing techniques alone. Therefore, the use of deep network models for landslide target detection using remote sensing images is the current mainstream trend. However, the complexity and variability of landslides, whether in terms of causes, morphology, or the environment in which they are located, will be a great challenge for deep network models in this field, along with insufficiently labeled landslide data, limited model generalization ability, the missed detection of many model parameters, and low detection accuracy and detection speed. Detecting landslide disaster areas quickly and accurately is a significant and challenging task.

2.2. Semantic Segmentation

Semantic segmentation is a technique for classifying images at the pixel level by labeling each pixel point to distinguish between different classes of objects but not between different objects within the same class. Representative semantic segmentation networks include fully convolutional networks (FCNs) [60], SegNet [61], pyramid scene parsing networks [62], and DeepLab [63] series. The semantic segmentation of landslides using high-resolution remote sensing images is called feature classification in the field of remote sensing, which refers to judging each pixel in an image by the category to which it belongs. Traditional remote sensing image classification is mainly based on the comprehensive consideration of the spectral features, shape, and texture of remote sensing images through a manual visual interpretation method, which requires the interpreter to have a wealth of experience in image interpretation and is subject to the influence of subjective factors; the cost is very high, and the efficiency is low [64]. With the continuous development of computer technology, people have begun to use computer-assisted manual classification and identification, which greatly reduces manual workloads and improves work efficiency [65]. Traditional pixel-based methods are prone to the Pretzel phenomenon, and object-oriented classification methods have difficulties in terms of their selection of parameters, such as segmentation scale, and are not suitable for handling large-volume data. Traditional machine learning algorithms are generally only applicable to small sample sets, with stringent requirements for the processing of data features and limited generalizability of the model. In the final analysis, these methods were assigned to specific categories to design specific features, and then the data were used for target recognition. The workload of human design features is large, and the designed features are relatively simple and cannot express the data well; on the other hand, the current remote sensing data are large in volume and high in complexity, so a more accurate and efficient semantic segmentation method for high-resolution remote sensing images is needed.

In recent years, deep learning (DL) [66] has performed excellently in the field of image recognition. Compared with traditional machine learning methods, where human-designed features are required, a deep learning model with a multi-layer structure, or perhaps a single-layer structure of expressive ability with many such structures organically in combination, can have stronger expressive ability. Introducing deep learning into the identification of landslides in high-resolution remote sensing images is important for intelligent landslide identification. Cheng et al. [26] designed the YOLO-SA landslide detection model based on YOLOV4, which was combined with Gconv, Gbneck, and an attention mechanism based on high-resolution remote sensing imagery; however, the target detection method could not accurately identify the landslide boundary information, which was not conducive to post-disaster rescue work. Ullo et al. [67] used Mask R-CNN to realize landslide segmentation using remote sensing images to obtain landslide boundary information, but the boundary obtained was more ambiguous when the deeper ResNet was used as the backbone network. In order to obtain clearer landslide boundaries, Bragagnolo et al. [6] utilized the coding and decoding capabilities of U-Net to effectively restore the boundary information, but due to the differences in the feature scales of remote sensing images, the identification of small landslides was significantly less effective than that of large landslide areas. Based on Bragagnolo’s study, Mu et al. [68] designed a multi-channel image segmentation algorithm based on feature maps, which can adjust the size of the sensing field according to the pixel content in order to obtain the semantic feature information of landslides at different scales. Yi et al. [69] designed a LandsNet model for landslide hazard recognition using fused residual blocks, attention modules, and multiscale fusion operations and achieved high recognition accuracy.

Current semantic segmentation models of landslide disasters have problems with the fuzzy identification of landslide boundary regions and the differentiation of classification accuracy of multiscale semantic information when using remote sensing images; therefore, we can try to apply more landslide parameter information to the semantic segmentation model. Meanwhile, in follow-up research, not only can we continue to expand the dataset and improve the accuracy of dataset labeling, we can also add the data to the model coding layer to further improve the recognition accuracy of the model and provide support for early warning and rescue operations in the case of landslide disasters.

2.3. Landslide Remote Sensing Databases

The establishment of a sample library is the foundation of deep learning landslide recognition, and a sufficient number of high-quality samples have made landslide recognition more than half successful. At present, three main open landslide remote sensing databases can be freely downloaded, as shown below:

(1)

Bijie Landslide Dataset

The Bijie Landslide Dataset is an open remote sensing landslide dataset used in the development of automatic landslide detection methods, which was created by Ji et al. [24] of Wuhan University in 2019. This dataset consists of satellite optical images, shapefiles of landslide boundaries (Figure 4), and digital elevation models. All the images with a panchromatic resolution of 0.8 m and a multispectral resolution of 3.2 m in this dataset, i.e., 770 landslide images and 2003 non-landslide images, were cropped from the TripleSat satellite images captured from May to August 2018. Landslide image files, shape files, and DEM files are all in a PNG format, while the boundary coordinate files are in a Txt format (

Table 1. The Bijie Landslide Dataset [24] (*-file name).

File Type	Number of Landslides	Number of Non Landslides
Landslide image (*.png)	770	2003
Landslide shape file (*.png)	770
DEM (*.png)	770	2003
Boundary coordinate file	770

). The size of each sample is different, and for each instance, a 40 m extension is preserved as the background.

(2)

HR-GLDD dataset

The High-Resolution Global Landslide Detector Database (HR-GLDD) is an open high-resolution (HR) satellite dataset with PlanetScope and a 3 m pixel resolution for landslide mapping. It is composed of landslide instances from 10 different global physiographical regions in East Asia, South and South East Asia, South America, and Central America. Five rainfall-triggered and five earthquake-triggered multiple-landslide events that occurred in varying geomorphological and topographical regions in the form of standardized image patches containing four PlanetScope image bands (red, green, blue, and NIR) and a binary mask are contained in this dataset for landslide detection [46].

(3)

CAS Landslide Dataset

The CAS Landslide Dataset is an open hybrid database created by Xu et al. of the Chinese Academy of Sciences (CAS). It comprises 20,865 images, integrating satellite and unmanned aerial vehicle data from nine regions for landslide identification (

Table 2. Detailed information on the CAS Landslide Dataset [47].

Subdataset	Amount	Acquisition Time	Sensor	Ground Resolution (m)
Palu	817	January 2021–November 2021	WorldView2/3	5
Lombok	436	May 2019–December 2019	WorldView2/3	5
Hokkaido Iburi-Tobu	1484	September 2018–October 2018	SAT	3
Tiburon Peninsula (Sentinel)	606	March 2020–Jnue 2020	Sentinel-2/L2A	5
Tiburon Peninsula (Planet)	325	September 2021–December 2021	Planet	4
Mengdong	1155	November 4th, 2018	SuperView-1	0.5
Moxitaidi (SAT)	652	September 2022–October 2022	Sentinel-2/L2A	0.6
Moxitaidi (UAV-0.6 m)	984	September 2022–October 2022	UAV	0.6
Moxitaidi (UAV-1 m)	483	September 2022–October 2022	UAV	1
Moxi town (0.2 m)	1635	September 2022–October 2022	0.2	0.2
Moxi town (1 m)	160	September 2022–October 2022	UAV	1
Longxi River (SAT)	1769	March 2015–December 2015	GF-1	0.5
Longxi River (UAV)	2504	March 2011–May 2011	UAV	0.5
Jiuzhai valley (0.2 m)	5925	Augest 2017–September 2017	UAV	0.2
Jiuzhai valley (0.5 m)	1752	Augest 2017–September 2017	UAV	0.5
Wenchuan	178	November 2008–December 2008	Landsat	5

). All of the sample images in this database are 512 × 512 size TIFF format, and the label files contain interpretations of landslides corresponding to each image in the same format [47].

3. Typical Deep Learning Models for Landslide Recognition

Deep learning technology can effectively identify landslide areas [70], mainly because it is implemented in excellent models, such as convolutional neural networks, recurrent neural networks, U-Net, Mask R-CNN, Yolo, etc. The general process of landslide detection methods based on a deep neural network is shown in Figure 5 [71]. The main deep learning models for landslide identification are introduced in chronological order of their first proposal, as follows.

3.1. Recurrent Neural Networks (RNN)

A recurrent neural network (RNN), as proposed by Jordan in 1986 [72], is an artificial neural network (ANN) with recurrent connections that can model sequential data for sequence recognition and prediction [73]. A simple RNN has three layers: input, recurrent hidden, and output layers (Figure 6a). The hidden layer has multiple hidden units that are connected to each other through time with recurrent connections (Figure 6b) [74]. The core idea of a RNN is to give the network a memory function by introducing an image interior. When processing sequence data, a RNN will update the internal state based on the current input and the previous moment’s state and produce the output. A RNN model can further improve the accuracy of landslide displacement prediction [75,76]. However, RNNs have two main shortcomings: errors often accumulate during the prediction process, and the position of attention is not always accurate. Cui et al. proposed the SG-BiTLSTM model based on traditional LSTM to solve the problems of cumulative errors in the prediction process and low localization accuracy. In the process of recognizing landslides, the accuracy of the improved LSTM model is much higher than the accuracy of landslide recognition [77].

In summary, the basic principles of deep learning in the field of landslide recognition include convolutional neural networks, recurrent neural networks, and so on. These principles provide a theoretical basis for the successful application of deep learning in the field of landslide identification, and in practical application, researchers need to establish appropriate deep learning models according to specific identification scenarios and optimize and improve the characteristics of the models. With the continuous deepening of deep learning research, more innovative neural network models have emerged, promoting the development of landslide recognition technology.

3.2. Convolutional Neural Network (CNN)

In 1998, LeCun proposed the LeNet-5 model, marking the initial formation of convolutional neural networks (CNNs) [78]. CNNs are feedforward neural networks that are mainly used in image recognition tasks. The core idea of CNNs is to capture the local features of an image through local sensory fields, weight sharing, and pooling operations. CNNs usually consist of multiple convolutional layers, activation function layers, pooling layers, and fully connected layers (Figure 7) [10,24]. The convolutional layer is responsible for extracting the features of the image, the activation function layer introduces nonlinearity, the pooling layer reduces the spatial dimension, and the fully connected layer implements the classification or regression task. Compared to shallow machine learning networks, convolutional neural networks have a deeper structure, contain more layers of hidden nodes, and can better utilize large sample data for learning [71]. Therefore, convolutional neural networks are widely used for automatic landslide recognition using remote sensing images.

Shi et al. [11] use a convolutional neural network (CNN) to realize the rapid detection of landslides in large areas of Lingfeng Mountain and Lantau Island in Hong Kong, China. Zhao et al. [23] used images of landslides in Jiuzhaigou captured by a drone to construct a model combining object-oriented influence and deep convolutional neural networks to automatically extract the depth features of landslides and recognize landslides. The highest accuracy of this improved model for landslide recognition reached 87.68%. Yang et al. [51] used a convolutional neural network model to recognize loess landslides, which added a new channel fusion layer after sample data input, and the improved model landslide recognition accuracy reached 95.7%, ensuring the high efficiency of recognizing landslides across large areas. In 2019, Ghorbanzadeh et al. [5] compared the machine learning methods of ANN, SVM, and RF (pixel-based) with different CNN-based patch-wise classifications for landslide detection in the Rasuwa district in Nepal and found that CNNs did not automatically outperform ANN, SVM, and RF. The performance of CNNs strongly depended on their design, i.e., their layer depth, input window sizes, and training strategies [77].

A CNN consists of a deep structure that can extract more and deeper information about landslides through model training in the process of automatic landslide identification, making the identification results more accurate. CNNs have proven to have excellent feature extraction capabilities and can effectively reduce time consumption [79]. However, owing to the complexity of the model structure, it not only increases the difficulty of identification but also increases the time cost dramatically, and CNN models are affected by the number of samples, leading to a reduction in efficiency.

3.3. AlexNet

AlexNet is a deep learning model first proposed by Krizhevsky et al. in 2012 [80]. The advent of AlexNet-30 made CNNs the mainstream models for landslide identification [81]. The overall structure of the network model consists of five convolutional layers, three pooling layers, and a fully connected layer [82]. Xia et al. [83] used the Wenchuan earthquake landslide area as the study area and used deep learning methods such as seven-layer CNN, AlexNet, ResNet4V152, DenseNet2, InceptionV201, Xception, and Inception ResNetV3 to detect landslides.

When faced with a small number of landslide samples, directly utilizing a deep-learning model will result in overfitting during the training process. The application of a nonlinear activation function enables the AlexNet model to have a faster convergence speed. After fine-tuning, the model can effectively avoid the occurrence of overfitting in the application process and achieve high accuracy in landslide identification [79,81]. Therefore, the relevant parameters introduced into AlexNet must be adjusted, particularly in the fully connected layer.

3.4. Mask R-CNN

Girshick et al. [84] proposed the earliest region-based convolutional neural network (R-CNN) model in 2014, to which many researchers have made various improvements. While some versions of R-CNN, such as Fast R-CNN [85] and Faster R-CNN [53], only have the function of bounding box localization, Mask R-CNN adds a branch to achieve the pixel-level segmentation of objects to predict the target’s mask [14]. In addition, Mask R-CNN replaces the ROI pooling operation with a new method called ROI alignment, which avoids the problem of incorrect position identification caused by ROI pooling [86].

Mask R-CNN is a more typical semantic segmentation model, and the framework is shown in Figure 8. The algorithm uses a convolutional neural network as the backbone model to extract the convolutional features of the image and construct the feature pyramid. In Mask R-CNN, ResNet and FPN are used as backbone networks to process the input image and generate the feature pyramid for further feature extraction. Candidate regions of different sizes and scales are generated on the feature pyramid by RPN to classify these potential targets and generate bounding boxes. Finally, ROI-aligned feature mapping provides two branches: one for target classification and bounding box regression and the other for segmentation mask prediction to improve the performance of target detection and segmentation.

Jiang et al. [34] enhanced the data for difficult samples and input them into the Mask R-CNN network for landslide fine detection segmentation. The average accuracy reached 90.3%, which successfully verified the feasibility of the model. Yang et al. [87] addressed the problem of erroneous extraction due to confusing features, added landslide triggering factors as auxiliary information in the input image data to achieve the purpose of background enhancement, and verified the feasibility of the method using the Mask R-CNN model: the accuracy was 88.68%. Wu et al. [88] combined the Mask R-CNN model with the deciphered markers of landslides in remote sensing imagery for landslide identification, and the results showed that the overall accuracy of the method for landslide identification was close to 90%.

To reduce modeling effort, more attention should be paid to improving the adaptability and flexibility of landslide extraction models in future research.

3.5. U-Net

U-Net is an improvement of the FCN model proposed by Ronneberger et al. [89] and has an encoder–decoder architecture similar to the letter “U” [36], as shown in Figure 9. The network structure of U-Net consists of compression and expansion paths corresponding to the encoder and decoder, respectively. The compression path is used to obtain contextual information and perform feature extraction, while the expansion path is used to accurately locate the feature positions and connect the feature maps by branching to achieve the fusion of surface and deep semantic information and reduce the loss of edge information [90]. The branching connection of the U-Net fully incorporates the features corresponding to the surface layer during the decoding process so that the small samples of the data are not easy to overfit.

Liu et al. [90] successfully identified large landslides in the Jiuzhaigou area using a U-Net network model with topographic factors. Meena et al. [36] used U-Net to identify landslides, achieving slightly better results than other machine learning methods. Despite the fact that the U-Net models depends on the architecture of the U-Net model and the complexity of the geographic elements in the images, the use of U-Net model is still in its early days in the field of landslide recognition. Dong et al. [91] proposed an improved model for landslide recognition, L-Unet, which has a greater perception of landslide features than the traditional U-Net model, expanding the sensory field and enhancing the model’s ability to extract multiscale landslide information, effectively improving its accuracy in terms of landslide recognition.

The terrain in landslide areas is often undulating, and errors are bound to occur in the process of landslide identification. Consideration can be given to making full use of digital elevation models (DEMs) to monitor terrain information to further improve the accuracy of the models.

3.6. ResNet

Deep image features are often obtained by increasing the network depth; however, a network depth that is too large is highly susceptible to gradient vanishing or gradient explosion. To solve this problem, He et al. [92] proposed a deep residual network (residual network, or ResNet), which uses two residual modules, identity and bottleneck, which have constant input and output dimensions and are commonly used in networks with fewer layers, such as ResNet18 and ResNet34, where the number of layers is 18 and 34, respectively. As a residual network, ResNet is a powerful framework for training deep neural networks and has a deeper learning level, better learning efficiency, and faster convergence speed than traditional CNN networks [22,93]. The emergence of ResNet not only solves the problem of the side effects caused by increasing network depth but also accurately detects the boundaries of landslides [18,22].

Ullo et al. [67] compared the performance and effectiveness of Mask R-CNN with the backbone model ResNet-50/101 in the case of landslide recognition, and the results showed that ResNet-101 was superior to ResNet-50. Mao et al. [40] utilized Xception on its backbone network based on DeepLabV+3, MobileNetV2, ResNet18, and ResNet50, which were optimized to identify landslides in Bijie City and the earthquake region in Wenchuan, and the experimental results showed that the four indexes of ResNet18 and ResNet50 were higher than those of other backbone networks. Zhang et al. [94] utilized the ResNet network to screen remote sensing images and select candidate images with landslide areas. The remote sensing images containing landslide areas were then input into a multiscale neural network to semantically segment the landslide areas, thereby accurately locating the landslide position and improving the accuracy of landslide detection. Hacıefendioğlu et al. [95] used the pre-trained ResNet50 model for the automatic identification of landslides, and the success rate was above 90%.

In the case of changing only the backbone residual units, different backbone networks determine the landslide recognition results, keeping the size of the convolution kernel constant. The deeper the depth, the better the model performance. It is also necessary to reduce the number of network parameters to further improve the recognition accuracy of the model.

3.7. PSPNet

The pyramid scene parsing network (PSPNet) is a semantic segmentation model jointly proposed by the Chinese University of Hong Kong and Shangtang Technology. It won the 2016 ImageNet Challenge championship [96]. PSPNet uses the pyramid pooling technique and a regional aggregation method with the aim of learning global context information. With this technique, PSPNet can fuse features under four different pyramid scales for multiscale feature learning [97]. PSPNet added a PSP module between the encoder and the decoder to improve the FCN. First, a CNN is used to obtain the last convolutional feature map from an input image. Then, a pyramid parsing module is applied to collect different subregion representations. The final feature representation is formed by upsampling and concatenation layers. Finally, the representation is fed into convolutional layers to obtain the final per-pixel predictions (Figure 10). PSPNet (ResNet50) obtained the best landslide recognition effect, with a mIoU value of 91.18% and a precision index of 93.76% [98].

3.8. YOLO

You Only Look Once (YOLO) is an object detection system based on a single neural network. After Redmon et al. proposed YOLOv1 in 2016 [99], with the continuous introduction of new versions such as YOLOv2 [100], YOLOv3 [101], YOLOv4 [102], YOLOv5 [103], YOLOvX [104], YOLOv6 [105], YOLOv7 [106], YOLOv8 [107], the detection accuracy and speed have been continuously improved. In 2024, the latest version, YOLOv9, was proposed [26].

YOLO is a real-time end-to-end target detection model (Figure 11). In YOLO, target detection is treated as a regression problem in which the position and category of the pre-checked boxes are directly predicted in the output layer. The YOLO model consists of DarkNet, which extracts backbone features, and a feature pyramid network (FPN), which combines multiscale features to detect small landslides. This method can fully utilize precise location information in low-level features and rich semantic information in high-level features simultaneously. Due to their excellent remote sensing big data processing capabilities and high accuracy, they have been widely used for landslide identification [26,27,108,109,110,111]. Specifically, after improving the model by utilizing the SE compression attention mechanism and VariFoucil loss function, YOLO can more accurately locate the position of landslides, detect the range of landslides more accurately, and have fewer missed detections [110].

The YOLO network is characterized by its ability to quickly obtain the output categories and corresponding localization through a network model based on a CNN [113]. Liu et al. [114] proposed the SSD network, which combines the method of direct regression of candidate frames and classification probabilities in YOLO with the anchor mechanism in Faster R-CNN to improve the recognition accuracy while maintaining a high recognition speed. In a study conducted by Cheng [115], a proposed YOLO-SA model based on the YOLOv4 model was used for the target detection of landslides, and the results showed that the improved YOLO-SA model has better detection accuracy, speed, and a small number of parameters. Replacing the PANet structure of YOLOV5 with the BiFPN structure enhances the detection ability when using images with large size differences. Replacing the CIoU loss function with the GIoU loss function improves the regression loss and detection accuracy of the target box [116]. Guo et al. [112] analyzed and compared SBAS-InSAR, the YOLO model, and its combination with the SBAS-InSAR model for landslide identification in an alpine county. After analysis and comparison, the combination of the two methods is better than a single model.

YOLO algorithm training relies on labeled landslide datasets, which are more suitable when identifying small samples of landslides; however, the YOLO model has a limited number of detected landslides per square and can only detect a single landslide when a square contains more than one landslide, leading to low accuracy when using YOLO for the detection of smaller landslides.

3.9. Transformer

Due to the limited sensory field of CNNs, they cannot make good use of global information. To solve this limitation, the transformer model (Figure 12) proposed by Vaswani et al. in 2017 [117] plays a non-negligible role in obtaining global information, which is based on the self-attention mechanism and has a great advantage in terms of global modeling. In this research, Dosovitskiy et al. [118] proposed an improved transformer model ViT based on the self-attention mechanism, which is the same as the transformer in terms of structural framework, and the proposed model has been at the leading edge in the field of image recognition. Carion et al. [119] proposed DETR (an end-to-end target detection structure) based on transformer whose performance is comparable to that of Faster-RCNN. Esser et al. [120] constructed the VQGAN model, which applies a transformer in combination with a CNN and can obtain more comprehensive information when recognizing landslides.

Tang et al. [122] used three different neural network modules: the convolutional attention neural network, which fuses multimodal remote sensing data; the transformer neural network, which automatically extracts features from hill shade maps; and the transformer neural network, which automatically extracts features from DEM data. Experiments were conducted to compare the ResU-Net, LandsNet, HRNet, and SeaFormer models. The findings indicated that the suggested model achieved the maximum accuracy in terms of landslide identification. In Yang et al.’s study [123], the transformer was fused with ResU-Net, and the model was used to recognize landslides in two different geological backgrounds, and the reasonableness of the model was verified, indicating that ResU-Net embedded with the transformer can be used as a relatively novel method for landslide recognition.

Future research should focus on reducing the training weight dependence on small sample data in order to fully understand the mechanism and strength of the transformer when fused with CNN models. This will enable more skillful and efficient fusing of the transformer with each CNN model to recognize landslides.

3.10. DeeplabV3+

DeepLabv3+ is an improved DeepLab series model proposed by the Google team in 2018 [98]. It uses DeepLabv3 as the encoder, introduces atrous convolution for downsampling and uses a spatial pyramid pooling module to extract multiscale information (Figure 13). After optimizing the DeeplabV3+backbone network using Xception, MobileNetV2, and ResNet, the model can accurately identify and segment landslides in satellite images of the 5.12 Wenchuan earthquake [40]. By fusing low-level and high-level features, it improves the accuracy of landslide recognition [124]. The Deeplabv3+network model optimized by SENet (squeeze-and-extraction networks) achieves more accurate segmentation of landslide edge details, with fewer errors and omissions in terms of recognition [125]. The Deeplabv3+network model optimized by MobileNetV3 can distinguish interference factors such as bare land and roads and obtain more accurate landslide boundaries to effectively identify landslides [39].

3.11. EfficientNet

The deep learning models mentioned above generally adjust the number of convolutional channels in relation to the image to realize the expansion of the convolutional neural network when improving network accuracy; however, the accuracy gain will be saturated rapidly after reaching 80%, and no substantial improvement in accuracy can be realized [126]. Meanwhile, when the number of convolution channels in the model structure is adjusted to a certain degree, the recognition rate will decrease, and the parameters will increase. To solve the above problems, the EfficientNet algorithm was proposed by Tan and Le in 2019 [127]. The underlying network architecture of EfficientNet was obtained via the neural network architecture search technique.

Hu et al. [128] improved the architecture of the TransUNet model and proposed an improved landslide identification model, the FATransUNet model, the core of which was replacing the transformer module with the efficient transformer module, which can effectively reduce the computational complexity of the model. Experiments have shown that the accuracy of the FATransUNet model is higher than that of the other five comparative models. Li et al. [45] used the EfficientNet model for model training and the testing of landslide data. At the same time, three other convolutional neural network models with large structural differences were selected for comparison with the EfficientNet model, and the accuracy of the other models was lower than that of the EfficientNet model, indicating that the EfficientNet model can be very good for automatic identification of most landslides in the study area.

There are currently few instances of the EfficientNet model being used for landslide identification, as it is still in its primary stage. However, it is clear from the examination of the available study data that the original EfficientNet model was highly accurate in identifying landslides, and going forward, research can concentrate on further developing and exploring the EfficientNet model.

4. Shortcomings and Prospects

4.1. Shortcomings

Based on a summary of the literature reviewed in this paper, it can be concluded that while deep learning has made significant strides in the automatic identification of landslides, its identification process is still vulnerable to various factors like topography, vegetation cover, and other factors due to the limitations of the model and the intricate topographical environments in which landslides occur. The following is a discussion of the shortcomings and restrictions of automatic landslide identification processes:

(1)

A large amount of training data is required. Deep learning methods need to be driven by huge amounts of data to realize feature learning. It is very difficult to construct a huge dataset of slippery slope samples. The amount of landslide data has an important impact on the final classification results. For fully convolutional networks, a large amount of landslide sample data is required when training the model [129]. When the amount of landslide sample data is insufficient, the model will be overfitted. In order to avoid overfitting, researchers have chosen many remote sensing images under different conditions (cropping, flipping, scaling, etc.) to solve this problem, but the collected data still have some limitations, such as large sizes and small amounts, both in the prescribed scenarios as well as the scarcity of historical landslide sites themselves [94]. However, in this study, the determination of insufficient sample size is related to the selected study area, and the appropriate sample size was selected based on the actual scenario. Zhang et al. [71] selected 770 landslide images as training samples from the Bijie City landslide dataset, and due to the insufficient number of these samples, data enhancement was used to expand them to 3280 landslide training sample images. After model training was completed, two areas in Beijing’s Yanqing and Fangshan districts were used as the test areas, and high accuracy was achieved in the test. However, in order to further improve the accuracy of the landslide identification, the U-Net model still needs to be improved in a subsequent study, and the number of training sets will be increased to optimize the model. Taking the Loess Plateau as the main study area, Shi et al. [130] selected 2870 landslides from the landslide dataset as the training sample set and used a single-level instance segmentation network (YOLACT) to recognize the loess landslides. Due to the small number of samples, the model training was difficult to optimize, and it was necessary to further expand the sample size in future work. In order to solve the problem of insufficient landslide sample data in model training, the direction and attitude of landslide distribution can be adjusted via the circular view function to obtain samples of the same landslide in different directions and different altitudes to expand and enhance the sample set; moreover, part of the remote sensing image data of the non-landslide in the training sample dataset can be randomly added in order to enhance the generalizability of the training model.

(2)

Concerning the influence of the terrain, in practice, most landslides are distributed in mountains; due to the effects of this terrain, it is difficult to obtain samples to solve the problem. Furthermore, the irregular distribution of landslide displacement monitoring points for the extraction of landslide information increases these difficulties [131]. Many landslides cannot be identified using remotely sensed images due to immature imaging technology and the influence of complex terrain features. Not only that, due to terrain and weather conditions, it is usually difficult to obtain real-time images of pre- and post-disaster landslides within a short period before the occurrence of a landslide, which can lead to a large time difference between pre- and post-disaster remotely sensed images and affect the accuracy of landslide detection [132]. To eliminate the influence of terrain factors on landslide identification as much as possible, Cai et al. designed a lightweight volumetric neural network with simple structural features. After simplifying the volumetric neural network to avoid performance degradation in terms of the model, the improved data allowed for the construction of a volumetric neural network model that can extract enough landslide information [133]. In a study conducted by Ju et al. [134] on recognizing landslides based on Mask R-CNN, due to the variability in terms of landslide morphology and the complexity of the natural environment, there were more misidentifications existing in the work, and finer landslide classification and identification is needed, which can be achieved via its combination with InSAR and LiDAR techniques to further improve the accuracy of landslide hazard identification.

(3)

Concerning vegetation cover, landslide disasters mostly occur in mountainous areas, and most mountainous areas are rich in vegetation. During the automatic identification of landslides, vegetation cover will be observed in the collected remote sensing landslide images. High vegetation density will lead to high costs and lower efficiency in terms of landslide identification and even result in omission and misidentification; therefore, vegetation removal operations should be added to data preprocessing, which can improve the accuracy of landslide identification. At present, the most effective and widely used method for vegetation removal is the point cloud filtering algorithm, which obtains the point cloud data of the landslide identification area through the three-dimensional laser scanner and then filters the obtained point cloud data to remove the ground vegetation. Considering that the topography of densely vegetated areas has diverse forms and causes irregularities in the collected point cloud data, Tao Ma’s research group [135] proposed a point cloud filtering algorithm based on windowing and slope, and the results of this algorithm showed that it can efficiently and accurately remove a large number of vegetation noise points on the ground and significantly improve the efficiency and accuracy of the point cloud filtering algorithm in densely vegetated areas. In future research, the comprehensive use of multisource remote sensing data such as synthetic aperture radar (SAR) imagery and optical imagery will be further investigated to establish a more accurate landslide identification model. At the same time, the interpretability of the deep learning model will also be investigated to reveal the bases for deep learning landslide identification models.

(4)

The model structure is too complex. Although deep learning can extract deeper features of landslides and improve recognition accuracy, the parameters become more and more numerous as the deep learning network model becomes more complex and deeper. Too many model parameters mean that more memory and time are consumed, and the results cannot be obtained quickly in emergencies, affecting the application of deep learning models for fast detection. Therefore, in the future, we need to explore more lightweight network models to realize the fast and accurate recognition of landslides. Fu et al. [136] designed the YOLOv4-MobileNetv3 landslide detection model by optimizing the structure on the basis of the YOLOv4 model, and the recognition accuracy of the improved YOLOv4 model reached 91.37%. Moreover, it improved the detection speed by 6.19 f/s (5.24%), and reduced the model parameter size by 80%. However, reducing the model parameters to improve the target detection speed while maintaining high accuracy is an urgent problem that needs to be solved.

Overall, the application of deep learning techniques to landslide recognition can realize the automatic detection of large landslides and better meet the requirements of disaster emergency response. However, there are still many challenges and problems, and future research will continue to explore new theories, methods, and techniques to address these challenges and promote the development of deep learning models for efficient landslide recognition. Detecting landslides using deep learning models requires the use of a large amount of landslide data for model training; however, in practical applications, landslides are mostly located in remote areas where landslide samples are difficult to obtain, and it is often difficult to meet the data volume requirements for model training. On the other hand, in the face of features such as bare ground, dry land, and some special artificial buildings, which are similar to landslide image features, effectively avoiding the misidentification of these easily confused features has always been a thorny problem in landslide recognition. In addition, the model parameters in deep learning model structures number millions or even billions, resulting in complex network structures, greatly reducing the efficiency of landslide recognition. Thus, it is necessary to continuously improve and simplify these models while ensuring the accuracy of landslide recognition. In addition, we can also try to combine deep learning models with InSAR technology for comprehensive landslide identification. The above problems are urgent issues relating to deep learning model landslide recognition, and the accuracy of the recognition and prevention of landslide disasters must be ensured.

4.2. Prospects

(1)

As the models used for landslide identification become more comprehensive and complex, the accuracy of identification is also increasing. The integration of multiple single models is the most prominent development trend in the use of deep learning models to identify landslides. Integrating a transformer into ResU-Net can enhance the network’s ability to model the global context of feature maps and generate an accurate regional landslide inventory and facilitate emergency rescue operations [123]. Hu et al. [137] replaced the transformer module in the original structure with the efficient transformer module, which effectively reduced the computational complexity of the model. Moreover, the flow alignment module (FAM) was introduced, which can simplify the operation process and effectively integrate the high-resolution information in the shallow layer. The accuracy of the FATransUNet model is higher than the other single models (FCN, U-Net, SegNet, DeepLabV3+, and TransUNet).

(2)

Multisource data with multiple ground resolutions and spectral resolutions are being used in deep learning landslide identification. Specifically, the use of high-resolution drone imagery has brought new vitality to this field [138,139]. Hyperspectral data have also begun to be used for deep learning landslide recognition [140].

(3)

The feature segmentation of remote sensing images is also an important development direction for improving the accuracy of landslide identification [141,142]. An approach to fuse both local and non-local features can outperform state-of-the-art general-purpose semantic segmentation approaches [8]. A feature-based constraint deep U-Net (FCDU-Net) method to detect rainfall-induced mountainous landslides can achieve better landslide detection results than the other semantic segmentation methods [143]. A feature-fusion-based semantic segmentation network (FFS-Net) can extract texture and shape features from 2-D HRSIs, and terrain features taken from DEM data can greatly improve the segmentation accuracy of old, visually blurred landslides [141].

(4)

The combination of deep learning and InSAR for early landslide prediction or reactivation identification of ancient landslides can achieve good prediction accuracy [141,144,145,146,147,148,149,150,151].

5. Conclusions

Landslide disasters are a frequent occurrence in complex and diverse geological formations, making landslide identification a fundamental yet challenging task that requires long-term investment in theoretical and practical research. Compared to machine learning (ANN, SVM, RF, etc.), deep learning has shown outstanding robustness and enormous potential in landslide recognition [126] and can better improve landslide recognition performance when sufficient training samples are used [5]. (1) In regions with particularly complex and unique geological conditions, traditional landslide identification methods are often hindered by environmental factors, leading to reduced efficiency and certain limitations. Deep learning models offer a promising solution to this problem by reducing the influence of subjective human factors and providing a deeper network structure. (2) However, overly complex model structures can lead to reduced efficiency. The lightweight design of the YOLO optimization model improves the redundancy caused by the complex structure to a great extent, but the structure of the model still needs to be optimized and adjusted to different degrees according to different real scenarios. (3) In the realm of landslide identification, there is a pressing need to address the issue of inadequate sample data. (4) InSAR can be combined with deep learning to improve recognition accuracy and efficiency for active landslides. (5) The introduction of LiDAR data can, to some extent, overcome the impact of vegetation on landslide identification.

6. Patents

The authors have submitted a patent application for the “AI identification and annotation method and system for geological disaster samples based on expert scoring system” to the Chinese National Invention Patent Office: patent application number 2022113200852. The patent is currently undergoing substantive examination.