Estimation of Landslide and Mudslide Susceptibility with Multi-Modal Remote Sensing Data and Semantics: The Case of Yunnan Mountain Area

Abstract

Landslide and mudslide susceptibility predictions play a crucial role in environmental monitoring, ecological protection, settlement planning, etc. Currently, multi-modal remote sensing data have been used for precise landslide and mudslide disaster prediction with spatial details, spectral information, or terrain attributes. However, features regarding landslide and mudslide susceptibility are often hidden in multi-modal remote sensing images, beyond the features extracted and learnt by deep learning approaches. This paper reports our efforts to conduct landslide and mudslide susceptibility prediction with multi-modal remote sensing data involving digital elevation models, optical remote sensing, and an SAR dataset. Moreover, based on the results generated by multi-modal remote sensing data, we further conducted landslide and mudslide susceptibility prediction with semantic knowledge. Through the comparisons with the ground truth datasets created by field investigation, experimental results have proved that remote sensing data can only enhance deep learning techniques to detect the landslide and mudslide, rather than the landslide and mudslide susceptibility. Knowledge regarding the potential clues about landslide and mudslide, which would be critical for estimating landslide and mudslide susceptibility, have not been comprehensively investigated yet.

1. Introduction

Landslide and mudslide susceptibility predictions play a crucial role in environmental monitoring, ecological protection, settlement planning, etc. [1,2]. Previous researchers have reported that such prediction could be enhanced by abundant data sources and computational tools from remote sensing techniques and geospatial information [3,4,5]. Thus, a number of approaches have been developed, including optical remote sensing image-based approaches [6,7], multispectral remote sensing-based approaches [7], digital elevation models (DEM)-based approaches [8], and GIS-auxiliary analyzed approaches [4,5].

However, single-modal remote sensing data might not be sufficient for accurate landslide and mudslide susceptibility prediction [7,9]. Multispectral remote sensing data, while providing ample waveband information to assess potential land changes, often have a medium spatial resolution (20–50 m), limiting the representation of spatial details. In addition, optical remote sensing image can capture land surface information but cannot directly depict landform distributions. DEM and SAR datasets can, respectively, represent the distribution and changes of landform, but they have limitations in describing the visual representations of land surface. Thus, the information derived from multi-modal remote sensing is essential for investigating landslide and mudslide detection and susceptibility prediction.

Since multi-modal remote sensing data might be complex in content, heterogenous structures and hyperdimensional features, as well as a variety of deep learning-integrated methodologies have been proposed to conduct landslide and mudslide disaster prediction [9,10,11,12,13]. However, the implementation of deep learning techniques raises concerns about explainable feature learning [14,15]. The feature learning of deep learning, such as CNNs and vision transformer (ViT), are often regarded as “black boxes” that lack human interpretability. Meanwhile, features regarding landslide and mudslide susceptibility are often hidden in multi-modal remote sensing images, making it challenging for deep learning models to extract them from labeled datasets. Consequently, developing explainable features becomes crucial for predicting landslide and mudslide susceptibility [16].

Considering that the clues about landslide and mudslide susceptibility might be invisible on remote sensing data, we developed a geospatial domain ontology to create a semantic-enhanced approach. Semantic modeling focuses on organizing and explicitly describing these features, allowing for the discovery of inherent meanings of landslide and mudslide that are not directly derivable from remote sensing images. Thus, this paper reports our efforts to develop an integrated framework for landslide and mudslide susceptibility prediction that combines optical remote sensing, DEM, a SAR dataset, and geospatial semantics.

The remainder of this manuscript is organized as follows. Section 2 describes the study area and dataset. Section 3 presents the details of each part of the methodological framework. Section 4 illustrates the experimental results and discussion. Section 5 summarizes the highlights of our efforts and outlines future prospects.

2. Study Area and Dataset

2.1. Study Area

Figure 1 illustrates the study area that the datasets cover, which includes Yunlong County, Yongping County, Longyang County, and Jianchuang County in Yunnan Province, China. The spatial coverage of these four counties has potentially suffered from the influence of landslides and mudslides. The area measures 14,247.81 km².

2.1.1. Longyang County

Longyang County is located in the western part of Yunnan Province, under the jurisdiction of Yunnan’s Baoshan City. Its geographic coordinates are between east longitude 98°42′ to 99°32′ and north latitude 24°45′ to 25°41′. It is situated within the Nujiang Mountain Range and the Gaoligong Mountain Range, between the Lancang River and the Nujiang River. The terrain is characterized by mountain ranges, with a maximum elevation of 3655.9 m (Daoren Mountain) and a minimum elevation of 648 m (Nujiang). There are numerous rivers and abundant water resources, with large drop in altitude. Landslides and mudslides are the most severe geological disasters in Longyang County. In 2015, a total of 373 geological disasters of various sizes had occurred in the district, resulting in a direct economic loss of CNY 54.201 million. Landslides caused 55.3% of the losses, while mudslides caused 44.3%.

2.1.2. Jianchuang County

Jianchuang County is located in the northwest of Yunnan Province and the north of Dali Bai Autonomous Prefecture. It is under the jurisdiction of Dali and situated in the middle section of the northwest Yunnan Hengduan Mountains, and spans from 26°12′ to 26°42′ north latitude and from 99°33′ to 100°33′ east longitude. The county is 58 km² wide from east to west and 55 km long from north to south, with a total area of 2250 km². The mountainous area accounts for over 90% of the county’s area, while the basin area accounts for 7%, and the rest comprises lakes and rivers. Landslides and mudslides are the main geological disasters in Jianchuang County. In 2015, a total of 74 small-scale landslides occurred, which destroyed 102 houses, caused 4 deaths, and resulted in a direct economic loss of CNY 723,200.

2.1.3. Yongpin County

Yongpin County is located on the east bank of the Lancang River, in the western part of Dali Bai Autonomous Prefecture, known as the “western gateway” of Dali. It is situated between the Bonan Mountain and the Yuntai Mountain, two branches of the Yunnan–Guizhou Plateau, with higher terrain in the northwest and lower in the southeast. Its geographical coordinates are 99°17′–99°56′ east longitude and 25°03′–25°45′ north latitude. Its maximum horizontal span from east to west is 64.5 km, and the maximum vertical span from north to south is 77.0 km. Yongpin County has complex geological structures and significant terrain differences, including over 100 rivers, of which 31 are perennial rivers with a length of over 5 km. In 2015, landslide was the most severe geological disaster in Yongpin County, causing a direct economic loss of CNY 36.30 million, and accounting for 65.4% of the entire losses of geological disasters. Mudslides caused a direct economic loss of CNY 17.55 million, accounting for 31.6% of the entire losses of geological disasters.

2.1.4. Yunlong County

Yunlong County is in the south of the Hengduan Mountains and the Lancang River valley, in the northwest of Dali Bai Autonomous Prefecture. Its geographical coordinates are 98°52′~99°46′ east longitude and 25°28′~26°23′ north latitude. The county has a maximum horizontal span of 91.8 km from east to west and a maximum vertical span of 109.0 km from north to south. Yunlong County is a mountainous area with a fragmented and complex landscape of high mountains and canyons, with the mountainous area accounting for 90% of its total area. The highest elevation is 3663 m, and the lowest elevation is 730 m. In 2015, landslide was the most severe geological disaster in Yunlong County, followed by mudslide. Landslides caused 80 deaths, resulting in a direct economic loss of CNY 61.92 million, and accounting for 61.5% of the entire losses of geological disasters. Mudslides caused a direct economic loss of CNY 38.09 million, accounting for 37.9% of the entire losses of geological disasters.

2.2. Dataset

The study dataset includes four types of modal remote sensing data: DEM, optical remote sensing image, the SAR dataset, and selected samples from each dataset.

(1)

DEM. We used DEM data to conduct terrain analysis and extraction of geological structures. The spatial resolution of DEM is 12.5 m, which is derived from an ALOS sensor.

(2)

Optical remote sensing image. Optical remote sensing images were utilized for detecting human activities, recognizing land cover, and capturing landscape characteristics. The optical remote sensing images used in this study are Gaofen-6 satellite images, with a resolution of 0.8 m. The cloud and snow coverage in the remote sensing data did not exceed 5%.

The optical remote sensing images were captured between 9 November 2020 and 22 March 2022, when the broadleaf forests had shed their leaves, leaving only coniferous forests; relatively sparse vegetation and a few crops were in the image, allowing for a clear reflection of the fine details of geological hazards.

(3)

SAR dataset. SAR dataset data has strong penetration, reducing atmospheric disturbance, and making it useful for analyzing the terrain and topographical features in cloudy weather, abundant rainfall, and vegetation coverage. We used L-band ALOS-2 (Advanced Land Observing Satellite-2) satellite data, with a spatial resolution of 1 m. For the critical area, we conducted hazard monitoring using TerraSAR-X data with a spatial resolution of better than 10 m. These datasets were accessed from 1 January 2019 to 31 December 2020, and contained no less than six issues of data each year.

2.3. Labeled Dataset

The labeled dataset included two categories, landslide and mudslide. All samples were generated by optical remote sensing images that were of reliable quality and free of cloud covering. The selected samples are shown in Figure 2.

Table 1. Statistics for the landslide and mudslide recognition.

County	Landslide Sample Number	Mudslide Sample Number	Total Sample Number
Longyang	255	22	277
Yongpin	167	63	230
Yunlong	225	78	303
Jianchuang	55	14	69
All	702	177	879

shows the statistics for the labeled dataset. The number of geological hazard samples produced in Longyang County was 277, while 230 geological hazard samples were produced in Yongping County, 303 geological hazard samples were produced in Yunlong County, and 69 geological hazard samples were produced in Jianchuang County.

3. Methodology

3.1. Optical Remote Sensing and DEM-Based Interpretation

3.1.1. Data Processing and Enhancement

The workflow of remote sensing data processing includes image preprocessing, image equalization, orthorectification, image fusion, image mosaic, and image enhancement [17].

First, we fused the panchromatic and multispectral images into optical remote sensing images using PANSHARP. For the optical remote sensing images, we conducted image preprocessing involving geometric transformation, image normalization, and image smoothing. Then, we performed image equalization to maintain color consistency among different remote sensing images. Finally, we mosaiced the adjacent optical remote sensing images with the same resolution, to avoid cracks, misalignments, blurs, distortions, and fringes.

3.1.2. Landslide/Mudslide Feature Recognition with an Integrated Deep Learning Approach

We employed a large-scale model of CNN-enhanced semantic segmentation [18] to detect landslide and mudslide morphology including back edge, front edge, wall, boundary, terrace, bulge, depression, tongue, and fissure, based on visual features and auxiliary features including terrain, hydrology, and vegetation. More details regarding landslide and mudslide morphology detection can be found in Reference [11].

3.1.3. Terrain Interpretation

We employed the Geomorphons [19] and spatial-contextual approach [20] to extract the ridgelines and valleylines from DEM. Then, we performed orthorectification to correct the results generated from DEM and optical remote sensing image, based on specified spatial and elevation control points. The fusion of ridgeline and valleyline extraction and landslide/mudslide feature recognition were used to map the watershed, based on hydrological rules.

3.2. SAR-Based Interpretation

3.2.1. SAR Processing and DEM Matching

To process the original SAR dataset, a master image was used to form a sequence of interferometric pairs, which included selecting stable target points with high backscattering over a significant period to analyze and eliminate atmospheric phase, terrain errors, and other noise factors, ensuring accurate deformation information.

In this study, we used the Permanent Scatters InSAR (PS-InSAR) approach [21], SBAS-InSAR (Small Baseline Subset InSAR) approach [22], and Interferometric Point Target Analysis (IPTA) approach [23], respectively, to geocode the deformation and generate three deformation maps.

Moreover, for significant nonlinear deformation processes, the residual phase needs to be filtered in the spatial and temporal domains. Thus, the singular value decomposition method was used to solve the nonlinear deformation sequence and calculate the deformation sequence of each coherent target, completing the estimation of deformation parameters.

3.2.2. InSAR Interpretation

We utilized ascending and descending SAR data to conduct deformation monitoring using SBAS-InSAR, PS-InSAR, and IPTA-InSAR approaches, thereby obtaining deformation amounts and rates for each period. Subsequently, we performed time series analysis to generate a series of interferometric pairs based on different master images by combining image pairs within a certain range of temporal and spatial baselines. For each pair of interferometric results, we selected stable points as ground control points to refine orbit attributes, remove the orbit phase, and filter the atmospheric phase. This enabled us to obtain deformation rate and cumulative deformation results.

By fusing the results of optical remote sensing feature recognition and the regional deformation rates and cumulative deformation values, we detected the areas of concentrated surface deformation based on geological structures, visual features, and terrain slope and aspect. Additionally, we extracted parameters including the area, maximum deformation amount, minimum deformation amount, and average deformation rate for each deformation area.

The process of SAR feature recognition is represented in Figure 3.

3.3. Semantics-Enhanced Interpretation with Visual and Terrain Features

For landslide and mudslide susceptibility areas, many features are not visually recognizable. Thus, besides the conventional approaches that identify possible landslide/mudslide areas based on DEM, optical remote sensing images and a SAR dataset, we developed an ontology to organize the information regarding landslide/mudslide susceptibility into geospatial semantics, and employed the semantics to further identify landslide and mudslide susceptibility areas based on the result. The geospatial information for landslides and mudslides is listed in

Table 2. Geospatial information for landslides and mudslides.

Landslide
Category	Individual	Description
Composition	Soil landslide	Landslides occurring in loose layers such as alluvial, flood, colluvial, rock fall, and residual soils.
	Rock landslide	Landslides occurring in rock layers.
Time	New landslide	Historically recorded landslides, or landslides with well-preserved.
	Old landslide	Non-recorded landslides, or landslides with no traces.
Size	Small	<1 × 104 m3
	Medium	1 × 104 m3–10 × 104 m3
	Big	10 × 104 m3–100 × 104 m3
	Grand	100 × 104 m3–1000 × 104 m3
Mudslide
Category	Individual	Description
Position	Hill mudslide	Canyon terrain.
	Piedmont mudslide	Wide valley terrain.
Morphology	Valley-shaped basin	The basin has a fan shape or elongated shape.
	Ridge-shaped basin	The basin has a bucket shape, with no obvious drainage area.
Stage	Growth stage	The slope is fragmented and unstable, and small scale.
	Peak stage	The gully is extremely unstable, and large scale.
	Decaying stage	The gully tends to be stable, with erosion and deposition on the river bed.
	Dead stage	The gully and channel are stable, with the restoration of vegetation coverage.
Size	Small	<1000 m3
	Medium	1000 m3–10,000 m3
	Big	10,000 m3–100,000 m3
	Grand	>100,000 m3

.

Then, we converted the information listed in Table 2 into various triples based on the W3C Semantics Standards [24]. The triples are expressed as follows, subject-property-object, where subject and object denotes two independent individuals, and property denotes the relationship between these two individuals. For sample, landslide-hasComposition-soil landslide.

Table 3. Features used for visual and terrain feature interpretation.

Category	Features	Description
Deformation	Size	Spatial area
	Shape	Ratio of length and width
	Phenomena	Creep, collapse, cracks, etc.
Visual	Landform	Gap, mineral crater, etc.
Terrain	Geomorphology	Slope, aspect, curvature, etc.
	Structure	Fault, cliff, etc.
	Vegetation	Normalized difference vegetation index

lists the features utilized for deformation interpretation from SAR, visual interpretation from optical remote sensing, and terrain interpretation from DEM. We also converted these features into various triples.

Based on the triples generated from Table 2 and Table 3, we developed a domain ontology according to the method used by Protégé [25] to organize these triples into the semantics of landslide and mudslide morphology. The semantics of landslide and mudslide morphology include back edge, wall, boundary, terrace, body, and fissure. More details of this landslide and mudslide morphology are shown in Figure 4.

Features of the back edge of landslide/mudslide (a): abnormal color and texture that can be observed due to the original natural landscape being destroyed. These features typically exhibit an abnormal boundary between a darker and a lighter color, with the darker color representing the original vegetation coverage and the lighter color indicating exposed soil or rock. Moreover, the back edge of the landslide generally has an abnormal shape, including straight, arc-shaped, or curved lines.

Features of the landslide/mudslide wall (b): The landslide wall refers to the wall-like boundary surface exposed between the landslide body and the original hill. As landslide walls primarily consist of fresh mud or bedrock, they tend to exhibit a light color, underdeveloped shadow textures, and abnormal features of various shapes.

Features of the landslide/mudslide boundary and terrace (c,d): The landslide boundary and terrace refer to the line between the landslide body and the surrounding rock and soil. The boundary or terrace with the gully generally shows a dark-colored, abnormal feature, which is reflected as a straight, broken or arc-shaped line due to the vegetation coverage. The slope-shaped boundary or terrace generally shows an abnormal feature of the boundary line between two distinct colors.

Features of the landslide/mudslide body (e): The landslide body refers to the entire sliding part of the landslide. It displays a funnel-shaped abnormality, with dark green blocks denoting densely vegetated areas, light green blocks indicating sparsely vegetated areas, and light-colored blocks representing soil or bedrock areas.

Features of the landslide/mudslide fissure (f): The arrows in two sides of f denotes the positions of two fissures. A landslide fissure refers to the cracks generated during landslide activity on the sliding body and its edges. Since landslide fissures always form a gully, the humidity in the gully is relatively high, and the vegetation is relatively developed. The fissures are mostly reflected as straight or arc-shaped dark-colored abnormal features on the image.

Based on the above descriptions of features, we organized every sentence into triples, and organized the triples into the semantics for conducting manual interpretation by logic reasoning and query with SPARQL techniques.

4. Experimental Results

The experimental framework included three groups. The first group followed the workflow detailed in Section 3.1, conducting a semantic segmentation based on a state-of-the-art CNN to extract landslide and mudslide susceptibility, and performing geomorphological analysis to extract ridgelines and valleylines. The second group followed the workflow detailed in Section 3.2 to perform ascending and descending InSAR deformation analysis, to detect the deformation anomaly areas and fuse the results generated from the first group. Based on the results generated by the first and the second group, the third group validated the landslide and mudslide susceptibility with interpretation enhanced by the semantics developed in Section 3.3.

4.1. Results

The positions of landslide and mudslide susceptibility areas in Longyang County, Jianchuang County, Yongpin County, and Yunlong County are shown in Figure 5, respectively. In the four figures, green, red and yellow points, respectively, denotes the landslide susceptibility points, mudslide susceptibility points, and landslide and mudslide susceptibility points.

After verification and correction by field investigation, Longyang County included 255 landslide susceptibility points and 22 mudslide susceptibility points, along with 23 landslide and mudslide susceptibility points. Jianchuang County included 167 landslide susceptibility points and 63 mudslide susceptibility points, along with 14 landslide and mudslide susceptibility points. Yongpin County included 225 landslide susceptibility points and 78 mudslide susceptibility points, along with 63 landslide and mudslide susceptibility points. Yunlong County included 55 landslide susceptibility points and 14 mudslide susceptibility points, along with 79 landslide and mudslide susceptibility points.

Table 4. Statistics for landslide and mudslide susceptibility estimation.

County	Landslide Number	Mudslide Number	Total Number
Longyang	255	22	277
Yongpin	167	63	230
Yunlong	225	78	303
Jianchuang	55	14	69
All	702	177	879
County	Total Number	Recognition Number (Group 1)	Precision	Recognition Number (Group 2)	Precision	Recognition Number (Group 3)	Precision
Longyang	277	66	0.2383	122	0.4404	220	0.7942
Yongpin	230	71	0.3087	106	0.4609	194	0.8434
Yunlong	303	86	0.2838	142	0.4686	131	0.9010
Jianchuang	69	19	0.2754	36	0.5217	26	0.8986
All	879	242	0.2753	406	0.4619	749	0.8521

lists the statistics of the landslide and mudslide susceptibility estimation by the three groups. Recognition numbers and the corresponding precisions of group 1 were generated from DEM and optical remote sensing images. Recognition numbers and the corresponding precisions of group 1 were generated from optical remote sensing images and SAR data. Recognition numbers and the corresponding precisions of group 3 were generated by geospatial semantic interpretation based on the results of group 2.

From the results listed in Table 4, although optical remote sensing images have been reported as a main data source for landslide and mudslide recognition, the results generated from these were rather poor. This mean that the visual features derived by the-state-of-the-art deep learning approaches from optical remote sensing images were irrelevant to the features of landslide and mudslide susceptibility. The clues for landslide and mudslide susceptibility might always be impossible to recognize visually.

Similarly, although the SAR dataset was a critical data source to represent the deformation of the land surface, the deformation information it was challenging to determine the features for landslide and mudslide, let alone landslide and mudslide susceptibility. This proved that the inner mechanism of the landform that induced a landslide or a mudslide would be irrelevant to deformation. In addition, deformation and land surface could not support the estimation of landslide and mudslide susceptibility.

The interpretation enhanced by semantics outperformed the results generated by DEM, optical remote sensing image, and the SAR dataset. The results also proved that the state-of-the-art deep learning techniques for visual features still had the ability to achieve satisfactory results. The key to implementing AI techniques into landslide and mudslide susceptibility estimation might be the approaches of incorporating knowledge into the statistical learning.

4.2. Discussion

The results generated by optical remote sensing images and the SAR dataset proved that state-of-the-art deep learning techniques might not be able to deal with landslide and mudslide susceptibility estimation, for three reasons. First, features regarding landslide and mudslide susceptibility were always hidden in optical remote sensing images and SAR data. This mean that visual features could not be relevant to the clues for landslide and mudslide susceptibility. Moreover, deformation information might not be a direct factor for prediction, and visual features and deformation information need a semantic-based reasoning to predict the susceptibilities. Last, the improvement obtained by semantic interpretation proved that the mechanisms of landslide and mudslide susceptibility are still beyond the capability of remote sensing, including spectral information, deformation information etc. DEM, optical remote sensing images and the SAR dataset could precisely detect the landslide and mudslide sites; however, the features derived from these datasets were still useless for estimating landslide and mudslide susceptibility.

Figure 6 illustrates the selected recognition results in the study area. Several conclusions about recognition are listed below.

Figure 6a shows the results for the detection of landslide and mudslide areas by visual features from an optical remote sensing image. In these three samples, the scenes of landslide and mudslide were visually recognized on remote sensing images. This proved that state-of-the-art deep learning models for object detection could effectively identify the areas where landslides and mudslides occur.

Figure 6b shows the results for the detection of landslide and mudslide areas by visual features from an optical remote sensing image and the deformation of the SAR dataset. In these areas, some visual features predicting landslide and mudslide were identified. The extract deformation involving the ascending and descending directions could affirm the identification. Moreover, the deformations were generally useful for estimating the periodic stage of a landslide or mudslide.

Figure 6c shows the results for detection of landslide and mudslide susceptibility by deformation from SAR but no optical remote sensing image. In these three remote sensing images, no apparent visual features regarding landslide and mudslide could be identified. Meanwhile, obvious deformations were observed in the ascending and descending orbits from these SAR datasets. The spatial distribution of maximal deformation in the ascending or the descending orbit direction might demonstrate potential for a landslide or a mudslide.

Figure 6d shows that detection of landslide and mudslide susceptibility was impossible by optical remote sensing image and SAR datasets. As mentioned above, visual features were impractical for predicting landslide and mudslide susceptibility. The visual features for these areas were also not useful to derive any clues about the potential for landslide or mudslide. Moreover, although the deformation in the ascending and descending orbits were apparent in the SAR datasets, the spatial distribution of this information was affected by landform change, including humans’ activities, making it challenging to determine the potential for landslide and mudslide. In practice, we identified these areas by a variety of factors, including erosions (e.g., developed gully erosion, fragmented terrains), landform type (e.g., karst block mountain), vegetation coverage, hill slope, and lithological conditions (e.g., karstified limestone, dolomite). Apart from terrain slope and vegetation coverage, these factors might be inaccessible from optical remote sensing images and SAR datasets.

Several conclusions are listed below, based on the identification of landslide and mudslide susceptibility.

• There is a strong correlation between susceptibility to landslides/mudslides and the type of landform. Landform type is unobtainable from remote sensing data including DEM, optical remote sensing images, and SAR.
• The gradient of a slope is a crucial factor that influences terrain stability and determines the susceptibility of landslides/mudslides. Landslides predominantly occur on slopes with gradients ranging from 15 to 35 degrees, while mudslides are more common on slopes within the same gradient range.
• Lithology and the structure of rocks and soil play a fundamental role in the development of landslides/mudslides. When comparing slopes under similar conditions, harder rock and soil formations exhibit greater resistance to deformation and improved terrain stability, whereas softer formations are associated with poorer stability.
• The geological structure has a significant impact on the occurrence of landslides/mudslides. On the one hand, intense tectonic movements can disrupt the integrity of rock formations, creating favorable conditions for the development of landslides and mudslides. On the other hand, new tectonic activity, such as earthquakes and seismic events, often increases the likelihood of landslides and mudslides.
• Human activities, particularly engineering activities, have a significant influence on the development of landslides and mudslides. Currently, human-induced modifications to the environment are the primary contributing factor to the occurrence of landslides and mudslides.

5. Conclusions

Landslide and mudslide are major geological disasters that always cause a large-scale socio-economic loss. DEM, optical image, and SAR integrated remote sensing techniques have been utilized to effectively detect landslide and mudslide. However, the spectral, spatial and deformation information derived from remote sensing datasets are irrelevant to the clues about landslide and mudslide susceptibility. In this paper, we have attempted to develop a framework involving remote sensing, DEM, and geospatial semantic interpretation for landslide and mudslide prediction. Although this framework incorporates state-of-the-art deep learning techniques for object detection and deformation recognition, these deep learning techniques cannot extract and learn the features regarding landslide and mudslide susceptibility from remote sensing datasets.

This paper reports our efforts to conduct a comprehensive investigation on landslide and mudslide susceptibility estimation in four counties and districts. We designed three research groups for landslide and mudslide susceptibility with multi-modal remote sensing. The first group focused on using DEM and optical remote sensing images, the second group focused on using DEM, optical remote sensing images, and a SAR dataset. The last group focused on interpretation with semantics, based on the results generated by the previous two groups.

In this manuscript, we developed the ontology to conduct the semantic reasoning, based on the results generated by multi-modal remote sensing data. The experimental results showed that the semantics were significant to landslide and mudslide susceptibility. A majority of landslide and mudslide susceptibility cannot be automatically detected by integrated remote sensing techniques. A remote sensing dataset might be able to identify the features that are relevant to a landslide or mudslide having occurred, or a landslide or mudslide possibly occurring. How to integrate the semantic-enhanced deep learning framework (such as a graph neural network, and knowledge graph-based statistical learning) is worthy of future attention.