Research on a Non-Stationary Groundwater Level Prediction Model Based on VMD-iTransformer and Its Application in Sustainable Water Resource Management of Ecological Reserves

Abstract

The precise forecasting of groundwater levels significantly influences plant growth and the sustainable management of ecosystems. Nonetheless, the non-stationary characteristics of groundwater level data often hinder the current deep learning algorithms from precisely capturing variations in groundwater levels. We used Variational Mode Decomposition (VMD) and an enhanced Transformer model to address this issue. Our objective was to develop a deep learning model called VMD-iTransformer, which aims to forecast variations in the groundwater level. This research used nine groundwater level monitoring stations located in Hangjinqi Ecological Reserve in Kubuqi Desert, China, as case studies to forecast the groundwater level over four months. To enhance the predictive performance of VMD-iTransformer, we introduced a novel approach to model the fluctuations in groundwater levels in the Kubuqi Desert region. This technique aims to achieve precise predictions of the non-stationary groundwater level conditions. Compared with the classic Transformer model, our deep learning model more effectively captured the non-stationarity of groundwater level variations and enhanced the prediction accuracy by 70% in the test set. The novelty of this deep learning model lies in its initial decomposition of multimodal signals using an adaptive approach, followed by the reconfiguration of the conventional Transformer model’s structure (via self-attention and inversion of a feed-forward neural network (FNN)) to effectively address the challenge of multivariate time prediction. Through the evaluation of the prediction results, we determined that the method had a mean absolute error (MAE) of 0.0251, a root mean square error (RMSE) of 0.0262, a mean absolute percentage error (MAPE) of 1.2811%, and a coefficient of determination (R²) of 0.9287. This study validated VMD and the iTransformer deep learning model, offering a novel modeling approach for precisely predicting fluctuations in groundwater levels in a non-stationary context, thereby aiding sustainable water resource management in ecological reserves. The VMD-iTransformer model enhances projections of the water level, facilitating the reasonable distribution of water resources and the long-term preservation of ecosystems, providing technical assistance for ecosystems’ vitality and sustainable regional development.

1. Introduction

Groundwater is a crucial natural resource in arid and semi-arid regions, serving as a vital source of replenishment for human survival and agricultural activities [1]. Nevertheless, the overuse of underground water reserves has emerged as a significant issue in recent times [2]. Groundwater levels vary in reaction to fluctuations in weather conditions, extraction operations, and agricultural cultivation [3]. The current situation in arid and semi-arid regions is characterized by water shortages, unpredictable rainfall, and over-exploitation of groundwater [4]. These factors pose a danger to groundwater resources, as they disturb the equilibrium of the water table [5]. This has resulted in significant environmental issues. One of the consequences of reducing groundwater supplies is the potential for salinization and pollution issues [6]. Utilizing deep learning models may greatly enhance efficiency in regional groundwater level monitoring by substantially decreasing labor and time expenses, while direct monitoring is challenging.

The fast development of artificial intelligence modeling in the age of big data has provided advanced technological tools for predicting water resources [7]. The primary research methodologies now used to address the non-stationarity issue in predictions of groundwater levels are regression analysis, trend analysis, and artificial intelligence [8]. The primary use of the regression analysis technique is to forecast patterns and cyclical fluctuations in data on the groundwater level [9]. Despite its ability to provide valuable outcomes with less data and its comprehensibility, the multivariate linear model is constrained by its inability to address the issue of non-linearity between the inputs and outputs of the model. Hodgson used the MLR linear regression model to forecast the variability of groundwater in aquifers [10]. This approach is favorable due to its straightforward data needs. However, when it comes to predicting the non-linear fluctuations of groundwater, its accuracy is unsatisfactory. Abhishek A. Pathak et al. used cluster analysis and the Mann–Kendall test to examine the pattern of groundwater levels in the watersheds of drought-prone regions in India [11]. The research’s findings indicated that the overuse of groundwater resources is the primary cause of the substantial decrease in the number of monitoring wells in the studied region [12]. Machine learning is extensively used in water resource prediction because of its ability to effectively detect intricate patterns and trends in data [13]. However, when it comes to data that are not stationary, machine learning may struggle to process complex data structures, rely heavily on the training data, and not effectively adapt to new data. In the early 21st century, several academics have effectively used machine learning to forecast groundwater levels in unconfined aquifers [14]. In 2001, Coulibaly et al. created IDNN and PNN models to mimic changes in groundwater levels in aquifers [15]. These models used monthly data on water table depth, precipitation, temperature, and river level as inputs for predicting deep groundwater levels. M. Sharafeldin et al. proposed the application of non-destructive geophysical techniques in groundwater level monitoring, providing effective methods for characterizing subsurface aquifer structures and determining groundwater levels [16]. Using integrated geophysical surveying techniques, M. Sharafeldin et al. conducted a groundwater level investigation near the Great Pyramid of Giza in Egypt, making significant contributions to the field of groundwater monitoring [17]. This present research has optimized groundwater monitoring methods and offers valuable insights for future water resource management.

The primary obstacle we face is the unpredictable variation in groundwater levels [18]. Groundwater levels in arid and semi-arid environments exhibit substantial fluctuations and are difficult to anticipate. Transformer models have shown exceptional performance in time series prediction and have emerged as a foundational model that adheres to the scaling rule [19]. They possess a strong capability to capture interdependencies between pairs of data points and extract hierarchical representations in the series. Nevertheless, the Transformer model often incorporates several variables with identical timestamps into indistinguishable channels and prioritizes attention to temporal identifiers to capture temporal interdependence [20]. Accurate forecasts become very challenging without a reset of the Transformer’s design. Liu et al. contended that the Transformer is not inherently useless for time series prediction, but instead it is being used incorrectly [21]. The architecture of the Transformer was reset by including each time series as a distinct token and emphasizing the analysis of multivariate correlation. In addition, layer normalization and feed-forward network modules were used to obtain more accurate sequence-global representations of time. The iTransformer served as a crucial component for time series forecasting. Their suggested iTransformer has demonstrated exceptional performance in global forecasting models, using inverted modules and architectural decisions, and effectively resolved the challenges faced by the Transformer model in forecasting. Nevertheless, the VMD technique, suggested by Dragomiretskiy et al., is capable of decomposing intricate non-stationary long-time series data with more efficiency to reduce noisy signals [22]. He et al. discovered that VMD may significantly enhance the performance of deep learning models in predicting runoff [23]. Sun et al. used VMD-coupled modeling to surpass other signal analysis techniques in accurately predicting the daily discharge of the Han River [24].

The objective of this project was to enhance the precision of groundwater level forecasting. To address the issue of non-stationarity, a significant obstacle in analyzing lengthy time series data, we wanted to integrate the Variational Mode Decomposition (VMD) with the iTransformer model. In this linked model, each variable’s whole time series is individually represented as tokens, hence increasing the local acceptance domain. The embedded tokens aggregate the global representation of the sequence via the inversion operation and are more effectively employed through the growing multivariate correlation attention mechanism. On the other hand, the feed-forward network excels in learning how to convert and interpret information from previous sequences to make predictions about future sequences. This allows it to accurately forecast diverse variables in long-term sequence data. The model was used for real-time monitoring at nine groundwater level monitoring sites in the Kubuqi Desert ecological reserve, which is a representative dry and semi-arid area in China (Figure 1). The study focused on two primary research inquiries. (1) Can the combination of the Variational Mode Decomposition (VMD) and the iTransformer model considerably enhance the performance of the Long Short-Term Memory (LSTM) model? (2) How can the non-stationarity issue of dynamic fluctuations in groundwater levels be addressed utilizing VMD combined with the iTransformer model? The findings demonstrate that the VMD-iTransformer coupled model can greatly enhance the accuracy of predictions and serve as a reliable and efficient tool for forecasting groundwater levels in the context of managing water resources in dynamic situations.

2. Materials and Methods

2.1. Study Area

The study area is situated within the central ecological reserve of the semi-arid and arid Kubuqi Desert region in northwest China. Specifically, it is located in Hangjin Banner, Ordos City, Inner Mongolia Autonomous Region, with the coordinates of 39°34′–40°87′ N 108°83′–109°68′ E. The area has an altitude ranging from 1068 to 1620 m, as shown in Figure 2. It covers approximately 1.8 × 10⁴ square kilometers and is home to a population of around 116,000 individuals [25]. The terrain of the region is also a notable characteristic. The topography of the territory is characterized by a higher elevation in the southern and eastern parts, while the northern and western parts have a lower elevation. The region is within a semi-agricultural and semi-pastoral zone. The research region exhibits a moderate continental climate characterized by an average annual temperature of 6.7 °C, an annual precipitation of 275 mm, a frost-free period lasting 122–144 days, and an average relative humidity of 48% [26]. The yearly duration of sunlight ranges from 3150 to 3200 h. The temperature variation among the four seasons is significant, and the climate is marked by aridity, high evaporation rates, an uneven distribution of precipitation, intense sunlight, and strong winds with sand particles.

The pH of the water in the research region is mostly found to be between 5.3 and 6.5, with an average of 6.1, indicating an overall acidic character. The formation water exhibits a wide range of mineralization, ranging from 10.7 to 105.0 g/L. The majority of the mineralization falls between the range of 20.0 to 80.0 g/L, with an average value of 42.5 g/L. The mineral content of this water is much greater than that of surface water, and most measurements even surpass the entire mineral content of saltwater (35 g/L). As a result, this water is classified as a brackish brine type due to its high mineralization. The formation waters have their genesis in freshwater from terrestrial depositional habitats. Through the ongoing deposition of sediments, the water within the sedimentary layers have experienced various transformations including evaporation, blending with surface water or other formation water, and interactions with rocks. These processes have caused a substantial rise in the mineral content of the formation water, resulting in its current high mineralization properties. In the Hangjinqi region, the mineral content in the formation water is much greater than that found in surface water and saltwater [27]. Therefore, if groundwater resources are excessively utilized or contaminated, it might result in an escalation of groundwater mineralization, thus impacting water quality and subsequently affecting human life and agricultural irrigation. Hence, there is a pressing want for a precise forecasting technique for the variation in groundwater levels to safeguard groundwater resources from overuse and contamination. This is crucial for maintaining ecological sustainability.

2.2. Data and Processing

The data utilized in the experiments of this study were obtained from the time series of groundwater levels in the ecological reserve of Hangjin Banner, Ordos City, Inner Mongolia. This area is located on the outskirts of the Kubuqi Desert and serves as a crucial location for assessing groundwater resources. The management of groundwater resources and water quality in China and globally is heavily reliant on these data. The time series groundwater level data in the ecological reserve were obtained from the Hangjin Banner Water Affairs Bureau. The data collection period spanned from July 2021 to September 2022, with measurements taken every 4 h. In total, 12 monitoring locations were included in the dataset. As a result of a sensor anomaly in the groundwater level monitoring system, there was a partial absence of raw data from some locations. We eliminated monitoring locations with over 10% missing data from the research and verified the final 9 monitoring sites.

2.3. Methodology

2.3.1. Variational Mode Decomposition

The VMD is a decomposition model that effectively addresses the issue of smoothness and non-smoothness in time series. VMD contains modes that are concentrated around the center frequency, leading to enhanced accuracy in reconstructing time series using Intrinsic Mode Functions (IMFs) [28]. To determine each mode and its corresponding center frequency, we can establish a constrained variational problem using the following formulation

$$\left\{\begin{array}{c}{\left\{u_k\left(t\right)\right\}}_{,}^{min}\left\{{\omega }_k\left(t\right)\right\}\left\{\sum _k‖{\partial }_t\left[\left(\delta \left(t\right)+{\displaystyle \frac{i}{\pi t}}\right)\otimes u_k\left(t\right)\right]e^{-j{\mathsf{\omega }}_{k}t}‖\right\}\\ s.t.\sum _k{u}_k\left(t\right)=\int \left(t\right)\end{array}\right.$$

(1)

where t represents the time step, δ(t) represents the Dirac distribution, u_k(t) represents the kth mode, ω_k(t) represents the associated center frequency, and f(t) signifies the t-th data of the signal being evaluated. Moreover, the Hilbert transform of u_k(t) may be represented as the convolution of (δ(t) + j/πt) and u_k(t). This transformation allows u_k(t) to be converted into analytical data, resulting in the creation of a frequency spectrum that only contains positive frequencies. Hence, based on the index term e^−jωkt, the collection of modes may be shifted to a baseband. To transform the provided optimization issue into a non-objective optimization term, two factors must be taken into account: Lagrange multipliers λ and the quadratic penalty parameter [29]. This research decomposed the groundwater level time series data, with the whole mode converging to a steady state at k = 3. The equation shown below represents the increased Lagrange function

$$L\left(u_k,{\omega }_k,\lambda \right)=\alpha {\sum }_{k=1}^k‖{\partial }_t\left[\left(\delta \left(t\right)+{\displaystyle \frac{j}{\pi t}}\right)\otimes u_k\left(t\right)\right]e^{-j{\omega }_{k}t}‖2+‖\int \left(t\right)-{\sum }_{k=1}^k{u}_k\left(t\right)‖2+\langle \mathsf{\lambda }\left(t\right)\int \left(t\right)-{\sum }_{k=1}^k{u}_k\left(t\right)\rangle$$

(2)

where $‖\int \left(t\right)-\sum _{k=1}^k{u}_k\left(t\right)‖2$ represents a quadratic penalty function used to minimize the convergence time. The alternative direction method of multipliers (ADMM) is an optimization technique that enables the updating of u_k(t) and ω_k(t) at two distinct locations to carry out the VMD method. Consequently, the altered equations are

$$u_k^{n+1}={\displaystyle \frac{\int \left(\omega \right)-\sum _{i\ne k}{u}_i\left(\omega \right)+\frac{\lambda \left(\omega \right)}{2}}{1+2\alpha \left(\omega +{\omega }_k\right)}}$$

(3)

$${\omega }_k^{n+1}={\displaystyle \frac{{\int }_0^{\infty }\omega {\left|u_k\left(\omega \right)\right|}^{2}d\omega }{{\int }_0^{\infty }{\left|u_k\left(\omega \right)\right|}^{2}d\omega }}$$

(4)

$${\lambda }^{n+1}\left(\omega \right)={\lambda }^n\left(\omega \right)+\eta \left(\int \left(\omega \right)-{\sum }_k{u}_k^{n+1}\right)$$

(5)

where n is the number of repetitions, λ(ω), f(ω) and u_i(ω) are the Fourier transform parameters, and η denotes the iterative factor.

2.3.2. VMD-iTransformer

Figure 3 demonstrates that iTtransformer employs the unadulterated encoder design of the Transformer model, which encompasses embedding, projection, and Transformer blocks. However, the majority of modern predictive models often use numerous variables simultaneously as indicators and adhere to a generative formula for the task of prediction [30]. Exploring numerical modalities has significant consequences for attention in learning, whereas the effectiveness of linear encoders has motivated the need for enhancements in the Transformer design. iTtransformer offers an encoder that prioritizes the learning of representations for sequences with many variables and adaptive correlation. VMD decomposes non-smooth time series into compact IMFs, which guarantees that the IMFs can accurately recreate the original data sequence. The time series of each IMF generated by the underlying complicated process is first tokenized to characterize the attributes of the variables. Subsequently, they interact with each other via self-attention and are separately processed by the feed-forward network to provide a representation of the series [31]. Therefore, in iTtransformer, the task of forecasting the future sequence of a specific variable Y, n, based on the historical sequence X, n, may be succinctly expressed as

$$\begin{gathered} h_n^0=Embedding\left(X_{:,n}\right), \\ {H}^{l+1}=TrmBlock\left(H^l\right),l=0,\dots ,L-1, \\ {\widehat{Y}}_{:,n}=Projection\left(h_n^L\right), \end{gathered}$$

(6)

where H, represented as H = {h₁, …, h_N} ∈ $\mathbb{R}$^N×D, consists of N embedded tokens with a dimension of D. The superscript indicates the layer’s index. The embedding of $\mathbb{R}$^T + $\mathbb{R}$^D and projection of $\mathbb{R}$^D $\mapsto$ $\mathbb{R}$^s are implemented using a multilayer perceptron (MLP). The acquired variable tokens engage in self-attention and are individually processed by the shared feed-forward network in each TrmBlock. Since the order of the sequence is already kept in the neuron permutation of the feed-forward network, the position’s embedding in the vanilla Transformer is unnecessary in this case.

In addition to its application to multivariate correlations, the VMD-iTransformer design does not have any special predetermined prerequisites. Hence, the attention method of the VMD-iTtransformer may serve as an add-on, effectively decreasing the complexity as the number of variables grows [32]. In addition, the attention mechanism’s input flexibility enables the model to handle varying numbers of tokens during both training and inference (Figure 3). This means that the model may be trained on any number of variables [33].

2.3.3. Inversion of the VMD-iTransformer

We developed a collection of L-shaped building blocks that include VMD decomposition, layer normalization, the feed-forward network, and self-attention modules. These blocks were optimized and reversed in the original Transformer model.

Layer normalization was used to normalize the multivariate representation of the same timestamp in this module of Transformer, eventually achieving the fusion of the variables. Nevertheless, this module has the drawback of producing disruptive noise during various occurrences [34].

To tackle the issue of non-stationary long-time series data, this technique employs normalization of the series representation of a single variable, as seen in Equation (7). Furthermore, by normalizing all sequences as (variable) markers to a Gaussian distribution, any differences resulting from inconsistent measurements may be minimized. In comparison with the original Transformer model, the revised model exhibits enhanced smoothness and efficiency in handling time series data [35].

$$LayerNorm\left(H\right)=\left\{\left.{\displaystyle \frac{h_n-Mean\left(h_n\right)}{\sqrt{Var\left(h_n\right)}}}\right|n=,\dots ,N\right\}$$

(7)

The Transformer FN is used to encode the fundamental components of the token format and is uniformly applied to every token. The VMD-iTransformer FN is used to process the sequential representation of each variable token to enhance the original model. This is achieved by reversing the modular method, since it addresses issues such as mislocalization and excessive localization of many variables, which may hinder the model’s predictive capabilities [36]. Through the use of a comprehensive approximation theorem, they may effectively characterize intricate time series. The majority of the current research has focused on MLPs. However, VMD-iTransformer is a method that decodes future time series by arranging inverted blocks and encoding observed time series using dense non-linear connections [37].

Recently, there has been a focus on using linear predictors to highlight the importance of sharing the temporal characteristics retrieved by MLPs across various time series. VMD-iTransformer enhances the performance by training the neurons of MLPs to capture essential characteristics of time series data, such as amplitude, periodicity, and spectra (acting as filters). This approach has been found to be a more effective predictive representation compared with using self-attention on individual time points [38]. Our findings demonstrated that lengthening the backtracking sequence leads to increased performance and enhances generalization to unknown variables.

Self-attention refers to a mechanism in machine learning where an input sequence is processed to compute a weighted sum of its elements. In contrast with traditional Transformer models, the iTransformer approach treats each variable sequence as an independent process, rather than using an attention mechanism to improve time-dependent modeling. To be precise, the vector h_y ∈ R^D is created by extracting the representation of each time series H = (h₀) completely. The self-attention module then uses a linear projection to obtain the query, key, and value vectors Q, K, and V ∈ R^N×dk, where dk is the projection dimension [39].

The pre-Softmax score for a certain query and key, represented as q_i and k_j, respectively, is calculated as follows. Each entry is indicated as $A_{i,j}={ \left(Q{K}^T/\sqrt{d_k}\right)}_{i,j} \propto q_i^T{k}_j$ and is equal to the dot product of the transpose of the query vector Q_i and the key vector K_j, divided by the square root of the dimension of the key vector dk. In mathematical notation, $A_{i,j}={\left(Q{K}^T/\sqrt{d_k}\right)}_{i,j} \propto q_i^T{k}_j$. As each token is standardized on the basis of its feature dimensions, these entries may partially disclose the correlation between variables, and the whole score graph A ∈ ${\mathbb{R}}^{N\times N}$ displays the multivariate correlation between pairs of variable tokens [40]. Therefore, in the subsequent representation that interacts with the value V, variables that are strongly connected will be assigned a higher weight. The VMD-iTransformer mechanism is considered to be more intuitive and interpretable for multivariate sequence prediction, based on this understanding.

2.3.4. Assessment of the Indicators

Due to their exceptional resilience to outliers, we chose MAE, MAPE, and MSE as loss functions during the training of the model. During the evaluation of the model’s performance on the test set, we used the coefficient of determination, R², to measure the level of agreement between the predicted and observed values [41].

$$e_{\mathrm{M}\mathrm{A}\mathrm{E}}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n|y_i-{\widehat{y}}_i|$$

(8)

$$e_{\mathrm{M}\mathrm{S}\mathrm{E}}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{|y_i-{\widehat{y}}_i|}^2$$

(9)

$$e_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}=\sqrt{{\displaystyle \frac{1}{n{\sum }_{i=1}^n{\left(|y_i-{\widehat{y}}_i|\right)}^2}}}$$

(10)

$$e_{\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n|y_i-{\widehat{y}}_i|∕y_i$$

(11)

where y_i is the true value of the original data, ${\widehat{y}}_i$ is the predicted value of the original data, n is the test sample size, and i is the test sample point sequence number, i ∈ {1, 2, …, n}.

3. Results

3.1. Advantages of Using VMD to Improve iTransformer’s Predictive Performance

When there is a large amount of lengthy time series data available, iTransformer is a very successful deep learning model for enhancing the dynamic prediction of non-stationary data. The improvement achieved is statistically significant, with a p-value of less than 0.001. VMD accomplishes this primarily by decomposing the lengthy time series data into Intrinsic Mode Functions (IMFs) that have a restricted range of frequencies (Figure 4). It then extracts the dynamic information at various scales from the original data. In contrast to the conventional Transformer model, iTransformer reorganizes the time series dimension by treating each variable as a time series. This approach is particularly useful for handling lengthy time series of non-stationary data, where the time step is transformed into a dimension for each variable. The MAPE for predicting groundwater levels’ dynamics can be enhanced by 80%. The primary rationale behind iTransformer is to convert an entire series of the same variable into high-dimensional features, which serve as representations. These feature vectors are then described as variables. Despite the iTransformer model’s proficiency in handling non-stationary data, it falls short in reliably forecasting time series data with intricate dynamic features. To rectify the deficiencies in groundwater levels’ time series data, we present a hybrid model incorporating VMD and iTransformer. VMD can efficiently split the data on groundwater levels into various frequency components and noise signals, yielding clearer and more significant signals for the subsequent iTransformer. VMD-iTransformer’s robust time series modeling capabilities effectively capture both long-term dependencies and short-term variations in the decomposed signals, enabling superior performance in addressing the non-stationarity and limited data inherent in groundwater levels’ time series predictions. Consequently, the integration of VMD with iTransformer not only resolves the challenges of the data’s simplicity and non-stationarity but also improves the capacity to identify and forecast intricate oscillations.

The predictive accuracy of VMD-iTransformer was markedly enhanced and assessed for significance at a confidence level above 95%. The mean MAPE among the three forecasts was about 1.7%, but the maximum MAPE in the prediction exhibiting the most inaccuracy was 1.9%. The findings demonstrated that the discrepancy between the experimental and simulated values was statistically significant. Furthermore, the VMD-iTransformer exhibited only a small increase in performance deterioration as the step size of prediction increased, with average losses per step of −0.0002 and −0.0001. Our research revealed that traditional Transformer-based models struggle to produce comparable results for predicting groundwater levels in arid and semi-arid areas. However, we discovered that by combining VMD with iTransformer, we can successfully capture the dynamics of groundwater levels even in highly non-stationary conditions. This approach also significantly enhances the accuracy of high time-frequency predictions.

3.2. Impact of VMD Decomposition on iTransformer Performance

The VMD decomposition approach may greatly enhance the accuracy of predicting groundwater levels’ dynamics, especially when dealing with limited data. Nevertheless, the Transformer deep learning model cannot provide accurate predictions directly because of the limited quantity of training data available in the target domain. This is evident from the average MAPE value of 21.3% observed across the nine regions. It is important to highlight that the performance of iTransformer, employing the VMD decomposition approach, underwent a significance test at a 95% confidence level. The findings indicated that its performance was markedly superior to that of the Transformer trained solely in a local context (p < 0.001). In addition to Transformer, iTransformer can also explicitly choose a source backbone for the target domain, but with some differences. However, its accuracy was significantly lower than the 10.3% achieved by the VMD-iTransformer model. In an evaluation of the selection of the source domain for predicting groundwater levels, VMD-iTransformer demonstrated the highest prediction performance, with the lowest average MAE of 0.05 across the nine stations. In contrast, Transformer performed the worst, with an average MAE of 0.21. Within the realm of multivariate predictions, we examined the impact of agricultural irrigation on the groundwater level. As a result, we saw a minor improvement in the predictive accuracy. This indicates that forecasts of groundwater levels are more precise compared with those made by the VMD-iTransformer model. This suggests that agricultural irrigation, particularly during the summer, might have an impact on the groundwater level. In our study on the dynamic prediction of groundwater levels, we observed that the average MAE for the climatic component, the basin factor (change in flow), and the human factor was 0.073, 0.079, and 0.081, respectively. Interestingly, we discovered that the human factor had very little impact on the groundwater level. Our findings indicated that the Transformer model’s performance may surpass that of models that only analyze a single aspect. This might be attributed to the Transformer model’s robust flexibility in handling the source domain. Additionally, the model may struggle to accurately capture the similarity across monitoring points. The Transformer model initially experienced a loss of 0.24. However, the iTransformer and VMD-iTransformer models can decrease the loss value more rapidly during training. This is achieved by utilizing an inversion module to handle non-stationary time series data more efficiently. Additionally, the coupling of VMD decomposition has proven to be more effective in reducing the training and validation losses, as well as removing multiscale dynamic information from the time series data. VMD-iTransformer demonstrated rapid convergence during training, requiring just 30 epochs to achieve a satisfactory loss level (Figure 5). This suggests that the model is very efficient in handling non-stationary time series data. These findings indicate that when selecting a source domain to forecast a target domain, taking the overall similarity between the two domains into account can lead to a more suitable choice of source for the target domain. This, in turn, can enhance the modeling performance of the VMD-iTransformer model.

3.3. Comparison Between Deep Learning Performance and Data Volume

Given the multitude of elements at play, we investigated how the amount of data impacts the accuracy of groundwater level predictions made by the VMD-iTransformer model. The neural network model’s weights are iteratively adjusted during training using optimization algorithms such as backpropagation and gradient descent. Several factors influence the weights’ values, including the initial weight selection, the performance of the decomposition algorithm, the learning rate’s setting, and the quality and quantity of the training data. These variables contribute to an inherent ambiguity in the ultimate determination of the weights, leading to a corresponding variability in the model’s performance. Nevertheless, as the volume of data in the source domain expands, the VMD-iTransformer model can acquire more knowledge, hence enhancing its predictive capabilities. Increasing the number of source domains used to train the VMD-iTransformer model leads to a significant improvement in the model’s prediction accuracy. This aligns with the findings presented in Section 3. Increasing the number of source domains leads to the most notable increase in prediction accuracy, which aligns with the findings presented in Section 3.2. When the number of source domains was increased to five, the predictive accuracy of the VMD-iTransformer model approached its maximum possible value, also known as the growth threshold. However, when the number of source domains was increased to nine, there was only a slight improvement in the model’s performance, with an MAE of 0.0251, as depicted in Figure 6. This indicates that the relationship between the number of source domains and the performance of the VMD-iTransformer model is not a simple linear positive correlation. Instead, it follows an approximately logarithmic positive correlation. In other words, when the number of source domains is small, increasing the number of source domains can greatly enhance the model’s performance. However, once the number of source domains reaches a certain level, further increasing the number of source domains has only a limited effect on improving the model’s performance. Hence, taking both computational efficiency and model performance into account, it is quite suitable to set the number of source domains to four.

Increasing the number of target domains utilized to train the deep learning model enhanced its performance. This demonstrates the significant effect of the quantity of training data on the performance of the model. After evaluating the performance of the Transformer, iTransformer, and VMD-iTransformer models, we discovered that VMD-iTransformer, which utilizes four source domains for pre-training, exhibited the highest level of performance. However, it only showed a slight improvement in accuracy compared with the iTransformer model. Additionally, when the volume of data of the target domains surpassed 6 months, the average MAE value stabilized, as depicted in Figure 7. When the training data for the model consisted of a limited number of target domains, there was a significant disparity in performance between the iTransformer model and the VMD-iTransformer model. Nevertheless, as the quantity of target domain data grew, the disparity in the predictive accuracy between these two models rapidly diminished. However, regardless of the quantity of target domain data, the VMD-iTransformer model consistently outperformed the iTransformer model. These findings provide more evidence of the exceptional performance of VMD-iTransformer in predicting groundwater levels.

4. Discussion

4.1. Importance of VMD Coupled with iTransformer in Deep Learning

This work demonstrates the successful integration of the VMD approach with an advanced iTransformer model to effectively address the issue of non-stationarity in predicting groundwater levels. For this work, we used the VMD approach to create the linked model. The VMD approach decomposes the original sequence into numerous subsequences (IMFs), serving as a data preprocessing tool before training the iTransformer model [41]. Furthermore, the findings indicate that VMD offers more accurate insights when handling non-stationary data in comparison to Empirical Mode Decomposition (EMD) and Complementary Ensemble Empirical Mode Decomposition (CEEMD) [42]. When there is enough amount of training data, the VMD approach may accurately capture the fluctuations in groundwater levels even in situations when the circumstances are very variable [43]. This is achieved by dissecting each subsequence separately. These findings align with the outcomes of prior research. As an example, Wen et al. conducted a study where they examined the performance of coupled EMD, EEMD, and VMD models in predicting the daily flow of the Jingjiang River in China [42]. The findings of their research revealed that the VMD model exhibited the highest level of performance. The VMD-iTransformer model we devised effectively handles the non-stationarity of groundwater level changes while making long-term predictions. The pre-trained VMD-iTransformer model enhances the groundwater level dynamics in the target domain by providing precise time-series data from specified source domains, effectively filling in the information gaps. It is important to mention that VMD-iTransformer requires each decomposed subsequence to be processed individually in iTransformer [3]. This results in a modeling time that is approximately 20 to 100 times longer than that of the traditional Transformer. Additionally, the modeling time increases as the number of source domains and the length of the time series increase. However, the VMD-iTransformer model proposed in this study can still be fine-tuned with limited data to enhance the predictive accuracy relatively easily.

Regarding long-term forecasting, iTransformer adeptly captures long-term dependencies in time series via its self-attention mechanism, hence excelling in extended-duration forecasts. Nonetheless, its capacity to isolate low-frequency signals is constrained, and it may exhibit instability when subjected to substantial noise interference [16]. Furthermore, it often neglects regional discrepancies. Conversely, VMD disaggregates the data into many modes, facilitating the separate processing of different frequency components; nonetheless, this results in heightened computational complexity and more intricate parameter tweaking [19]. Consequently, iTransformer possesses a considerable edge in long-term forecasting, whereas VMD-iTransformer demonstrates superiority in short-term predictions, managing limited datasets, and tackling non-stationarity.

4.2. Insights from Regional Projections of Ecological Reserves in Arid Zones

So far, there have been few studies that have focused on choosing deep learning models and suggesting suitable approaches for predicting the data of changes in groundwater levels in the original area [42]. Yao et al. utilized a comprehensive approach called CF-LT, which combines a full ensemble of empirical modal decomposition, adaptive noise, fuzzy entropy, long- and short-term memory, and Transformer to accurately predict total phosphorus (TP) levels in lakes [44]. This approach was specifically designed to overcome the issues of overfitting and underfitting that commonly occur when applying machine learning models to high-dimensional data. In 2022, Bai et al. employed a Koopman convolutional autoencoder to handle pre-training data [45]. They utilized an adaptive noise-based and Transformer-based deep learning model, which also incorporated the Koopman convolutional autoencoder for processing the pre-training data. This approach effectively yielded precise estimations of the contaminant concentration field in groundwater across a broader range of time intervals. Utilizing extensive time series datasets, deep learning models may greatly enhance predictive accuracy by training pre-existing models [46]. However, this process often necessitates substantial computer power. The computational time of the deep learning model may be influenced by the quantity of long-time series datasets [47]. Hence, in situations when there is only a limited amount of data in the source domain, it is necessary to consider pre-training based on this. This study presents a precise method for forecasting changes in groundwater levels in arid and semi-arid areas [48]. We achieved this by combining information about the ecological reserve, climatic factors, watershed factors (such as variations in the water flow), and human activities. By considering the similarities among these factors, we created a comprehensive model for predicting groundwater dynamics. We chose source domains that are representative of dry and semi-arid zones as target domains. This approach eliminated the need for the specific optimization parameters that are necessary for other zones and allowed us to fully use the predictive capabilities of the VMD method.

4.3. Exploring the Threshold of the Impact of Data Volume on Deep Learning Models’ Performance

The research revealed that augmenting the number of monitoring points in the source domain, which refers to the training data, has a substantial impact on enhancing the predictive accuracy of the VMD-iTransformer model. The VMD-iTransformer model exhibited superior performance in our pre-trained model, which was trained using all time series datasets. The source domains in this investigation consisted of nine monitoring points. Nevertheless, once the number of source domains exceeded five, the VMD-iTransformer model’s predictive accuracy approached its maximum limit. This indicates that there could be a specific limit for the number of source domains in VMD-iTransformer when modeling, perhaps because of the regularity of groundwater dynamics in the ecological reserve [49]. Our findings indicate that there is no linear correlation between the number of source domains and the level of detail in the data. This lack of correlation might potentially result in a decline in the computing efficiency of the VMD-iTransformer model as the number of source domain monitoring points grows. This raises the following question. While using the VMD-iTransformer model, is it imperative to enter a substantial quantity of data, or is a modest quantity of data satisfactory? When evaluating modeling efficiency, it is necessary to make a trade-off.

When comparing the Transformer, iTransformer, and VMD-iTransformer models, it was shown that the VMD-iTransformer model had the highest predictive accuracy independent of the target domain (i.e., the test data). The VMD-iTransformer model could accurately capture the specific characteristics of the changing groundwater levels in the original well, even after being trained on a large dataset for an extended period. The benefit of this model lies in the fact that, when used in the target domain, it requires just appropriate parameter tweaking and optimization to effortlessly achieve the necessary performance [50]. Our investigation revealed that when the target domain has a limited quantity of data, there is a significant disparity in performance between the Transformer model and the VMD-iTransformer model. However, the difference in performance between the iTransformer and the VMD-iTransformer model is comparatively lower. Nevertheless, when the target domain has enough data, the disparity in performance between the iTransformer and VMD-iTransformer models is significantly diminished [51]. This indicates that VMD has a notable enhancing impact on iTransformer and also highlights the significance of the VMD approach when dealing with restricted data.

The VMD-iTransformer model we present offers an effective solution for target domains located in arid and semi-arid zones. The VMD approach is capable of efficiently extracting information that can be trained using iTransformer. It can reliably forecast the dynamic groundwater levels in non-stationary long-time series. In summary, the combination of VMD and iTransformer is a powerful solution for accurately predicting changes in groundwater levels. This approach successfully overcomes the difficulty of predicting non-stationary data.

4.4. Limitations and Prospects

The primary objective of this work was to tackle a significant obstacle in predicting groundwater levels, which involved managing the lack of consistency in long-term data series. Nevertheless, our suggested technique exhibits a slower training pace compared with the iTransformer model. Future studies may increase the optimization of this approach by including more indicators associated with groundwater levels as inputs to the model [52]. This will allow for a more thorough investigation of the important drivers that cannot be adequately investigated in the model. Furthermore, the interpretability of the model can be improved by the use of interpretation methods. Furthermore, the VMD decomposition approach requires lengthy time series data in the target domain to fine-tune the pre-trained model, making it unsuitable for addressing the issue of absent data [53]. To enhance our suggested prediction technique, we may use a regionalization methodology to determine the groundwater levels in different regions. This would enable us to choose more suitable source domains for the target domain [54]. These enhancements and expansions will further optimize the efficacy of our approach in predicting groundwater levels.

5. Conclusions

This study presents a novel approach to improve short-term predictions of fluctuating groundwater levels, particularly in arid and semi-arid areas with limited monitoring systems. The proposed strategy integrates a variational mode decomposition method with the iTransformer model, resulting in a high-performance linked modeling tool. VMD-iTransformer, possessing robust time series modeling capabilities, effectively captures long-term dependencies and short-term fluctuations in decomposed signals, hence enhancing the model’s performance in predicting time series of groundwater levels characterized by non-stationarity and limited data. We assessed the efficacy of this model at nine groundwater-level monitoring stations located inside an ecological reserve on the outskirts of the Kubuqi Desert. Our technique achieved much higher accuracy than the classic Transformer model by moving the VMD-iTransformer model from the source domain to the target domain after pre-training and improving it primarily using the decomposition method. This investigation validated the significant capability of the VMD decomposition approach for enhancing the efficiency of iTransformer. The linked model has superior predictive capabilities for forecasting groundwater levels, particularly in the presence of significant non-stationarity, when compared with the iTransformer model. Nonetheless, VMD-iTransformer exhibits significant computational complexity and difficult parameter tweaking, which impose restrictions in managing large amounts of time series data and multiobjective prediction problems. Our suggested approach is capable of efficiently addressing the issues associated with predicting outcomes in arid and semi-arid regions, where there is a scarcity of monitors and restricted data availability. In summary, this work offers a successful method for addressing the issue of non-stationarity in predicting groundwater levels. Additionally, we provide an optimal strategy that allows for precise predictions on a broad scale in the future.