Next Article in Journal
A Baroclinic Fluid Model and Its Application in Investigating the Salinity Transport Process Within the Sediment–Water Interface in an Idealized Estuary
Previous Article in Journal
Design and Experimental Study of a Robotic Tuna with Shell-like Tensegrity Joints
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Nonstationary Control Parameter Prediction for Underwater Gliders Utilizing an Innovative Transformer-Based Model

by
Yv Jing
,
Chao Ma
,
Jianpeng An
,
Zezhong Li
,
Xinlin Sun
and
Wei Guo
*
School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
*
Author to whom correspondence should be addressed.
J. Mar. Sci. Eng. 2024, 12(11), 2106; https://doi.org/10.3390/jmse12112106
Submission received: 28 October 2024 / Revised: 14 November 2024 / Accepted: 18 November 2024 / Published: 20 November 2024
(This article belongs to the Section Ocean Engineering)

Abstract

:
As a new type of unmanned autonomous marine observation platform, underwater gliders (UGs) offer advantages such as low energy consumption and long operational ranges. However, during the gliding process, the complex marine environment often leads to abrupt changes in short-term control parameters, complicating the data and rendering them more challenging to predict. This typically poses difficulties in adjusting control parameters based on human experience, thereby significantly reducing UG control efficiency. To address this issue, this paper proposes a novel method termed DFFormer, aimed at enhancing the predictive accuracy of the rudder angles during UG motion. The proposed method integrates discrete wavelet transform (DWT) for rudder angle signal decomposition and employs a fast Fourier transform-based attention mechanism (FFT-Attention) to effectively capture and analyze its frequency- and time-domain characteristics. Notably, the method leverages a Transformer architecture to process the decomposed signals through multiple parallel pathways, substantially improving the capability to forecast the complex and variable control parameters of UGs. The effectiveness and practicality of the proposed method are demonstrated through actual sea trials. The experimental results indicate that the proposed method surpasses traditional approaches in terms of accuracy and computational efficiency, exhibiting superior performance in predicting UG control parameters and, to some extent, enhancing their heading-keeping ability.

1. Introduction

Underwater exploration is critical in understanding our oceans, impacting fields such as climate science, environmental monitoring, and resource management [1,2]. Recent technological advancements have enabled the development of sophisticated autonomous underwater vehicles (AUVs), equipped with high-resolution sensors, advanced navigation systems, and powerful propulsion capabilities, allowing them to operate in complex and deep-sea environments. Among these AUVs, underwater gliders (UGs) represent a unique subclass, first conceptualized by Henry Stommel in 1989. UGs utilize buoyancy adjustments to glide through the water, collecting data on essential parameters like temperature, salinity, and currents [3,4]. Their energy-efficient design enables long-term autonomous operation, making them particularly valuable for monitoring complex oceanic phenomena such as mesoscale eddies, internal waves, and hurricane activity [5].
Despite their inherent advantages, during operation, UGs confront considerable challenges arising from complex and time-varying currents, which can significantly compromise their control precision and operational efficiency [6,7,8]. The accurate prediction of their operational parameters is crucial for optimizing mission planning and ensuring data quality [9,10]. The current control methods for UGs in practical applications can generally be categorized into onboard control methods and remote control algorithms [11,12]. The former operate within the UG control unit, focusing extensively on constructing the dynamic model and selecting parameters specifically for the glider’s body. This tends to result in highly specific control approaches, lack universality, and struggle with the complexity and dynamic nature of underwater environments [7,13]. Remote control algorithms, operated from shore-based control centers, not only allow the application of complex control algorithms but also enable human intervention and real-time adjustments to the control precision of underwater gliders, thereby avoiding mishaps in this regard [14,15]. However, this undoubtedly increases human and time costs. Currently, UG control parameter tuning primarily relies on manual experience, which significantly lowers the accuracy and efficiency of control in practical missions [16].
In the research on the control and navigation of underwater gliders, scholars have proposed various methods to improve control accuracy, address environmental disturbances, and optimize energy efficiency. Yang et al. [17] employed an anti-system approach to decouple the original multi-input, multi-output (MIMO) system into two independent single-input, single-output (SISO) linear subsystems, then used sliding mode control (SMC) to control each subsystem individually, followed by simulation verification. Joo and Qu [18] applied the LQR method to control the depth of a zigzag gliding path, and also conducted simulation validation. La et al. [19] designed a layered PID controller that adjusted the heading via the rudder, achieving enhanced energy efficiency. Wang et al. [20] investigated how the initial heading error in underwater gliders affects trajectory accuracy and explore sensitivity analysis of navigation commands to enhance path precision.
In the context of interdisciplinary integration across various fields, researchers have introduced deep neural network models into UG control studies, aiming to reduce reliance on physics-based modeling. Shan et al. [21] proposed a model predictive control method based on recurrent neural networks to regulate the pitch angle, and its effectiveness was verified through simulation. Isa et al. [22] utilized neural networks to fit dynamic models and designed a neural network predictive controller for UG motion control, comparing its performance with model predictive controllers (MPCs) and linear quadratic regulators (LQRs) through simulation. Wang et al. [23] proposed a novel roll center compensation method (RCCM) utilizing variational mode decomposition and long short-term memory (VMD-LSTM) to accurately predict and minimize the roll regulation unit’s (RRU’s) energy consumption. Zhang et al. [10] introduced a hybrid model that integrates single prediction models with optimal weights, utilizing a simulated annealing-optimized Frank–Wolfe method for weight derivation, in order to accurately forecast the surfacing positioning point (SPP) of UGs at various time scales.
Building upon these advancements, other studies have explored various deep learning architectures and hybrid models to further enhance the predictive accuracy and operational efficiency of underwater gliders in diverse and dynamic marine environments. Immas et al. [24] proposed two prediction tools using deep learning, LSTM, and Transformer, to perform real-time in-situ prediction of ocean currents at any location. Mu et al. [25] proposed a novel navigation method for AUV with hybrid recurrent neural networks, which could also satisfy the real-time requirement. Liu et al. [26] proposed a VMD-BiGRU model for short-term buoy motion prediction, effectively enhancing accuracy by decomposing signals and capturing motion patterns across frequency scales. Hou et al. [27] proposed an SSA-optimized Informer model with CEEMDAN to forecast long-sequence ship motion attitude, enhancing prediction performance across varied sea conditions. Jiang et al. [28] proposed a joint multi-model machine learning method based on confidence to improve ship stability prediction accuracy, reducing input features and enhancing robustness. In order to further elaborate the literature survey, we show the main contributions of past studies in Table 1.
Despite significant advancements in predictive modeling for control parameters within the domain of UG research, the majority of studies have been validated through simulations, reflecting an absence of extensive empirical data. Consequently, there remains a substantial opportunity for enhancing the predictive accuracy of these models, particularly through the integration of more diverse and extensive datasets [29,30,31]. UGs are driven by buoyancy and have weak maneuverability when working within a strong ocean current [32,33]. Furthermore, UGs cannot be positioned underwater because of their lack of acoustic devices or inertial sensors, which introduces uncertainty in the control parameters within a single profile and exhibits strong nonlinear characteristics [34]. Therefore, the development of a novel parameter prediction method is crucial for the local or global control of UGs. Concurrently, the prediction of key UG control parameters, particularly the rudder angle, aids in the dissemination of control decisions and heading maintenance during task execution.
Table 1. Studies related to underwater glider parameter prediction.
Table 1. Studies related to underwater glider parameter prediction.
CategoryReferenceMethodDescription and Conclusions
LSTM variants[35,36,37,38]Multiscale attention-based LSTM, self-attention LSTM (SALSTM), LSTM, LSTM with multi-head attentionVarious LSTM variants have been proposed, including multiscale attention, self-attention, and multi-head attention LSTMs, to improve prediction accuracy for ship motion, roll, surge, and pitch by enhancing robustness and handling different input-output ratios.
Deep learning and attention mechanism models[24,25,39]Hybrid RNN, LSTM, Transformer, weighted attention modelA range of deep learning and attention mechanisms, such as hybrid RNNs, Transformers, and weighted attention models, have been used for real-time predictions of ocean currents and AUV navigation, enhancing prediction accuracy across broad temporal and spatial ranges.
Data decomposition and hybrid models[26,40]VMD-BiGRU, TVF-EMD + SVRData decomposition and hybrid models like VMD-BiGRU and TVF-EMD + SVR have been developed to improve the accuracy of short-term buoy motion and ship roll predictions by addressing frequency-specific patterns, nonlinearity, and time-varying dynamics.
Informer and Its Variants[27,41]Conv-Informer, SSA-Optimized CEEMDAN-InformerInformer variants, such as Conv-Informer and SSA-Optimized CEEMDAN-Informer, are used for multi-step and long-sequence ship motion predictions, capturing both local and long-range dependencies, and enhancing performance across varied sea conditions.
Multi-model and combination methods[10,28]Multi-model machine learning, combination modelMulti-model and combination methods like confidence-based machine learning and optimized weighting models improve the prediction accuracy of ship stability and surfacing positioning for underwater gliders across different time scales.
To sum up, due to the large time–space span during the observation mission, UG control parameters, particularly the rudder angle, are challenging to predict accurately utilizing traditional deep neural network methods, which exhibit a deficiency in this regard. Considering the vast and variable dataset of UGs, deep learning (DL) [42,43] is applied to establish the method that can predict the new parameter of the rudder angle in the next profile to guide the adjustment for heading correction in advance. The integration of DWT [44] and FFT-Attention mechanisms [45] into DL models has shown promise in capturing both high-frequency details and low-frequency trends in UG data. Such hybrid models can significantly enhance the robustness and accuracy of predictions by leveraging the strengths of both time- and frequency-domain analyses. Furthermore, the use of recursive neural networks (RNNs) [46], such as long short-term memory (LSTM) [47] and gated recurrent unit (GRU) [48] networks, has proven effective in capturing temporal dependencies in glider data, thereby improving the prediction of dynamic behaviors over time [49]. These models can learn from sequential data and provide robust predictions, even in the presence of noise and missing data, which are common in marine environments [50,51].
This paper proposes a novel method termed DFFormer to predict UG control parameters, with a particular focus on rudder angles. The proposed method integrates discrete wavelet transform (DWT) for rudder angle signal decomposition of the rudder angle and employs a fast Fourier transform-based attention mechanism (FFT-Attention) to effectively capture and analyze its frequency- and time-domain characteristics. Notably, the method leverages a Transformer architecture to process the decomposed signals through multiple parallel pathways, substantially improving the capability to forecast the complex and variable UG control parameters. The incorporation of an attention mechanism significantly bolsters the model’s capacity to identify pivotal features within time-series data, culminating in a more robust and dependable forecasting framework. The efficacy of the proposed method is validated through actual sea trials. The results demonstrate significant improvements in UG prediction accuracy and operational efficiency of UGs, thereby enhancing their heading-keeping ability. This study not only advances the field of underwater vehicle parameter prediction but also offers valuable insights for the application of deep learning techniques in other types of autonomous underwater systems.
The rest of this paper is organized as follows: Section 2 introduces the modeling approach for underwater gliders and the working principles of the proposed DFFormer method, focusing on the kinematic equations, control strategies, and the design and implementation of the discrete wavelet transform, fast Fourier transform-based attention mechanism, and recursive layers. Section 3 presents the experimental results and analysis, comparing the performance of various prediction methods using actual sea trial data. Finally, Section 4 concludes the research findings and outlines future research directions.

2. Underwater Glider Modeling

2.1. The Petrel-L Underwater Glider

Petrel-L, depicted in Figure 1, is an advanced underwater glider developed by Tianjin University in China. It features a cylindrical pressure hull that accommodates several components, including the emergency device, fixed wings, two fairings, and a trailing antenna. Within this pressure hull, the glider is equipped with numerous subsystems such as buoyancy-regulating, pitch-regulating, roll-regulating, and control units, as well as communication, navigation units, and a battery pack.
As illustrated in Figure 2, UGs performing ocean observation operations are generally remotely controlled via satellite communication from a shore-based control center. Upon receiving instructions, the glider undertakes profile observation maneuvers, while onboard sensors simultaneously gather data on the marine environment. The conventional workflow comprises three phases: the surface communication, diving, and surfacing phases. Due to technological limitations, current technologies do not allow for stable communication during deep water operations; the glider is, thus, in a no-communication state while diving.
Surface communication phase: The buoyancy drive unit of the underwater glider adjusts the total displacement volume to ensure its net buoyancy exceeds gravity, allowing it to float on the surface. According to a predetermined optimal communication angle, the attitude adjustment unit configures the glider in a nose-down posture, exposing the stern antenna above the water surface for satellite link communication with the shore-based control center. During this phase, the glider transmits its location and attitude information and receives control commands.
Diving phase: Following the reception of a “start gliding” command from the control center, the buoyancy drive unit adjusts the displacement volume to render the glider’s net buoyancy less than gravity. The attitude adjustment unit continuously modifies the pitch and heading angles to maintain a pre-set pitch angle for the diving maneuver. The onboard control unit concurrently activates the task sensors to begin data collection.
Climbing phase: Once the depth sensor detects that the actual depth has reached a pre-set threshold, the buoyancy drive unit adjusts the buoyancy of the glider to exceed gravity, initiating upward movement. Simultaneously, the attitude adjustment unit transitions the glider from a nose-down to a nose-up posture, maintaining the pre-set pitch angle during the ascent. The task sensors operate similarly to the diving phase until the depth sensor indicates that the actual depth has reached the surface level, after which the various modules of the underwater glider enter the next profile surface communication phase.

2.2. Underwater Glider Kinematic Modeling

The ‘Petrel-L’ underwater glider is elliptical, and it navigates through the Earth’s oceans by pitching, rolling, and yawing. Therefore, it is necessary to establish body coordinate B 0 ( b 1 , b 2 , b 3 ) , velocity coordinate V 0 ( v 1 , v 2 , v 3 ) , and inertial coordinate systems E 0 ( e 1 , e 2 , e 3 ) , as shown in Figure 3.
The vector b = [ X , Y , Z ] T and Φ = [ ϕ , θ , ψ ] T represent the position and attitude angles of the underwater glider’s buoyancy center in the inertial coordinate system, respectively. Among them, X and Y are the horizontal plane coordinates, Z is the depth; ϕ is the roll angle, which is the angle between B 0 b 1 b 2 and plane E 0 e 1 e 2 . A positive value indicates a right roll when viewed from the tail to the nose of the vehicle; θ is the pitch angle, which is the angle between the longitudinal axis B 0 b 1 of the underwater glider and the plane E 0 e 1 e 2 . A positive value indicates a pitch-down angle; a negative value indicates a pitch-up angle; ψ is the yaw angle, which is the angle between the projection of the longitudinal axis B 0 b 1 of the underwater glider on the plane E 0 e 1 e 2 and E 0 e 2 . A positive value indicates a right yaw when viewed from the tail to the nose of the vehicle.
The vector V = [ u , v , w ] T and Ω = [ p , q , r ] T represent the velocity and angular velocity of the underwater glider’s buoyancy center in the body coordinate system, respectively. Among them, u, v, and w are the linear velocities of the glider’s buoyancy center along the b 1 , b 2 , and b 3 directions, and p, q, and r are the angular velocities of the glider’s buoyancy center rotating around the b 1 , b 2 , and b 3 axes, respectively.
The angle of attack α and sideslip angle β are defined in the velocity coordinate system. The projection of the glider’s buoyancy center velocity vector on the B 0 b 1 b 3 plane, the angle between the projection and the b 1 axis is the angle of attack α . A positive value indicates that the projection is below the b 1 axis, and a negative value indicates that it is above the b 1 axis; the angle between the glider’s buoyancy center velocity vector and the projection is the sideslip angle β . When viewed from the tail to the nose of the vehicle, a positive value indicates that the velocity vector is to the right of the b 1 axis, and a negative value indicates that it is to the left of the b 1 axis.
To design control strategies for UGs, it is necessary to comprehensively consider the glider’s position, angle, velocity, and angular velocity information. Therefore, these variables must be transformed into the same coordinate system. First, rotate the inertial coordinate system around the e 3 axis by an angle ψ , with the rotation matrix T Z ( ψ ) . Then, rotate around the e 2 axis by an angle θ , with the rotation matrix T Y ( θ ) . Finally, rotate around the e 1 axis by an angle ϕ , with the rotation matrix T X ( ϕ ) , to obtain the body coordinate system. Let the rotation matrix from the inertial frame to the body frame be denoted as T B E ; then, T B E is specifically expressed as follows:
T B E = T Z ( ψ ) T Y ( θ ) T X ( ϕ )
= cos ψ sin ψ 0 sin ψ cos ψ 0 0 0 1 cos θ 0 sin θ 0 1 0 sin θ 0 cos θ 1 0 0 0 cos ϕ sin ϕ 0 sin ϕ cos ϕ
= cos ψ cos θ cos ψ sin θ sin ϕ sin ψ cos ϕ cos ψ sin θ cos ϕ + sin ψ sin ϕ sin ψ cos θ sin ψ sin θ sin ϕ + cos ψ cos ϕ cos ψ sin θ cos ϕ cos ψ sin ϕ sin θ cos θ sin ϕ cos θ cos ϕ
The rotation matrix from the body to the velocity coordinate system is denoted as T V B . The specific expression of T V B is as follows:
T V B = cos α cos β cos α sin β sin α sin β cos β 0 sin α cos β sin α sin β cos α
Studying the kinematic equations of UGs is equivalent to studying their motion in an inertial coordinate system. Therefore, it is necessary to use the rotation matrix T B V to transform the velocity V in the frame of the UG, and then use the rotation matrix T V B to transform the velocity in the frame of the UG to V E . The transformation expression for V E is as follows:
V = u v w = T B V V 0 0 = V cos α · cos β V sin β V sin α · cos β
V E = V 1 V 2 V 3 = X ˙ Y ˙ Z ˙ = T E B V = T E B R V cos α · cos β V sin β V sin α · cos β
In addition to velocity, the state variables of an underwater glider in the inertial coordinate system also include the angular variables Φ = [ ϕ , θ , ψ ] T . The expression, using the angular velocity in the body coordinate system through the rotation matrix T V B as Ω = [ p , q , r ] T , is as follows:
θ ˙ = 1 sin ϕ tan θ tan θ cos ϕ 0 cos ϕ sin ϕ 0 sec ϕ sin ϕ cos ϕ sec θ Ω
The expressions for obtaining the magnitude of the velocity vector, the angle of attack, and sideslip angle are as follows:
V = u 2 + v 2 + w 2 α = arctan w u β = arcsin v V
By substituting the expression of the rotation matrix into Equations (5) and (6), we obtain the kinematic equations of the underwater navigation robot as follows:
X ˙ = u cos θ cos ϕ + v ( cos ψ sin θ sin ψ sin ψ cos ϕ ) + w ( cos ψ sin θ sin ϕ + sin ψ sin ϕ ) Y ˙ = u sin ψ cos θ + v ( sin ψ sin θ sin ϕ + cos ψ cos ϕ ) + w ( sin ψ sin θ cos ϕ cos ψ sin ϕ ) Z ˙ = u sin θ + v cos θ sin ϕ + w cos ϕ cos θ ϕ ˙ = p + q tan θ sin ϕ + r tan θ cos ϕ θ ˙ = q cos ϕ r sin ϕ ψ ˙ = 1 cos ϕ ( q sin ϕ + r cos ϕ )

3. Methodology

This paper proposes a novel model for forecasting UG control parameters based on the Transformer, including an encoder and decoder, as shown in Figure 4.
In the encoder and decoder, DWT is utilized to decompose the series into high-frequency details and low-frequency trend sections. In the former, these signals have a large number of frequency-domain features. FFT-Attention is, therefore, used to extract their frequency features. In the low-frequency trend part, there are fewer frequency features, and the time-domain features are considered to be more dominant. As such, the attention mechanism is used to extract them. In order to enhance the model’s feature extraction capability, parallel recursive layers are created. Moreover, a fusion block is proposed to integrate these extracted features’ information. Finally, multi-fusion block layers are used to integrate the learned information into the output for prediction. Specifically, the model construction process is as follows.
For every time step, let X = { x 1 , x 2 , , x n } represent the input time series, where x n b × ( s + p ) × 1 , x i = { h 1 , h 2 , , h l , , h s + p } ( s + p ) × D ; b indicates the batch size, s represents the input length, p denotes the prediction steps, and D denotes the input dimension. Similar to the Transformer, the encoder and decoder inputs of the proposed model are X E b × s × D and X D b × l × D , respectively. l denotes the length of the decoder, i.e., the label length.

3.1. Using Discrete Wavelet Transform to Decompose the Input Series

In this paper, DWT is performed to decompose these time-series signals into trend and detail components utilizing low- and high-pass filters. DWT decomposes a given discrete signal into orthogonal wavelet functions. In the case of 1D signals, like a time series, results are shown in a transformed vector of equal length. The vector is first filtered with a low-, and then, a high-pass filter. We can mathematically represent a DWT as follows in Equation (8):
Ψ l , m ( t ) = 2 1 2 ψ ( 2 l t m )
The terms l and m in the above equation for DWT represent the scale factor and translation index, respectively. ψ ( ) denotes the mother wavelet function. In detail, the input series X is decomposed by convolving a low- L = L 1 , L 2 , , L K and high-pass filter H = H 1 , H 2 , , H K , where K s and K denote the lengths of the input series, as shown in Equations (9) and (10):
A 1 i = k = 1 K x 2 i + K 1 × L K
D 1 i = k = 1 K x 2 i + K 1 × H K
where A 1 i and D 1 i are the i t h elements of the trend and detail components obtained from the single-level discrete wavelet decomposition, and x denotes the element. Therefore, the DWT results of these input series can be obtained according to Equations (9) and (10), as depicted in Equations (11) and (12), as follows:
X E D , X E T = D W T ( X E )
X D D , X D T = D W T ( X D )
where X E D and X E T b × s × ( D × K ) , X D D and X D T b × ( l + p ) × ( D × K ) , and D W T represents the discrete wavelet transform according to Equations (8)–(10).

3.2. Using FFT-Attention to Extract Parallel Features in Frequency Domain

For the encoder section, embedding is carried out to enhance the information of the input time series for more effective feature capture utilizing FFT-Attention. Firstly, these vectors can be obtained from Equation (13):
X E D E , X E T E = E m b e d d i n g ( X E D , X E T )
where X E D E and X E T E b × s × d , d denotes the feature dimension, and E m b e d d i n g indicates the embedding structure.
In the high-frequency details section, based on signal decomposition and an adapted mechanism, FFT-Attention was utilized to exact the feature information in terms of frequency. In particular, the input vectors X E D E are transformed to query Q , key K , and value V vectors by utilizing the feature augment method, as in Equations (14)–(16):
Q = X E D E × W Q
K = X E D E × W K
V = X E D E × W V
where W Q , W K , and W V denote the related parameter matrices, respectively. Then, these vectors are transformed multiple times to obtain multiple self-attention sub-layers, i.e., Q = { Q 1 , Q 2 , , Q h } , where h is head size and Q i b × s × ( d h ) indicates the i t h sub-layer. Similarly, { K = K 1 , K 2 , , K h } and V = { V 1 , V 2 , , V h } can be obtained. In every sub-layer, these vectors Q i , K i , and V i are evaluated using fast Fourier transform in order to transfer the features to the frequency domain, as depicted in Equation (17):
Q i ^ , K i ^ , V i ^ = F F T ( Q i , K i , V i )
where d f denotes the feature dimension after the fast Fourier transform, and Q i ^ , K i ^ , V i ^ b × s × d f ( F F T denotes fast Fourier transform). Moreover, every sub-layer can be calculated using Equation (18):
A t t n ( i ) = A t t n ( Q i ^ , K i ^ , V i ^ ) = S o f t m a x ( Q i ^ K T ^ d f ) × V i ^
Therefore, all sub-layer outputs are concatenated to acquire the feature vectors X E D E A in Equation (19):
X E D E A = C o n c a t ( A t t n ( 1 ) , A t t n ( 2 ) , , A t t n ( h ) ) W h
where W h indicates the related parameter matrix. Furthermore, inverse fast Fourier transform is performed to make the signal return to the time domain in Equation (20):
X E D E A T = i F F T ( X E D E A )
where X E D E A T b × s × d , and i F F T denotes the inverse fast Fourier transform. Next, one feed-forward layer and two residual links are created, as in Equation (21):
X E D E A F = N o r m ( R e L U ( N o r m ( X E D E A T W n 1 + b n 1 ) + W n 2 ) + b n 2 ) W n 3 + b n 3
where N o r m indicates one residual links function, R e L U denotes one feed-forward layer function, and W n 1 , W n 2 , W n 3 d f × d f represent learnable parameters.

3.3. Using the Recursive Layer to Capture Inference Features

In addition, the recursive layer is utilized to better extract feature information by capturing inference features. In this recursive layer, four structures can be chosen for information capture, including MLP, RNN, LSTM, and GRU. In detail, these structures can be represented as follows:
  • Multi-layer perceptron.
MLP is a feed-forward network that learns weights θ , mapping inputs to outputs. For a model with two hidden layers, the chain structure of multiple stacked layers provides depth. In order to improve the learning ability of the model, nonlinear functions, such as R e L U or s i g m o i d functions, are applied to the neuronal output. The optimal weight is determined by minimizing a differentiable loss function using backpropagation, which updates the network weight by propagating the gradient of the weight relative to the loss function back through the network.
2.
Recurrent neural network.
The main purpose of the RNN, a structural diagram of which is shown in Figure 5, is to process and predict sequence data. The calculation process of the classic RNN can be expressed as Formulas (22) and (23):
h t = f ( U x t + W h t 1 + b )
y = s o f t m a x ( V h t + c )
where the inputs are x 1 , x 2 , ⋯, x t , the corresponding hidden states are h 1 , h 2 , ⋯, h t , and the outputs denote y 1 , y 2 , ⋯, y t . U, W, and V denote the weight parameters, b and c represent biases, and f denotes the activation function, which is usually the t a n h function, as follows:
3.
Long short-term memory.
LSTM, a structural diagram of which is shown in Figure 6, is composed of forget, input, and output gates. These jointly control the storage and deletion of data, as shown in Equations (24)–(28):
f t = δ ( W f h × h t 1 + W f x × x t + b f )
i t = δ ( W i h × h t 1 + W i x × x t + b i )
o t = δ ( W o h × h t 1 + W o x × x t + b o )
c t = f c × c t 1 + i t × t a n h ( W o × [ h t 1 , x t ] + b o )
h t = o t × t a n h ( c t )
where f t denotes the forget gate, i t represents the input gate, and o t indicates the output gate. W f h , W f x , W i h , W x h , W o h , and W o x are the related weight matrices. b f , b i , and b o are the related bias; c t denotes the data retained from the beginning to the current moment; h t 1 saves the information of the previous time step t 1 , while h t controls how much retained data can be transferred to the next moment.
4.
Gated recurrent unit
As shown in Figure 7, a GRU has only two gates. It combines the input and forget gates found in LSTM into a single gate; this is called the update gate, as depicted in Equations (29)–(32):
z t = δ ( W z h × h t 1 + W z x × x t + b z )
r t = δ ( W r h × h t 1 + W r x × x t + b r )
h t = t a n h ( W x h × x t + r t × W h h × h t 1 )
h t = z t × h t 1 + ( 1 z t ) × h t
where z t denotes the update gate and r t represents the reset gate. W z h , W r h , and W x h denote the related weight matrices. b z and b r indicate the related biases. h t denotes new information and h t denotes the output.
These layers are replaced with R L . Therefore, the feature vectors extracted by this layer can be represented as shown in Formula (33):
X E D E L = R L ( X E D E )
where X E D E L represents the feature vector extracted by the GRU layer, and R L denotes the nonlinear activation function that replaces it. The feature vector X E D E L is a three-dimensional tensor with dimensions b × s × d , where b denotes the batch size, s denotes the sequence length, and d denotes the feature dimension.

3.4. Integrating Extracted Features Using the Fusion Block

Different features often have different complexities, and simple addition cannot determine the corresponding weight, resulting in inadequate integration. To effectively integrate this feature information, for the first time, in this paper, we present the fusion block. In particular, we define the gate argument as g, as shown in Equation (34):
g = σ × ( X E D E A T × W E 1 + X E D E L × W E 2 ) + E g
where g b × s × 1 , W E 1 , W E 2 d × 1 are the relevant weight matrices, respectively, and E g denotes the learnable parameter. Furthermore, this feature information can be integrated into Equation (35):
X E D = ξ × ( ( 1 + g ) X E D E A T + ( 1 g ) X E D E L )
where X E D b × s × d , ξ is a parameter set to 1 2 , and ⨂ represents the dot product.

3.5. Extracting Parallel Features in the Time Domain Using Attention

In this low-frequency trends section, as with the detailed part, fast Fourier transform is not used. More specifically, referring to Formulas (14)–(16), in the i t h sub-layers, the feature vectors being characterized can be defined as Q i , K i , and V i . Then, the feature vectors X E T E A can be obtained according to Equations (18) and (19). The remaining steps are consistent with the detailed section based on Equations (21)–(35), and the trend characteristics can be presented as X E T . Next, one feed-forward layer and two residual links are created, as shown in Equation (36):
X E T = N o r m ( R e L U ( N o r m ( X E T W n 4 + b n 4 ) + W n 5 ) + b n 5 ) W n 6 + b n 6
where N o r m indicates one residual link function, R e L U denotes one feed-forward layer function, and W n 4 , W n 5 , W n 3 d × d represent learnable parameters. The rest of the structure mirrors that of the high-frequency details, and the features X E T b × s × d can be obtained in the low-frequency trends section.

3.6. The Output Prediction Results Utilizing the Decoder Structure

In the decoder, to realize the prediction function, the information must be masked in the training. As such, the masking attention mechanism is used. Similar to before, except for the masking operation, the attention computation steps are the same as in the encoder. The detailed steps are as follows.
Firstly, embedding is executed to obtain enhanced feature information, as in Equation (37):
X D D E , X D T E = E m b e d d i n g ( X D D , X D T )
where X D D E and X D T E b × ( l + p ) × d . Secondly, similar to the steps of the encoder, the feature vectors of the i t h sub-layers Q i ^ , K i ^ , and V i ^ can be obtained according to Equations (14)–(17). Thirdly, a mask matrix is added to the sub-layers of the attention calculation, as referred to in Equation (18), and depicted in Equation (38):
M A t t n ( i ) = S o f t m a x ( Q i ^ K i T ^ d f × W m a s k ) × V i ^
where W m a s k b × ( l + p ) × d f denotes the mask matrix, i.e., the 0–1 diagonal matrix. The feature vectors X D D E A are calculated using Equation (39).
X D D E A = C o n c a t ( M A t t n ( 1 ) , M A t t n ( 2 ) , , M A t t n ( h ) ) W h 2
where W h 2 denotes the related parameter matrix. Referring to Equations (20) and (21), the masked feature vectors X D D E A T can be obtained. The masked feature vectors X D D E L are calculated in the same way. The fusion block is utilized to integrate X D D E A T and X D D E L to obtain X D D , as shown in Equation (40):
X D D = ξ × ( ( 1 + g ) X D D E A T + ( 1 g ) X D D E L )
where X D D b × ( l + p ) × d , with ξ denoting a scaling factor and g denoting a gating parameter. This produces the fused feature vector X D D , which serves as the integrated feature representation for the subsequent decoding steps.
Next, let X D D be the input as query vectors to FFT-Attention, and X E D be the key and value vectors. This allows the extracted feature information in the encoder to be integrated into the decoder for forecasting. Referring to Equations (14)–(19), the feature vectors X D D A can be computed using Equations (41)–(44):
X D D A = C o n c a t ( C A t t n ( 1 ) , C A t t n ( 2 ) , , C A t t n ( h ) )
C A t t n ( i ) = S o f t m a x ( ( X D D i × W Q ) × ( X E D i × W K ) T d f ) × ( X E D i × W K )
X D D = { X D D 1 , X D D 2 , , X D D h }
X E D = { X E D 1 , X E D 2 , , X E D h }
where X D D i b × ( l + p ) × d f , and X E D i b × s × d f . W Q , W K , and W V denote the related parameter matrices, respectively. The extracted trend feature vectors X D T b × ( l + p ) × d can be obtained in a similar manner. Then, similarly, according to Equations (20)–(35), the feature vectors X D D F b × ( l + p ) × d of the already fused details are obtained.
In a similar way, the fused trend feature vectors X D T F b × ( l + p ) × d can be computed. Furthermore, the fusion block is created to integrate the fused detail and trend feature vectors and obtain the output feature vectors X o u t b × ( l + p ) × d in Equations (45) and (46), according to Equations (34) and (35).
g = σ × ( X D T F × W E 3 + X D D F × W E 4 ) + E g
X o u t = ξ × ( ( 1 + g ) X D T F + ( 1 g ) X D D F )
where g b × s × 1 , W E 3 , W E 4 d × 1 are the relevant weight matrices, respectively, and E g denotes the learnable parameter. Finally, inverse wavelet transformation with a linear layer is utilized to obtain prediction information, as in Equation (47).
X P = L i n e a r ( i D W T ( X o u t ) )
where X P b × ( l + p ) , i D W T denotes the inverse wavelet transformation function, and L i n e a r denotes the linear layer function.

3.7. Optimizing the Neural Network Structure Using the Huber Loss Function

Because of the harsh running environment, which is strongly affected by outliers, the model’s precision must be based on a robust loss function for optimization. The Huber loss function is robust, combining the advantages of mean absolute error (MAE) and mean square error (MSE). The Huber loss function not only has continuous derivatives but can also use the MSE gradient to reduce errors and obtain more accurate results. The Huber loss function can be expressed as in Equation (48):
L o s s = 1 2 ( y y ) 2 , y y θ θ y y 1 2 θ 2 , y y > θ
where y denotes the original series and y represents the prediction series.

4. Experimental Results and Discussion

In this section, we use a dataset derived from sea trials to build predictive models. Specifically, the dataset captures the rotational angle of the regulatory unit’s mass during the steady-state gliding phase, hereinafter referred to as the rudder angle, and determines the mean rudder angle across both the descent and ascent phases. First, we detail the acquisition and preprocessing of the raw data. Then, we employ five established predictive algorithms, including Informer, Autoformer, VMD-SSA-LSSVM, VMD-LSTM, and DFFormer, to train and forecast the rudder angle from the training dataset. The validation set is used to validate and evaluate the performance of these methods. After a comprehensive comparison and analysis of the predictive results from various methods, DFFormer is conclusively identified as the optimal forecasting method for UGs.

4.1. Research Data

From April to June 2023, the Petrel-L glider underwent sea trials in the South China Sea. Figure 8 illustrates its area and trajectory during the oceanic experiment, encompassing a longitudinal span from 110° E to 117° E and a latitudinal span from 16° N to 21° N. Additionally, an inset in the upper left corner displays a deployment photograph of the glider. Throughout the 70-day trial period, the glider functioned without any reported issues, successfully compiling a dataset comprising 1110 individual profiles.
Figure 9 displays the depth and rudder angle data for the Petrel-L glider over a specific period, corresponding to profiles 480 to 483. As shown, both the depth (indicated by the blue line) and the rudder angle (represented by the red line) fluctuate with gliding time, particularly during the descent and ascent phases when the RRU operates frequently. Prolonged underwater operations can lead to biofouling, the accumulation of RRU errors, and unpredictable disturbances from ocean currents, all of which may induce lateral imbalances in the glider. These imbalances can affect the stability of heading-keeping. Consequently, it is essential to make dynamic predictions for the roll center’s imbalances, particularly focusing on predicting rudder angles. By accurately forecasting these angles, the frequency of rudder adjustments can be minimized, thereby significantly enhancing the glider’s heading-keeping performance.
To ascertain the authenticity and reliability of the dataset, several stringent prerequisites must be satisfied before utilizing the raw UG data for estimating the mean rudder angle:
  • Eliminate the time-series raw data obtained when the buoyancy adjustment unit is operating. Data collected during buoyancy adjustments capture rapid changes that are unrepresentative of the glider’s steady-state gliding dynamics. By removing these intervals, the dataset retains only stable conditions, reducing noise and preventing the model from learning patterns that could reduce reliability.
  • Eliminate the raw data during UG’s surface positioning and communication periods. During surface positioning, environmental influences such as wave and current impacts cause irregular rudder adjustments. Excluding these segments allows the model to focus on stable underwater data, eliminating inconsistencies that might compromise prediction authenticity.
  • Since the mean rudder angle differs between ascent and descent, process the data for these two phases separately. The hydrodynamic conditions for ascent and descent differ, leading to distinct control requirements for the rudder angle. Treating these phases separately ensure that phase-specific patterns are accurately captured, enhancing the model’s capacity to provide robust and realistic parameter predictions.

4.2. Evaluation Indicators

In order to substantiate the precision of the predictive methodologies, an array of rigorous scientific evaluative indicators were meticulously selected to appraise the forecast outcomes’ efficiency. These indicators include the mean absolute error (MAE), the root mean squared error (RMSE), and the symmetric mean absolute percentage error (SMAPE).
MAE represents the average magnitude of the errors between the estimated values and the original data, without considering their direction. MAE is defined as follows:
MAE = 1 n i = 1 n x i ( t ) y i ( t )
where x i ( t ) represents the original values, y i ( t ) represents the estimated values, and n represents the number of observations.
RMSE is another commonly used metric that represents the square root of the average squared differences between the estimated values and the original data. It is defined as follows:
RMSE = 1 n i = 1 n x i ( t ) y i ( t ) 2
where x i ( t ) represents the original values, y i ( t ) represents the estimated values, and n represents the number of observations.
RMSE is sensitive to larger errors and, thus, provides a measure that emphasizes larger discrepancies between the predicted and observed values.
Additionally, the symmetric mean absolute percentage error (SMAPE) is used to further evaluate the performance of the prediction methods. SMAPE is described as follows:
SMAPE = 100 % n i = 1 n | x i ( t ) y i ( t ) | ( | x i ( t ) | + | y i ( t ) | ) / 2
where x i ( t ) represents the original values, y i ( t ) represents the estimated values, and n represents the number of observations.
Generally, lower MAE, RMSE, and SMAPE values indicate a better performance on the part of the prediction method.
To further assess the predictive accuracy, Theil’s inequality coefficient (TIC) and the Index of Agreement (IA) were also employed.
TIC is a measure of the relative accuracy of the predictions, where a lower TIC indicates better predictive performance. It is defined as follows:
TIC = 1 n i = 1 n ( x i ( t ) y i ( t ) ) 2 1 n i = 1 n x i ( t ) 2 + 1 n i = 1 n y i ( t ) 2
where x i ( t ) represents the original values, y i ( t ) represents the predicted values, and n represents the number of observations. TIC ranges between 0 and 1, with values closer to 0 indicating better model accuracy.
The Index of Agreement (IA) is another indicator of model performance, providing a normalized measure of the match between observed and predicted values. The IA is defined as follows:
IA = 1 i = 1 n ( x i ( t ) y i ( t ) ) 2 i = 1 n | x i ( t ) x ¯ ( t ) | + | y i ( t ) x ¯ ( t ) | 2
where x ¯ ( t ) denotes the mean of the observed values. IA values range from 0 to 1, with values closer to 1 indicating better agreement between the predicted and observed data.
By combining the indicators MAE, RMSE, SMAPE, TIC, and IA, a comprehensive evaluation of the prediction method’s performance can be achieved.

4.3. Comparison of Different Prediction Methods

In this study, the dataset was divided into training and validation sets, with 80% of the samples used for the former and the remaining 20% for the latter. To stay on course, we applied several classic forecasting methods, including Informer [52], Autoformer [53], VMD-LSTM [23], and VMD-SSA-LSSVM [54]. Figure 10 shows the prediction results and the error of the three best-performing methods (Informer, Autoformer, and DFFormer). These prediction results are very close to the raw data, indicating that these methods have high accuracy and reliability in maintaining course prediction.
In order to select the optimal prediction method among these forecasting methods, the errors between the predicted values and the raw values are calculated, as shown in Figure 11 and Figure 12 and Table 2. The indicators in Table 2 were used to quantify the aforementioned results. Overall, DFFormer has the lowest MSE and MAE among all control variables and performs the best. In contrast, VMD-SSA-LSSVM performs the worst among all control variables, with the highest MSE and MAE. The performances of Informer, Autoformer, and VMD-LSTM fall between DFFormer and VMD-SSA-LSSVM, but are noticeably inferior to the excellent performance of DFFormer.
These results indicate that DFFormer demonstrates higher accuracy and stability when handling these control variables, clearly outperforming other methods. This superiority is attributed to improvements in the design and training process of the DFFormer model, which exhibits a better generalization ability and prediction accuracy when dealing with complex time-series data.
Furthermore, this article also tested the running time and computational resources of DFFormer and baseline models, as shown in Table 3. These metrics include training time, inference time per prediction, and computational resources, all measured on the same hardware configuration (NVIDIA RTX 3090 GPU, 32 GB RAM).
As shown in Table 3, although DFFormer requires more training time, runtime, and storage space compared to Informer, Autoformer, VMD-LSTM, and VMD-LSTM, it is worth noting that DFFormer has higher accuracy. It can be expected that with the development of computer technology, this disadvantage of running time and storage can be further reduced. The accuracy of DFFormer can be attributed to its architecture, which combines discrete wavelet transform (DWT) for signal decomposition and fast Fourier transform-based attention (FFT-Attention) for feature extraction. These methods effectively enhance the model’s ability to capture temporal features by focusing on the frequency domain, enabling DFFormer to achieve accurate predictions with stronger feature extraction capabilities. Based on the above tests, a conclusion can be drawn that DFFormer can achieve higher prediction accuracy with minimal running time and computational resources cost.

4.4. Comparative Analysis of Model Performance with Different Recursive Layers

Based on the aforementioned analysis, the DFFormer method demonstrates a superior performance compared to other methods. To further enhance the model’s predictive capabilities, we focused our attention on the recursive layer’s design. To assess its contribution to the DFFormer method, we conducted a series of experiments designed to analyze the recursive layer’s impact on the model’s overall predictive performance. By comparing models both with and without recursive layers, as well as those incorporating various types thereof (MLP, RNN, GRU, and LSTM), we extensively explored the performance of these structures in managing time-series data.
In the experiments, we constructed various model configurations to evaluate the effects of different recursive layers. Initially, the baseline model was modified by removing the recursive layer, establishing a comparison benchmark. Subsequently, we integrated MLP, RNN, GRU, and LSTM as distinct recursive layers within the model to process time-series data. We introduced these layers aiming to enhance the model’s capacity to capture temporal dependencies, thereby improving predictive performance. The experimental results are shown in Figure 13 and Figure 14 and Table 4.
The experimental results indicate that the selection of different recursive layers within the model significantly influences predictive performance. Specifically, when utilizing LSTM as the recursive layer, the model achieved the best performance across all evaluation metrics, highlighting its strengths in managing time-series data with long-term dependencies. In contrast, the performance of MLP exhibited a relatively weaker performance, suggesting its limited capacity to capture complex temporal dependencies. Additionally, the GRU demonstrated a commendable performance, effectively balancing model accuracy and computational efficiency in certain scenarios. Meanwhile, the RNN, despite its potential to enhance model performance, was marginally less effective.
Through these experiments, the significance of the recursive layer in the new method becomes evident. The LSTM’s primary advantage is its robust capacity to capture temporal dependencies, whereas the GRU maintains an effective equilibrium between performance and computational efficiency. Consequently, these results furnish compelling evidence for selecting the most appropriate recursive structure.
The effectiveness of DFFormer depends not only on prediction accuracy but also on computational efficiency, particularly with different recurrent layer configurations, such as LSTM, GRU, MLP, and RNN. Table 5 shows a comparison of computational resource consumption for DFFormer with different recurrent layers, including training time, inference time per prediction, and memory usage. All metrics were measured on the same hardware setup (NVIDIA RTX 3090 GPU, 32 GB RAM).
As shown in Table 5, the MLP configuration achieves the shortest training and inference times with the least memory usage, making it suitable for resource-constrained environments. However, the LSTM recurrent layer, while requiring more computational resources, provides a stronger sequence modeling capability, which enhances predictive accuracy. These comparisons indicate that DFFormer can effectively balance computational load and prediction accuracy, allowing users to choose the optimal configuration based on specific application requirements.
While DFFormer achieved high predictive accuracy and stability during trials in the South China Sea, its potential generalizability to other UG types and different marine environments warrants further discussion. Notably, the fundamental sensor configurations and operational mechanisms of various UGs are often quite similar. Many UG models, regardless of their specific design variations, use comparable sensors for measuring parameters like pitch, yaw, roll, and depth, as well as environmental variables such as temperature and salinity. These shared sensor and data configurations suggest that DFFormer’s core predictive algorithms can be feasibly adapted to other UG models, allowing it to generalize effectively across various UG types with minimal modification.
In terms of generalizability to different marine environments, while oceanographic conditions such as temperature, salinity, and current patterns do vary across seas, DFFormer’s design leverages a multi-scale feature extraction method that is particularly robust in capturing and adjusting for local variability. The integration of discrete wavelet transform (DWT) and fast Fourier transform-based attention (FFT-Attention) allows DFFormer to effectively model both high- and low-frequency characteristics in UG data. This dual approach enables DFFormer to capture complex, location-specific patterns, thus potentially enhancing its adaptability across different oceanic conditions.

5. Conclusions

This paper presents DFFormer, a novel Transformer-based framework designed to enhance control parameter prediction for underwater gliders (UGs) by leveraging advanced signal processing and neural network techniques. DFFormer utilizes discrete wavelet transform (DWT) and fast Fourier transform-based attention (FFT-Attention) to effectively analyze the frequency- and time-domain features critical for UG operation. Its Transformer-based architecture processes signals through multiple parallel pathways, significantly improving its ability to manage complex control parameters, particularly in precisely adjusting rudder angles. Extensive testing with actual sea trial data demonstrates that DFFormer outperforms traditional predictive methods in both accuracy and computational efficiency. Furthermore, the integration of recursive layers, such as multi-layer perceptron (MLP), recurrent neural network (RNN), long short-term memory (LSTM), and gated recurrent unit (GRU), enhances the model’s feature extraction capabilities, enabling the effective capture of detailed inference features. The inclusion of a fusion block for integrating these extracted features further improves the robustness and reliability of DFFormer’s predictions. The use of the Huber loss function refines the neural network structure, providing robustness against outliers and increasing precision. Overall, this research advances UG control parameter prediction, offering valuable insights into the application of machine learning in autonomous systems. DFFormer improves course-keeping and control efficiency, laying a strong foundation for future developments in UG control technology.

Author Contributions

Conceptualization, Y.J. and J.A.; methodology, Y.J. and C.M.; software, W.G.; validation, W.G., Z.L. and X.S.; formal analysis, J.A. and Y.J.; investigation, W.G.; resources, J.A. and C.M.; data curation, Z.L.; writing—original draft preparation, Y.J.; writing—review and editing, W.G. and C.M.; visualization, J.A.; supervision, W.G. and C.M.; project administration, Y.J.; funding acquisition, X.S. and C.M. All authors have read and agreed to the published version of the manuscript.

Funding

This work is jointly supported by the National Key R&D Program of China, the National Natural Science Foundation of China under grant nos. 62373278 and 52005365, and the Natural Science Foundation of Tianjin, China, under grant no. 21JCJQJC00130.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data are available from the corresponding authors upon reasonable request.

Acknowledgments

The authors would like to express their sincere thanks to Wei Ma for providing valuable support with the sea trial data.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Joshi, B.; Xanthidis, M.; Roznere, M.; Burgdorfer, N.J.; Mordohai, P.; Li, A.Q.; Rekleitis, I. Underwater Exploration and Mapping. In Proceedings of the 2022 IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), Singapore, 19–21 September 2022; pp. 1–7. [Google Scholar]
  2. Mirza, J.; Kanwal, F.; Salaria, U.A.; Ghafoor, S.; Aziz, I.; Atieh, A.; Almogren, A.; Haq, A.U.; Kanwal, B. Underwater temperature and pressure monitoring for deep-sea SCUBA divers using optical techniques. Front. Phys. 2024, 12, 1417293. [Google Scholar] [CrossRef]
  3. Siregar, S.; Trilaksono, B.R.; Hidayat, E.M.I.; Kartidjo, M.; Habibullah, N.; Zulkarnain, M.F.; Setiawan, H.N. Design and Construction of Hybrid Autonomous Underwater Glider for Underwater Research. Robotics 2023, 12, 8. [Google Scholar] [CrossRef]
  4. Li, Z.; Wen, J.; Yang, J.; He, J.; Ni, T.; Li, Y. Energy-Efficient Space–Air–Ground–Ocean-Integrated Network Based on Intelligent Autonomous Underwater Glider. IEEE Internet Things J. 2022, 10, 9329–9341. [Google Scholar] [CrossRef]
  5. Wagawa, T.; Kawaguchi, Y.; Igeta, Y.; Honda, N.; Okunishi, T.; Yabe, I. Observations of oceanic fronts and water-mass properties in the central Japan Sea: Repeated surveys from an underwater glider. J. Mar. Syst. 2020, 201, 103242. [Google Scholar] [CrossRef]
  6. Wu, Q.; Wu, H.; Jiang, Z.; Tan, L.; Yang, Y.; Yan, S. Multi-objective optimization and driving mechanism design for controllable wings of underwater gliders. Ocean Eng. 2023, 286, 115534. [Google Scholar] [CrossRef]
  7. Liu, F. System Design and Motion Behaviors Analysis of the Hybrid Underwater Glider. Ph.D. Dissertation, Tianjin University, Tianjin, China, 2014. [Google Scholar]
  8. Xie, Y.; Wang, W.; Zhang, L.; Yang, X. Trajectory prediction of autonomous underwater vehicles using a decision tree approach. Ocean Eng. 2017, 137, 1–12. [Google Scholar]
  9. Smith, R.; Zhang, Q.; Sun, Y. Machine learning for predicting the performance of underwater gliders. IEEE Trans. Neural Netw. Learn. Syst. 2016, 27, 2477–2486. [Google Scholar]
  10. Zhang, R.; Niu, W.; Wan, X.; Wu, Y.; Xue, D.; Yang, S. Surfacing Positioning Point Prediction of Underwater Glider with a New Combination Model. J. Mar. Sci. Eng. 2023, 11, 972. [Google Scholar] [CrossRef]
  11. Jiang, Z.; Li, J.; Wang, W. Kalman filter-based estimation of underwater glider positions and velocities. IEEE J. Ocean. Eng. 2018, 43, 752–761. [Google Scholar]
  12. Wang, J.; Wu, Z.; Dong, H.; Tan, M.; Yu, J. Development and Control of Underwater Gliding Robots: A Review. IEEE/CAA J. Autom. Sin. 2022, 9, 105671. [Google Scholar] [CrossRef]
  13. Zhao, C.; Li, J.; Zhang, X. Ensemble learning for underwater glider performance prediction. IEEE Access 2019, 7, 130077–130086. [Google Scholar]
  14. Tan, Z.; Wang, W.; Sun, Y. Adaptive path planning for underwater gliders using reinforcement learning. IEEE Trans. Cybern. 2020, 50, 1960–1972. [Google Scholar]
  15. Chen, Y.; Zhang, X.; Sun, Y. Real-time obstacle avoidance for underwater gliders using deep reinforcement learning. Appl. Ocean Res. 2021, 105, 102456. [Google Scholar]
  16. Yu, Z.; Li, J.; Wang, W. Reinforcement learning-based path planning for underwater gliders. IEEE Access 2019, 7, 109211–109224. [Google Scholar]
  17. Yang, H.; Ma, J. Nonlinear Control for Autonomous Underwater Glider Motion Based on Inverse System Method. J. Shanghai Jiaotong Univ. (Sci.) 2010, 15, 713–718. [Google Scholar] [CrossRef]
  18. Joo, M.G.; Qu, Z. An Autonomous Underwater Vehicle as an Underwater Glider and Its Depth Control. Int. J. Control Autom. Syst. 2015, 13, 1212–1220. [Google Scholar] [CrossRef]
  19. La, S.; Ko, S.; Ji, D.; Kim, H.; Kim, C. Development of Control Method for Improving Energy Efficiency of Unmanned Underwater Gliders. J. Adv. Navig. Technol. 2022, 26, 105–112. [Google Scholar]
  20. Wang, Y.; Zhang, L.; Liang, Y.; Yang, M.; Yang, S.; Niu, W. Sensitivity Analysis of Profile Navigation Command of Underwater Gliders to the Initial Heading Error for Improving Trajectory Accuracy. Ocean Eng. 2024, 300, 117478. [Google Scholar] [CrossRef]
  21. Shan, Y.; Yan, Z.; Wang, J. Model Predictive Control of Underwater Gliders Based on a One-Layer Recurrent Neural Network. In Proceedings of the 2013 Sixth International Conference on Advanced Computational Intelligence (ICACI), Hangzhou, China, 19–21 October 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 328–333. [Google Scholar]
  22. Isa, K.; Arshad, M.R.; Ishak, S. A Hybrid-Driven Underwater Glider Model, Hydrodynamics Estimation, and an Analysis of the Motion Control. Ocean Eng. 2014, 81, 111–129. [Google Scholar] [CrossRef]
  23. Wang, C.; Wang, Y.; Zhang, R.; Niu, W. Analysis of Roll Center Compensation Method for Underwater Gliders Based on Deep Learning. Ocean Eng. 2022, 246, 110529. [Google Scholar] [CrossRef]
  24. Immas, A.; Do, N.; Alam, M.-R. Real-time in situ prediction of ocean currents. Ocean Eng. 2021, 228, 108922. [Google Scholar] [CrossRef]
  25. Mu, X.; He, B.; Zhang, X.; Song, Y.; Shen, Y.; Feng, C. End-to-end navigation for Autonomous Underwater Vehicle with Hybrid Recurrent Neural Networks. Ocean Eng. 2019, 194, 106602. [Google Scholar] [CrossRef]
  26. Liu, Y.; Ning, C.; Zhang, Q.; Yuan, G.; Li, C. Utilizing VMD and BiGRU to predict the short-term motion of buoys. Ocean Eng. 2024, 313, 119237. [Google Scholar] [CrossRef]
  27. Hou, L.; Wang, X.; Sun, H.; Sun, Y.; Wei, Z. A long sequence time-series forecasting model for ship motion attitude based on informer. Ocean Eng. 2024, 305, 117861. [Google Scholar] [CrossRef]
  28. Jiang, C.; Xiang, X.; Xiang, G. A joint multi-model machine learning prediction approach based on confidence for ship stability. Complex Intell. Syst. 2024, 10, 3873–3890. [Google Scholar] [CrossRef]
  29. Xu, Q.; Zhang, X.; Sun, Y. Data assimilation for oceanographic forecasts using machine learning. J. Atmos. Ocean. Technol. 2021, 38, 573–586. [Google Scholar]
  30. Evensen, G. Data Assimilation: The Ensemble Kalman Filter; Springer: Berlin/Heidelberg, Germany, 2009. [Google Scholar]
  31. Lewis, J.M.; Lakshmivarahan, S.; Dhall, S. Dynamic Data Assimilation: A Least Squares Approach; Cambridge University Press: Cambridge, UK, 2019. [Google Scholar]
  32. Marini, S.; Gjeci, N.; Govindaraj, S.; But, A.; Sportich, B.; Ottaviani, E.; Marquez, F.P.G.; Bernalte Sanchez, P.J.; Pedersen, J.; Clausen, C.V.; et al. ENDURUNS: An Integrated and Flexible Approach for Seabed Survey Through Autonomous Mobile Vehicles. J. Mar. Sci. Eng. 2020, 8, 633. [Google Scholar] [CrossRef]
  33. Ullah, B.; Ovinis, M.; Baharom, M.B.; Ali, S.S.A.; Khan, B.; Javaid, M.Y. Effect of Waves and Current on Motion Control of Underwater Gliders. Auton. Robots 2020, 25, 549–562. [Google Scholar] [CrossRef]
  34. Jiang, C.; Li, J.; Xu, W.; Feng, W. Improvement of the Position Estimation for Underwater Gliders with a Passive Acoustic Method. IEEE J. Ocean. Eng. 2021, 46, 1015–1028. [Google Scholar] [CrossRef]
  35. Zhang, T.; Zheng, X.-Q.; Liu, M.-X. Multiscale attention-based LSTM for ship motion prediction. Ocean Eng. 2021, 230, 109066. [Google Scholar] [CrossRef]
  36. Xun, S.; Zhu, P.; Yang, B.; Xiong, J. Multi-direction prediction based on SALSTM model for ship motion. In Proceedings of the 4th International Conference on Information Science, Electrical and Automation Engineering, Wuhan, China, 24–26 March 2023. [Google Scholar]
  37. Tian, X.; Song, Y. Machine Learning for Short-Term Prediction of Ship Motion Combined with Wave Input. Appl. Sci. 2023, 13, 5298. [Google Scholar] [CrossRef]
  38. Wang, Y.; Dou, Y.; Yang, Z.; Fu, H. Ship pitch prediction method based on LSTMC and multi-head attention. Ocean Eng. 2024, 309, 118236. [Google Scholar]
  39. Liu, J.; Yang, J.; Liu, K.; Xu, L. Ocean Current Prediction Using the Weighted Pure Attention Mechanism. J. Mar. Sci. Eng. 2022, 10, 592. [Google Scholar] [CrossRef]
  40. Xu, D.; Yin, J. An enhanced hybrid scheme for ship roll prediction using support vector regression and TVF-EMD. Ocean Eng. 2024, 307, 117951. [Google Scholar] [CrossRef]
  41. Wang, Y.; Dou, Y.; Peng, C.; Gu, Z.; Fu, H. Multi-step prediction method of ship pitch based on Conv-Informer model. In Proceedings of the OCEANS 2023—Limerick, Limerick, Ireland, 5–8 June 2023; pp. 1–6. [Google Scholar]
  42. Yang, M.; Zhao, L.; Wang, Y.; Liang, Y.; Zhao, X.; Yang, S. Motion Condition Monitoring of Underwater Gliders Based on Deep Learning and Dynamic Identification. J. Mar. Sci. Eng. 2023, 272, 113793. [Google Scholar] [CrossRef]
  43. Lei, L.; Tang, T.; Gang, Y.; Jing, G. Hierarchical Neural Network-Based Hydrological Perception Model for Underwater Glider. Ocean Eng. 2022, 260, 112101. [Google Scholar] [CrossRef]
  44. Daubechies, I. Orthonormal Bases of Compactly Supported Wavelets. Commun. Pure Appl. Math. 1988, 41, 909–996. [Google Scholar] [CrossRef]
  45. Wang, Z.; Zhao, Y.; Chen, J. Multi-Scale Fast Fourier Transform Based Attention Network for Remote-Sensing Image Super-Resolution. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 2728–2740. [Google Scholar]
  46. Elman, J.L. Finding Structure in Time. Cogn. Sci. 1990, 14, 179–211. [Google Scholar] [CrossRef]
  47. Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
  48. Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 25–29 October 2014; Moschitti, A., Pang, B., Daelemans, W., Eds.; Association for Computational Linguistics: Stroudsburg, PA, USA, 2014; pp. 1724–1734. [Google Scholar]
  49. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
  50. Zhang, X.; Li, Y.; Wang, Z. LSTM networks for dynamic system prediction. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 2097–2108. [Google Scholar]
  51. Hu, Y.; Zhang, W.; Sun, Y. Time-series prediction using LSTM networks for autonomous underwater gliders. Ocean Eng. 2021, 237, 109671. [Google Scholar]
  52. Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, X.; Zhang, W.; Li, H. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Association for the Advancement of Artificial Intelligence: Palo Alto, CA, USA, 2021; pp. 11106–11115. [Google Scholar]
  53. Wu, H.; Xu, J.; Wang, J.; Long, M.; Jiang, J.; Li, C.; Zong, B. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Online, 6–14 December 2021; Association for Computational Linguistics: San Diego, CA, USA, 2021; pp. 22419–22430. [Google Scholar]
  54. Hu, H.; Zhou, Y.; Peng, X. Prediction of Underwater Glider Depth-Averaged Current Velocities Based on VMD-SSA-LSSVM. In Proceedings of 2021 International Conference on Autonomous Unmanned Systems (ICAUS 2021); Springer: Singapore, 2022; pp. 1976–1985. [Google Scholar]
Figure 1. Petrel-L underwater glider.
Figure 1. Petrel-L underwater glider.
Jmse 12 02106 g001
Figure 2. Operational phases of an underwater glider.
Figure 2. Operational phases of an underwater glider.
Jmse 12 02106 g002
Figure 3. Relationship diagram of the inertial, body, and velocity frames.
Figure 3. Relationship diagram of the inertial, body, and velocity frames.
Jmse 12 02106 g003
Figure 4. Schematic diagram of the model structure.
Figure 4. Schematic diagram of the model structure.
Jmse 12 02106 g004
Figure 5. Diagram of the structure of an RNN.
Figure 5. Diagram of the structure of an RNN.
Jmse 12 02106 g005
Figure 6. Diagram of the structure of an LSTM.
Figure 6. Diagram of the structure of an LSTM.
Jmse 12 02106 g006
Figure 7. Diagram of the structure of a GRU.
Figure 7. Diagram of the structure of a GRU.
Jmse 12 02106 g007
Figure 8. The sea trial area of the Petrel-L underwater glider.
Figure 8. The sea trial area of the Petrel-L underwater glider.
Jmse 12 02106 g008
Figure 9. Changes in rudder angle and depth over time.
Figure 9. Changes in rudder angle and depth over time.
Jmse 12 02106 g009
Figure 10. Prediction results with three different methods.
Figure 10. Prediction results with three different methods.
Jmse 12 02106 g010
Figure 11. The evaluation indicators of different methods in the diving phase.
Figure 11. The evaluation indicators of different methods in the diving phase.
Jmse 12 02106 g011
Figure 12. The evaluation indicators of different methods in the climbing phase.
Figure 12. The evaluation indicators of different methods in the climbing phase.
Jmse 12 02106 g012
Figure 13. The evaluation indicators of different recursive layers in the diving phase.
Figure 13. The evaluation indicators of different recursive layers in the diving phase.
Jmse 12 02106 g013
Figure 14. The evaluation indicators of different recursive layers in the climbing phase.
Figure 14. The evaluation indicators of different recursive layers in the climbing phase.
Jmse 12 02106 g014
Table 2. Prediction errors of different methods during the diving and climbing phases.
Table 2. Prediction errors of different methods during the diving and climbing phases.
PhaseMethodMAERMSESMAPETICIA
DivingInformer1.33512.23640.13950.23250.8745
 Autoformer0.94231.36510.07120.11230.9234
 VMD-SSA-LSSVM1.86573.96510.26540.26140.8642
 VMD-LSTM1.13422.43620.18620.23510.8623
 DFFormer0.72031.12350.03780.07120.9321
ClimbingInformer1.24512.26510.37510.12640.8431
 Autoformer1.32412.89620.32410.19230.9131
 VMD-SSA-LSSVM1.85423.13210.20740.23410.8542
 VMD-LSTM1.53142.86210.26420.17260.8862
 DFFormer0.83201.23680.04230.08230.9265
Table 3. Comparison of computational resources required by DFFormer and baseline models.
Table 3. Comparison of computational resources required by DFFormer and baseline models.
ModelTraining Time (s)Inference Time per Prediction (ms)Memory Usage (MB)
DFFormer76235.53560
Informer56826.32511
Autoformer68129.92956
VMD-SSA-LSSVM73234.21998
VMD-LSTM53924.71531
Table 4. Prediction errors for different recursive layers in the proposed method during the diving and climbing phases.
Table 4. Prediction errors for different recursive layers in the proposed method during the diving and climbing phases.
PhaseRecurrent LayerMAERMSESMAPETICIA
DivingNone1.84533.35620.25950.18250.7945
 MLP1.32312.78610.13870.13240.8546
 RNN1.19243.56210.12890.17210.8123
 GRU1.12232.43620.09750.11230.8934
 LSTM0.96542.12340.07950.09210.9121
ClimbingNone1.94533.45620.26950.19250.7845
 MLP1.42312.88610.14870.14240.8446
 RNN1.29243.66210.13890.18210.8023
 GRU1.22232.53620.10750.12230.8834
 LSTM1.11452.24630.10710.06320.9287
Table 5. Computational resource comparison for DFFormer with different recurrent layers.
Table 5. Computational resource comparison for DFFormer with different recurrent layers.
Recurrent Layer TypeTraining Time (s)Inference Time per Prediction (ms)Memory Usage (MB)
MLP13259.82340
RNN156312.12475
GRU148910.72395
LSTM167813.42585
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Jing, Y.; Ma, C.; An, J.; Li, Z.; Sun, X.; Guo, W. Nonstationary Control Parameter Prediction for Underwater Gliders Utilizing an Innovative Transformer-Based Model. J. Mar. Sci. Eng. 2024, 12, 2106. https://doi.org/10.3390/jmse12112106

AMA Style

Jing Y, Ma C, An J, Li Z, Sun X, Guo W. Nonstationary Control Parameter Prediction for Underwater Gliders Utilizing an Innovative Transformer-Based Model. Journal of Marine Science and Engineering. 2024; 12(11):2106. https://doi.org/10.3390/jmse12112106

Chicago/Turabian Style

Jing, Yv, Chao Ma, Jianpeng An, Zezhong Li, Xinlin Sun, and Wei Guo. 2024. "Nonstationary Control Parameter Prediction for Underwater Gliders Utilizing an Innovative Transformer-Based Model" Journal of Marine Science and Engineering 12, no. 11: 2106. https://doi.org/10.3390/jmse12112106

APA Style

Jing, Y., Ma, C., An, J., Li, Z., Sun, X., & Guo, W. (2024). Nonstationary Control Parameter Prediction for Underwater Gliders Utilizing an Innovative Transformer-Based Model. Journal of Marine Science and Engineering, 12(11), 2106. https://doi.org/10.3390/jmse12112106

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop