DAPNet: A Dual-Attention Parallel Network for the Prediction of Ship Fuel Consumption Based on Multi-Source Data

Abstract

The precise prediction of ship fuel consumption (SFC) not only serves to enhance energy efficiency to benefit shipping enterprises but also to provide quantitative foundations to aid in carbon emission reduction and ecological environment protection. On the other hand, SFC-related data represent typical multi-source characteristics and heterogeneous features, which lead to several methodological issues (e.g., feature alignment and feature fusion) in SFC prediction. Therefore, this paper proposes a dual-attention parallel network named DAPNet to solve the above issues. Firstly, we design a parallel network structure containing two kinds of long short-term memory (LSTM) and improved temporal convolutional networks (TCNs) for time-series analysis tasks so that different source data can be applied to suitable networks. Secondly, a local attention mechanism is included in each single parallel network so as to improve the ability of feature alignment from different-scale training data. Finally, global attention is employed for the fusion of all parallel networks, which can enrich representation features and simultaneously enhance the performance of SFC prediction. In experiments, DAPNet is compared with 10 methods, including baseline and attention models. The comparison results show that DAPNet and several of its variants obtain the highest accuracy in SFC prediction.

1. Introduction

The shipping industry plays an indispensable role in global trade logistics, with over 80% of international trade reliant on maritime transport [1]. With the continuous increase in shipping trade, the fuel consumption and carbon emissions of ships have attracted strong attention in recent years [2]. From the perspective of environmental protection, reducing ship fuel consumption (SFC) leads to energy savings and emission reductions. From the perspective of shipping enterprise operations, the cost associated with SFC constitutes a substantial proportion of the total operational expenses [3]. Consequently, the energy reduction of SFC plays a pivotal role in many fields, such as shipping management, route optimization, and path planning [4]. There is also an urgent need for accurate and efficient SFC prediction, which can provide discovery and decision support for researchers and managers [5].

SFC is predicted by using classic time-series analysis and multivariate regression, which usually demonstrate the effects of explanatory factors on fuel consumption [6]. Fan et al. used theoretical formulas and computational fluid dynamics analysis methods to construct a ship energy efficiency model under the influence of calm water resistance [7]. Yang et al. considered the effect of ocean currents on ship speed and reduced SFC via its optimization [8]. However, there still exists gap between prediction precision and actual requirement, especially during dynamic and diverse environments [9]. As a time-series analysis problem, Zhu et al. employed classic methods such as support vector regression (SVR) and artificial neural networks (ANNs) to train models for SFC prediction. A comparative analysis revealed that effective feature fusion with ANNs could improve the precision of SFC prediction [10]. The analysis suggested that autoregressive models have limitations in terms of identifying model parameters and capturing the nonlinear characteristics of sequences [11]. The multiple linear regression method was also employed to establish an SFC prediction model, and its effectiveness was validated using real-world data from bulk carriers [12].

In recent years, the feature fusion of multi-source data has emerged as both a challenging and essential process, particularly when applying traditional machine learning methods to SFC prediction. Han et al. proposed several approaches for the fusion of multi-source data, and they employed the long short-term memory (LSTM) method for SFC prediction [13]. A comparative analysis revealed that effective feature fusion could improve the precision of SFC prediction. Furthermore, several advanced models, e.g., bi-directional LSTM (BiLSTM) and LSTM with a self-attention mechanism (LA), are begun to be used for SFC prediction and obtain better prediction results [14,15]. However, limitations remain, as well-processed data, e.g., data alignment, association, and fusion, are still required by the methods in existing studies [16].

To address the above issues, this paper proposes a dual-attention parallel network named DAPNet. Firstly, DAPNet introduces a parallel network structure to pursue multi-source data, and transforms the multivariate regression into parallel process based on each independent network the multivariate regression task as a parallel process of an independent network. Secondly, multi-source data also represent heterogeneous features, which require an appropriate network. In DAPNet, two kinds of LSTM and an improved temporal convolutional network (TCN) are applied to feature extraction, where LSTM focuses on capturing temporal correlations and the improved TCN focuses on the dependent relationships among different variables [17,18]. Finally, DAPNet includes dual attention for feature alignment and feature fusion. Local attention is included in each single parallel network to enhance temporal feature extraction from individual factors in the same dimension. Global attention is included for the feature fusion of all parallel networks so as to enrich the dynamic diversity of representation features. In experiments, a case study of a ferry ship used as a reference ship is conducted, and it is equipped with many data sensors for the collection of multi-source data, including both internal and environmental factors. DAPNet is compared with 10 methods, including a traditional and advanced RNN and LSTM and state-of-the-art attention models. The experimental results reveal that DAPNet and several of its variants obtained a lower training loss and a higher prediction accuracy than other comparison models, with DAPNet achieving minimum values of 0.0065 and 0.0108 for the mean absolute error (MAE) and root mean square error (RMSE) in SFC prediction. The proposed DAPNet presents three main contributions accounting for an end-to-end framework with a parallel network, local attention, and global attention, which solve multivariable regression, multi-source feature alignment, and heterogeneous feature fusion.

The structure of this paper is organized as follows. In Section 2, we provide a detailed introduction to the proposed DAPNet. A data description of the case ship and comparative analyses between baseline models and DAPNet are presented in Section 3. Finally, in Section 4, the conclusions are summarized and future prospects are outlined.

2. Methodology Design of DAPNet

2.1. Overview of DAPNet for SFC Prediction

DAPNet, depicted in Figure 1, is based on an end-to-end framework that encompasses variable selection, data alignment, feature extraction, feature alignment, feature fusion, and final prediction. DAPNet features a parallel structure comprising LSTM and an improved TCN, each tailored for specific variable inputs and data alignment. Variable specialty decides the candidate network to use LSTM or the improved TCN. In each parallel network, the local attention is included for feature extraction and alignment. Global attention is used for the feature fusion of all parallel networks. The representation features are then used for final prediction through a fully connected neural network (FCNN). Detailed descriptions of the parallel network and the dual-attention mechanism are provided in the following sections.

2.2. Design of Parallel Network

2.2.1. Variable Selection and Separation

Numerous sensors are initially equipped on the ship to measure the ship navigation status, such as speed and pitch. As shown in Figure 2, these types of data represent continuous, contextual, and sequential characteristics. However, variance and volatility are still contained in the distribution of these variables, with most exhibiting a time-series trend. Therefore, this paper designs a parallel network based on LSTM to accept such variables.

Several sensors are additionally equipped on the ship to measure the external status and environmental factors, such as trim, draft, head wind, and cross wind. However, these data also follow time-series patterns and exhibit high variance and significant volatility, as shown in Figure 3. Notably, environmental factors simultaneously reveal spatio-temporal features. Consequently, this paper designs a parallel network based on a TCN to accept such variables.

In addition to the aforementioned measured data used as the input of model, the distribution of the actual SFC data is presented in Figure 4.

2.2.2. Data Alignment Based on Framing

The differing sampling frequencies of multi-source sensors result in the presence of heterogeneous data that necessitate synchronization within the temporal domain. According to an existing study [13], framing is still one of the most efficient approaches for data alignment from different-scaled data. In this paper, we also incorporate a mobile overlapping frame approach to address the challenges posed by heterogeneous SFC data. The specific process of the mobile overlapping frame can be represented as follows:

$$m={\displaystyle \frac{t_e^k-t_s^k-l}{\left(1-r\right)\times l}}$$

(1)

$${\mathcal{D}}_i^k=\left\{d_t^k|t\in \left(t_{i,s}^k,t_{i,e}^k\right)\right\}$$

(2)

$$t_{i,s}^k=t_s^k+\left(i-1\right)\times \left(1-r\right)\times l$$

(3)

$$t_{i,e}^k=t_s^k+\left(i-i\times r+r\right)\times l$$

(4)

where $m$ represents the number of frames; $l$ and $r$ denote the frame length and overlap ratio, respectively; $t_s^k$ and $t_e^k$ represent the start and end points in time for the $k$th sensor; and ${\mathcal{D}}_i^k$ signifies the collection of data $d_t^k$ for the $i$th frame within the $k$th sensor, where $t_{i,s}^k$ and $t_{i,e}^k$ correspondingly indicate the temporal start and end points within the $i$th frame of the $k$th sensor. Data from each sensor with varying sampling frequencies can be aligned after framing. For instance, Figure 5 illustrates the data acquisition frequencies of head wind and port draft over the same time duration, along with their respective framing procedures.

Figure 5 additionally illustrates both the frame length and overlap ratio. Fine-tuning the parameters of frame size and overlap size can significantly benefit the performance of the prediction model. Following multiple experiments, the selected frame length and overlap ratio were set to 30 s and 60%, respectively. Ultimately, the input matrix is constructed by extracting the statistical feature of the mean value of the data within each frame, as shown by the green dot in Figure 5.

$$\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_i^k\right)={\displaystyle \frac{\sum _{t\in \left(t_{i,s}^k,t_{i,e}^k\right)}{d}_t^k}{m_i^k}}$$

(5)

where $\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_i^k\right)$ denotes the average value of the data within ${\mathcal{D}}_i^k$, and $m_i^k$ represents the number of data points in ${\mathcal{D}}_i^k$. After the framing and extraction of statistical features, the data structure of the SFC variable $X^{\mathrm{a}\mathrm{v}\mathrm{g}}$ is as described in Equation (6), where $q$ represents the total number of SFC features contained in $X^{\mathrm{a}\mathrm{v}\mathrm{g}}$.

$$X^{\mathrm{a}\mathrm{v}\mathrm{g}}=\left[\begin{array}{ccccc}\mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_1^1\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_1^k\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_1^q\right)\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_i^1\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_i^k\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_i^q\right)\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_m^1\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_m^k\right)& \cdots & \mathrm{a}\mathrm{v}\mathrm{g}\left({\mathcal{D}}_m^q\right)\end{array}\right]$$

(6)

2.2.3. Design of Parallel Network Based on LSTM

Basic LSTM is proposed in [19], which has been widely adopted for time-series regression and contextual prediction tasks. In most situations, variables with the same dimension can be directly inputted into LSTM. In this paper, the selected variables are sampled by different frequencies. Accordingly, DAPNet provides a parallel network of LSTM for training each variable separately, as shown in Figure 6. Furthermore, the detailed internal structure of the LSTM is shown in Figure 7.

Notation $L_i$ denotes the size of cells in the LSTM network. For different variables, each individual LSTM network can use different sizes of cells. The final hidden state $h_i\left(t\right)$ in a signal network can be calculated as follows:

$$h_i\left(t\right)={\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}}_i\left(h_i\left(t-1\right),c_i\left(t\right)\right)$$

(7)

where $h_i\left(t-1\right)$ and $c_i\left(t\right)$ denote the previous hidden state and current cell state, $h_i\left(t\right)$ is the output of the LSTM part in DAPNet, and $i$ denotes the $i$th LSTM in the parallel network structure. In DAPNet, other variants such as gated recurrent unit (GRU) [20], BiLSTM [21], and LSTM embedded with the quaternion ship domain (QSD-LSTM) [22] can replace LSTM and can also be mixed in a parallel network.

2.2.4. Design of Parallel Network Based on TCN

Convolutional neural networks (CNNs) exhibit inherent attributes conducive to enhancing nonlinear fitting capabilities, whereas SFC data are fundamentally characterized by temporal features. As an advanced variant of CNN tailored to address spatio-temporal modeling challenges, a TCN holds considerable potential to further enhance model performance [23]. In this context, the TCN is incorporated into the proposed DAPNet as another type of parallel network. To be different from LSTM, the TCN can accept multivariate inputs provided they share the same dimension. Consequently, both the classic TCN and an improved TCN are included, as shown in Figure 8. The structure of Figure 8 represents the encoder structure of the improved TCN, which used an $m\times k$ convolutional kernel.

For multivariate inputs, hidden states ${(h_{11}\left(t\right),h_{21}\left(t\right))}^T$ of the first layer can be calculated as follows:

$${\left(h_{11}\left(t\right),h_{21}\left(t\right)\right)}^T=w_1\ast {\left(\left[\begin{array}{ccc}{x}_1\left(t-2\right)& x_1\left(t-1\right)& x_1\left(t\right)\\ x_2\left(t-2\right)& x_2\left(t-1\right)& x_2\left(t\right)\end{array}\right],\left[\begin{array}{ccc}{x}_2\left(t-2\right)& x_2\left(t-1\right)& x_2\left(t\right)\\ x_3\left(t-2\right)& x_3\left(t-1\right)& x_3\left(t\right)\end{array}\right]\right)}^T$$

(8)

where $w_1$ denotes the convolutional kernels in the first hidden layer. The improved TCN incorporates a temporal pyramid structure in the initial convolutional processing and represents the same dilated convolutional processing as a traditional TCN. The parallel network architecture of the TCN is illustrated in Figure 9.

2.3. Design of Dual Attention

Since the attention mechanism was introduced in [24], it has been widely applied across various networks (e.g., CNN and LSTM) and research fields (e.g., trajectory prediction and action recognition) [25,26,27,28]. In most attention models, a set of queries, keys, and value vectors is obtained from the inputs, with a key–value pair used to reassign weights of each value based on a compatibility function. In current studies, the attention mechanism has become one of the indispensable components, and can also be improved for several new functions, such as channel and spatial attentions and local and global attentions [29]. In DAPNet, local attention is incorporated within in each parallel network to enhance feature extraction, while global attention is applied across all parallel networks to enrich feature fusion.

2.3.1. Local Attention for Feature Extraction and Alignment

According to Figure 6 and Figure 8, both the LSTM and TCN output several final hidden states from the input layer through multiple hidden layers. The LSTM has the ability to extract features from both long-term and short-term memories, while the TCN can also capture temporal–spatial features based on dilated convolutional layers. Gradient vanishing is still a specific issue; however, gradient vanishing is different in the LSTM than TCN. In DAPNet, an attention mechanism is designed in each single parallel network so that the final hidden states depending on the independent LSTM or TCN can be recalculated and reallocated to obtain a new score based on local attention (see Figure 10).

Figure 10a shows the local attention in the LSTM. For each LSTM cell, the hidden state $h\left(i\right)$ is transformed into $(q_i,k_i,v_i)$, as follows:

$$q_i=w_q^{L}h\left(i\right)$$

(9)

$$k_i=w_k^{L}h\left(i\right)$$

(10)

$$v_i=w_v^{L}h\left(i\right)$$

(11)

where $w_q^L$, $w_k^L$, and $w_v^L$ denote the transformation weights of the query, key, and, value in local attention. Then, relevant relationships among new weights $a_i$ can be calculated by

$$a_i=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(s\left(q_i,k_i\right)\right)={\displaystyle \frac{exp\left(s\left(q_i,k_i\right)\right)}{{\sum }_{j=1}^{t}exp\left(s\left(q_i,k_j\right)\right)}}$$

(12)

$s(q_i,k_i)$ denotes the dot-product of the query and key as

$$s\left(q_i,k_i\right)={\displaystyle \frac{q_i·k_i}{\sqrt{D_k}}}$$

(13)

where $D_k$ denotes the key dimension. After calculating the relevance among all inputs via softmax, new hidden state $h^{\prime }\left(i\right)$ can be obtained by summation of attention scores as follows:

$$h^{\prime }\left(i\right)={\sum }_{i=1}^t{a}_i{v}_i$$

(14)

Figure 10b shows the local attention in the TCN, and the new hidden state $h\prime \left(i\right)$ can similarly be obtained based on the calculation methods corresponding to Equations (9)–(14).

2.3.2. Global Attention for Feature Fusion

Following feature extraction, DAPNet designs a global attention for the fusion of all outputs from parallel networks so that the diversity of local features can be enriched to generate representative features. Figure 11 shows the global attention in DAPNet. Global attention would recalculate and reallocate of all the representation hidden states $(h_1^{\prime }\left(t\right),\dots ,h_n^{\prime }\left(t\right))$ obtained by local attention from $x\left(t\right)$.

$$h_i^{\prime }\left(t\right)={\mathrm{L}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{A}\mathrm{t}\mathrm{t}}^i\left(x\left(t\right)\right)$$

(15)

where ${\mathrm{L}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{A}\mathrm{t}\mathrm{t}}^i(·)$ denotes the local attention score of the i-th parallel network. For each $h_i^{\prime }\left(t\right)$, $(q_t^i,k_t^i,v_t^i)$ are calculated as follows.

$$q_t^i=w_q^G{h}_i^{\prime }\left(t\right)$$

(16)

$$k_t^i=w_k^G{h}_i^{\prime }\left(t\right)$$

(17)

$$v_t^i=w_v^G{h}_i^{\prime }\left(t\right)$$

(18)

where $w_q^G$, $w_k^G$, and $w_v^G$ denote the transformation weights of the query, key, and value in global attention. Then, relevant relationships among new weights $a^i$ can be calculated by

$$a_t^i=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(s\left(q_t^i,k_t^i\right)\right)={\displaystyle \frac{exp\left(s\left(q_t^i,k_t^i\right)\right)}{{\sum }_{j=1}^{n}exp\left(s\left(q_t^j,k_t^j\right)\right)}}$$

(19)

where $n$ denotes the number of parallel networks. The global attention score ${\widehat{h}}^i\left(t\right)$ can be obtained from the summation of all combinations of $a_t^i{v}_t^i$ as follows:

$$\widehat{h}\left(t\right)=\mathrm{G}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}\mathrm{A}\mathrm{t}\mathrm{t}\left(h_1^{\prime }\left(t\right),\dots ,h_n^{\prime }\left(t\right)\right)={\sum }_{i=1}^n{a}_t^i{v}_t^i$$

(20)

For the final output of DAPNet, the FCNN is used to calculate global attention score as follows:

$$y\left(t\right)=\mathrm{F}\mathrm{C}\mathrm{N}\mathrm{N}\left(\widehat{h}\left(t\right)\right)$$

(21)

3. Experiments and Comparisons

3.1. Variable Definition and Data Explanation

The case ship in this study is a ferry that operates on the route from Tórshavn to Suduroy Island, as shown in Figure 12 [10]. As depicted in Figure 8, the solid line and the dotted line represent the route under normal sailing conditions ($R_N$) and the route under adverse sailing conditions ($R_A$), respectively. The datasets corresponding to the multi-source variables described in Figure 2, Figure 3 and Figure 4 were collected by the multi-source sensors installed on the ship, as shown in

Table 1. Variables explanation of data collection for ferry ship.

No.	Variables	Abbreviation
1	Ship Fuel Consumption	SFC
2	Speed	SPE
3	Port Pitch	PP
4	Starboard Pitch	SP
5	Trim	TR
6	Port Draft	PD
7	Starboard Draft	SD
8	Head Wind	HW
9	Cross Wind	CW

.

To ensure high-quality data for more effective model predictions, additional preparation is required following the processing of multi-source heterogeneous SFC data and prior to feeding it into the model. Firstly, the Min-Max normalization method is employed to bring all indicators to the same scale. Secondly, to further enhance the quality and quantity of the SFC input data, a sliding time window is employed in this paper for the subsequent framing of data. The final step involves data separation. The SFC dataset after framing can be divided into 52 complete voyages, totaling approximately 20,116 data points, and categorized into two datasets. One of the datasets includes 15,022 data points of the routes under normal sailing conditions, which is further divided into training, validation, and test sets with proportions of 70%, 10%, and 20%, respectively. The other includes 5094 data points of the routes under adverse sailing conditions, designated for validation experiments aimed at assessing the generality and effectiveness of the proposed model.

3.2. Parameters and Criteria

3.2.1. Parameter Settings

In this experiment, the training processes for all SFC prediction models are consistent. Additionally, we provide some key training parameters as follows: the initial learning rate is set to 0.01, the loss function used is mean squared error (MSE), the initial number of epochs is 10,000, and training is halted when the loss no longer decreases after 50 epochs. The batch size is 256 and the optimizer used is Adam. In addition, the time step is set to 16 for all models, the number of hidden layers is set to 1 for models with an RNN structure, and the filter is set to 3 for models with a TCN structure. Other parameters not explicitly mentioned here are set to default values. It is worth noting that the above parameters were determined based on extensive experimentation, and all models were trained for 20 cycles with the best performing models retained.

This paper compares the performance of DAPNet with ten distinct prediction models. These ten models encompass a traditional and advanced RNN and LSTM and state-of-the-art attention models. Due to the numerous parameters involved, which can form multiple combinations, it is challenging to list all of these combinations and their impact on the model’s predictive results. Therefore, this paper lists the parameter settings and the range of adjustments with representative features for each model, as shown in

Table 2. Parameter setting of different models.

No.	Model	Parameter Setting and The Range of Adjustments
1	Standard RNN (RNN) [30]	Hidden Layer Dimension (HLD) HLD = {128, 64, 32, 16, 8}
2	Standard LSTM (LSTM) [19]
3	BiGRU [20]
4	BiLSTM [21]
5	RNN-Attention (RA) [31]
6	BiLSTM-Attention (BA) [32]
7	Seq2Seq-Attention (SA) [27]
8	TCN [26]	Number of Res Block = {1, 2, 3} Kernel size = {2, 4, 8} d = {1, 2, 4}
9	TCN-Attention (TA) [28]
10	TCN-LSTM-Attention (TLA)	Number of Res Block = {1, 2, 3} Kernel size = {2, 4, 8} d = {1, 2, 4} HLD = {128, 64, 32, 16, 8}
11	DAPNet

.

The specific methods for adjusting the parameters of each model are described as follows. For the models RNN, LSTM, BiGRU, BiLSTM, RA, BA, and SA, the parameter adjustment range {128, 64, 32, 16, 8} signifies the corresponding number of neurons in the hidden layer. For the parameter adjustment range of the TCN and TA, the corresponding parameters of kernel size and d take on the values 2 and 1 in the first Res Block, 4 and 2 in the second Res Block, and 8 and 4 in the third Res Block. The parameter adjustment range of TLA and DAPNet can be summarized as including the range of the related parameters of the LSTM and TCN above. The complete training process of DAPNet is detailed in Algorithm 1.

Algorithm 1 Training Process of DAPNet in SFC Prediction Tasks

Input: $X=[SPE,PP,SP,TR,PD,SD,HW,CW]$;//Fuel consumption related variables

Output:  $y=SFC$;//The predicted fuel consumption

Initialization: All the parameters U.

for each epoch do

$h_k\left(i\right)=f_k\left(X_k\right)$;//Process Fuel consumption feature $X_k$ through networks ($k$ = LSTM, TCN, and improved TCN) to obtain feature representations $h\left(i\right)$

$h_i^{\prime }\left(t\right)={\mathrm{L}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{A}\mathrm{t}\mathrm{t}}^i\left(x\left(t\right)\right)$;//Apply local attention mechanism on $h\left(i\right)$ to enhance the feature representations and obtain $h^{\prime }\left(i\right)$

$\widehat{h}\left(t\right)=\mathrm{G}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}\mathrm{A}\mathrm{t}\mathrm{t}\left(h_1^{\prime }\left(t\right),\dots ,h_n^{\prime }\left(t\right)\right)$;//Fuse $h^{\prime }\left(i\right)$ using global attention to get global features $\widehat{h}\left(t\right)$

$y\left(t\right)=\mathrm{F}\mathrm{C}\mathrm{N}\mathrm{N}\left(\widehat{h}\left(t\right)\right)$;//The predicted fuel consumption $y\left(t\right)$ through fully connected layer

Update (U);//Update and optimize the parameters U of DAPNet by minimization loss

end for

return The trained DAPNet and the predicted fuel consumption $y\left(t\right)$

3.2.2. Evaluation Criteria

To validate the effectiveness of DAPNet, we employ MAE and RMSE to measure the disparity between actual and predicted values. MAE and RMSE are commonly used standards in the literature for assessing the performance of predictive models and are defined as follows:

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|y_i-y_i^{\prime }\right|$$

(22)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-y_i^{\prime }\right)}^2}$$

(23)

where $n$ represents the amount of test data and $y_i$ and $y_i^{\prime }$ denote actual and predicted value, respectively.

3.3. Results and Analyses

During the experimental stage, parameter tuning was initially conducted, and the optimal parameters for each model were selected based on the MAE and RMSE results. It is worth emphasizing that the TLA and DAPNet models employed a controlled variable approach during the parameter tuning process. Both models first optimized parameters related to the LSTM component. Subsequently, the optimal parameters for the TCN component were adjusted based on the optimal parameters obtained for the LSTM component of each model, resulting in distinct optimal parameter combinations for the two models. The outcomes are illustrated in Figure 13 and Figure 14.

In Figure 13, the parameter HLD refers to the Hidden Layer Dimension, whereas in Figure 14, the parameter Blocks represents the Number of Residual Blocks. Figure 13 illustrates the parameter tuning process for models such as RNN, LSTM, BiGRU, BiLSTM, RA, BA, SA, TLA, and DAPNet, where the evaluation criteria are specified as MAE and RMSE, respectively. It is noteworthy that TLA and DAPNet adjusted only the parameters in the LSTM component, leaving other parameters as default. Additionally, the optimal parameters for the aforementioned models are provided in

Table 3. Comparisons of prediction performance for different methods.

Methods	Optimal Parameter Values	MAE	RMSE
RNN [30]	HLD = 32	0.0105 ± 0.0008	0.0153 ± 0.0009
LSTM [19]	HLD = 16	0.0097 ± 0.0004	0.0147 ± 0.0004
BiGRU [20]	HLD = 16	0.0096 ± 0.0004	0.0158 ± 0.0009
BiLSTM [21]	HLD = 64	0.0094 ± 0.0003	0.0141 ± 0.0005
RA [31]	HLD = 16	0.0099 ± 0.0006	0.0152 ± 0.0014
BA [32]	HLD = 16	0.0092 ± 0.0008	0.0137 ± 0.0013
SA [27]	HLD = 64	0.0091 ± 0.0008	0.0138 ± 0.0013
TCN [26]	Number of Res Block = 2; (Kernel size, d) in each block = {(2, 1), (4, 2)}	0.0102 ± 0.0012	0.0155 ± 0.0021
TA [28]	Number of Res Block = 3; (Kernel size, d) in each block = {(2, 1), (4, 2), (8, 4)}	0.0094 ± 0.0008	0.0146 ± 0.0013
TLA	HLD = 64; Number of Res Block = 3; (Kernel size, d) in each block = {(4, 16), (8, 8), (16, 4)}	0.0092 ± 0.0007	0.0143 ± 0.0011
DAPNet	HLD = 16; Number of Res Block = 3; (Kernel size, d) in each block = {(4, 16), (8, 8), (16, 4)}	0.0073 ± 0.0003	0.0121 ± 0.0007

. From Figure 13, it is evident that DAPNet exhibits the best predictive performance. Figure 14 showcases the parameter tuning process for the models, including TCN, TA, TLA, and DAPNet. TLA and DAPNet performed TCN parameter tuning based on the optimal parameters obtained for the LSTM component in Figure 13. Similarly, the optimal parameters for these models are presented in Table 3. Notably, according to Figure 14, DAPNet continues to demonstrate optimal performance. To obtain the optimal model in terms of both accuracy and stability in predicting SFC, Table 3 also presents the MAE and RMSE values of each model under the aforementioned optimal parameters. The results of all comparative models presented in Table 3 are the means and standard deviations (mean ± standard deviation) obtained from 20 independent runs, ensuring a reduction in randomness. Furthermore, the comparisons of different SFC prediction models are displayed in Figure 15 to compare the predictive performance of SFC more clearly. The error bars in Figure 15 represent the standard deviation across 20 experimental runs.

As evident from Figure 15 and Table 3, DAPNet emerges as the model with the optimal SFC predictive performance, striking a balanced trade-off between accuracy and stability. The average values of MAE and RMSE for 20 experimental runs of DAPNet are 0.0073 and 0.0121, respectively, with minimum values reaching 0.0065 and 0.0108. The results indicate that DAPNet improves the MAE and RMSE values for SFC prediction by 23.96% and 17.50%, respectively, compared to classic and state-of-the-art models. Conversely, the LSTM and BiLSTM demonstrate commendable stability. However, their predictive performance is subpar, attributed to the inability of such simple networks to effectively extract relevant features associated with SFC. Further analysis reveals that the predictive performance of the RNN, LSTM, and BiLSTM, incorporating attention mechanisms, surpasses that of the single RNN, LSTM, BiGRU, and BiLSTM structures. Similarly, the predictive performance of the attention-based TCN surpasses that of the single TCN. Therefore, attention mechanisms contribute significantly to the enhancement of SFC predictive accuracy. Moreover, combining the LSTM and TCN models based on attention further improves SFC predictive accuracy. Specifically, TLA and DAPNet exhibit lower MAE and RMSE values compared to the other models in Table 3. The predictive performance of DAPNet is further enhanced compared to TLA, showing varying degrees of improvement in terms of MAE, RMSE, and their corresponding stabilities. Thus, the proposed model structure in this study demonstrates superior suitability for SFC predictive research compared to traditional hybrid model structures.

To validate the generalizability of DAPNet in the domain of SFC prediction, this paper conducts predictions on the dataset from an alternative route of the case ship outlined in Section 3.1. Similarly, the prediction effect of each model is compared under the best parameters, as presented in

Table 4. Investigations of generalization for different methods in validation dataset.

Methods	MAE	RMSE
RNN [30]	0.0228 ± 0.0012	0.0270 ± 0.0018
LSTM [19]	0.0221 ± 0.0018	0.0259 ± 0.0021
BiGRU [20]	0.0201 ± 0.0013	0.0229 ± 0.0016
BiLSTM [21]	0.0196 ± 0.0014	0.0234 ± 0.0012
RA [31]	0.0207 ± 0.0021	0.0247 ± 0.0020
BA [32]	0.0192 ± 0.0016	0.0223 ± 0.0014
SA [27]	0.0196 ± 0.0016	0.0224 ± 0.0015
TCN [26]	0.0216 ± 0.0021	0.0255 ± 0.0028
TA [28]	0.0195 ± 0.0013	0.0237 ± 0.0013
TLA	0.0189 ± 0.0024	0.0227 ± 0.0024
DAPNet	0.0157 ± 0.0012	0.0198 ± 0.0018

and Figure 16.

It is obvious from Table 4 and Figure 16 that DAPNet consistently outperforms other models for SFC prediction. The average values of MAE and RMSE across 20 experimental runs of DAPNet can reach 0.0157 and 0.0198, respectively. These results indicate that the DAPNet proposed in this paper demonstrates strong generalizability.

3.4. Comparisons and Discussions

The ablation study conducted in this study aims to validate the functionality of each structural component of the DAPNet. The ablation study separately discusses the impact of the framing, improved TCN, LSTM, local attention, and global attention structures utilized in this paper on DAPNet. The results presented in

Table 5. Ablation investigations of proposed method.

Methods	MAE	RMSE	Train Time	Test Time
DAPNet without data alignment	0.0190	0.0241	474.34 s	2.66 s
DAPNet without local attention	0.0094	0.0143	408.19 s	2.05 s
DAPNet without global attention	0.0095	0.0146	433.98 s	3.27 s
DAPNet without local/global attention	0.0098	0.0148	335.68 s	2.20 s
DAPNet using traditional TCN	0.0096	0.0149	431.80 s	3.19 s
DAPNet using BiLSTM	0.0097	0.0155	456.73 s	3.84 s
Proposed DAPNet	0.0073	0.0121	444.26 s	3.24 s

represent the average values derived from 20 experimental runs. Furthermore, Table 5 also presents training time and test time.

As shown in Table 5, the proposed DAPNet represents the SFC prediction approach introduced in this paper. It is observed that the SFC predictive performance of the proposed DAPNet remains the best, with the average values of MAE and RMSE across 20 experiments reaching 0.0073 and 0.0121, respectively. Generally, although more complex network structures result in longer prediction times, the difference in computational time among the various models remains relatively minor. Upon further observation, DAPNet without data alignment implies the removal of the framing structure from the proposed DAPNet. An analysis of the results reveals that the proposed DAPNet outperforms DAPNet without data alignment due to the framing structure’s ability to accomplish data alignment. DAPNet without local attention implies that the proposed DAPNet removes the local attention structure. The superior predictive performance of the proposed DAPNet is attributed to the feature alignment capability provided by local attention. Similarly, the proposed DAPNet performs better than DAPNet without global attention due to the assistance of global attention in feature fusion. An analysis of the results of DAPNet without local attention and global attention shows that it is evident that removing both local attention and global attention has the most significant negative impact on the predictive performance of the proposed DAPNet. Conversely, DAPNet with the traditional TCN indicates the replacement of the improved TCN in the proposed DAPNet with a traditional TCN. DAPNet with BiLSTM signifies the replacement of LSTM in the proposed DAPNet with BiLSTM. Both operations lead to a certain degree of reduction in the predictive accuracy for the proposed DAPNet. This can be attributed to that the improved TCN and LSTM are better suited for extracting spatio-temporal features from SFC data, ensuring the model’s generalization capability. Overall, the above comparative experiments of SFC prediction models indicate that each structural component adopted in the proposed DAPNet has a positive impact on the overall predictive performance of the model. Furthermore, the effectiveness and accuracy of DAPNet are reaffirmed validated.

4. Conclusions and Future Work

This paper constructs a dual-attention parallel network, namely DAPNet. The proposed method effectively tackles the challenges posed by the multi-source characteristics and heterogeneous features of SFC-related data, ultimately enhancing the overall predictive performance. At the core of DAPNet, a parallel network structure composed of LSTM and an improved TCN is designed to further tackle the challenge of feature extraction. This parallel network structure inherits the advantages of both LSTM and the TCN and improves nonlinear fitting and the ability to extract temporal features from the data adequately. In each parallel network, local attention is incorporated to achieve feature alignment. The features are subsequently aggregated into global attention for feature fusion, thereby enhancing the predictive performance. To evaluate the effectiveness of the proposed DAPNet, a ferry is chosen as the case ship. The results demonstrate that DAPNet outperforms classic neural networks and new hybrid neural networks in terms of predictive performance. According to ablation investigations, both the local and global attentions play a positive role in influencing the prediction outcomes. DAPNet achieves minimum MAE and RMSE values of 0.0065 and 0.0108, respectively. The results show that DAPNet improved the MAE and RMSE values for SFC prediction by 23.96% and 17.50%, respectively, compared to classic and state-of-the-art models. Furthermore, the generalization of DAPNet is also validated through comparisons across different routes of the same case ship.

However, there is considerable room for scalability in terms of data sources and model architecture, and several directions for further enhancement can be considered in the future. The SFC data currently used in this study are relatively limited in scope. The key data that significantly affect the prediction of SFC, such as rudder, are not included in the data sources of this study. Specifically, since the weather conditions in this study from the noon report were manually recorded, they may not accurately reflect the actual weather encountered during navigation. In the next phase of this study, we plan to incorporate additional SFC-related feature data, such as rudder, as well as reliable meteorological data corresponding to the navigation routes. This will address the issue of data quality and ensure that the conclusions drawn from the research are more in line with established knowledge in the maritime field. The application of the proposed model to various types of ships can be investigated to further explore the SFC predicting applicability. Additionally, future work could extend to research areas such as the optimization of ship energy and speed based on predicted SFC.