An Efficient Feature Augmentation and LSTM-Based Method to Predict Maritime Traffic Conditions

Round 1

Reviewer 1 Report

Editing Notes:

1. After all formulas, replace the ":" and "=" signs with "-" before the variable descriptions.

2. In all formulas, change the product sign from "x" to "." – the authors probably meant scalar product, not vector product.

3. In the descriptions of variables used under the formulas, the capitalization of letters should be standardized and a comma should be used in calculations and a period at the end of the task.

4. In the text, change "Fig." on "Figures".

 

Substantive remarks:

1. The manuscript lacks a description of other factors that may affect the forecasting of ship traffic, i.e. weather conditions, tidal and surface currents, maritime traffic regulation systems (including port regulations regarding speed limits), or information on the shape of the seabed (obtained e.g. from electronic navigational chart).

2. Section 4.2 should describe in more detail the AIS data set (transponder class, messages used) and the method of converting position coordinates into routes crossing the gate line (ellipsoidal to flat coordinates, e.g. UTM, or calculations performed directly on the ellipsoid).

3. Applications should be extended with proposals for further development of the conducted research.

Author Response

Reviewer#1, Concern # 1:  

After all formulas, replace the ":" and "=" signs with "-" before the variable descriptions.

Author response:  Thanks for comprehensively reviewing the  paper. We will update all formulas.

Author action: We have updated all  ":" and "=" signs with "-".

 

Reviewer#1, Concern # 2:

In all formulas, change the product sign from "x" to "." – the authors probably meant scalar product, not vector product.

Author response: Once again thanks for comprehensively reviewing the paper. We will update all formulas.

Author action: we have updated all the formula according to your concern.

 

Reviewer#1, Concern # 3:

In the descriptions of variables used under the formulas, the capitalization of letters should be standardized and a comma should be used in calculations and a period at the end of the task.

Author response:  I appreciate for comprehensively reviewing the paper. We will update all formulas.

Author action: we have updated all the formula according to your concern.

 

Reviewer#1, Concern # 4:

In the text, change "Fig." on "Figures".

Author response: Many thanks to comprehensively reviewing the paper. We will change all text “Fig.” to “Figures”.

Author action: In the revised version, we have checked and corrected all the term “Fig.” and replaced with the word “Figures”.

 

Reviewer#1, Concern # 5:

The manuscript lacks a description of other factors that may affect the forecasting of ship traffic, i.e. weather conditions, tidal and surface currents, maritime traffic regulation systems (including port regulations regarding speed limits), or information on the shape of the seabed (obtained e.g. from electronic navigational chart).

Author response:  Thank you for your suggestions on the lack part of the manuscript. It is important to mention it in the manuscript. We will update manuscript including other factors that may affect the forecasting of ship traffic.

Author action: In the revised version, we have mentioned the other factor for forecasting of ship traffic as reviewer’s suggestion at Section 4.3.

 

Reviewer#1, Concern # 6:

Section 4.2 should describe in more detail the AIS data set (transponder class, messages used) and the method of converting position coordinates into routes crossing the gate line (ellipsoidal to flat coordinates, e.g. UTM, or calculations performed directly on the ellipsoid).

Author response:  I appreciate for your detailed and professional suggestions about AIS data, we will review about the data with reference the Recommendation of International Telecommunication Union(ITU-R M.1371-5), and mention about the detail the AIS data set used in the paper.

Author action: In the revised version, we have mentioned about the detail the AIS data set used in the paper as reviewer’s suggestion at Section 4.2.

 

Reviewer#1, Concern # 7:

Applications should be extended with proposals for further development of the conducted research.

Author response:  Many thanks to your suggestion to extending the applications with proposals for further development. We will update about extending the applications with proposals for further development.

Author action: In the revised version, we have mentioned about extending the applications with proposals for further development in the paper as reviewer’s suggestion at Section 6.

Author Response File: Author Response.docx

Reviewer 2 Report

Review references 10 and 25, it are from 2003 and 2007, update the citations with recent works in state of the art.

Section 3 mentions "Guidelines on Maritime Traffic Safety Audit in accordance with the Maritime Safety Act in South Korea", explain previously and cite the source.

In section 4, a detailed description of the dataset is required. how can you access the dataset? What is the size, number of records in the dataset? What type of information does the dataset contain?

In the subsection "4.4 Training and Experiment" the design of the LSTM is described and later it is mentioned that in figure 6 the configuration is shown. However, in figure 6 only the 3 LSTM layers are shown, the input-output layers as well as the hidden layers are not illustrated, nor is it mentioned whether in each LSTM layer there are additional parameters ? Or is it a "base LSTM"? Also, in the Hyperparameters table it is not mentioned that "Optimizer" was used.

 

In the Results section, the RMSE and MAE metrics should be included.

Section 5 mentions "Algorithm validation was performed using additional data that had not been used in the training and validation processes for the model under scenario (h). This data belongs to 20% of the dataset? It was previously mentioned that it is used an 80/20 arrangement for training and testing Clarify that information in the paragraph.

Author Response

Reviewer#2, Concern # 1:  

Review references 10 and 25, it are from 2003 and 2007, update the citations with recent works in state of the art.

Author response:  Thanks for comprehensively reviewing the paper. We will update the references with recent works in state of the art. It is important to have the most recent and relevant information in a rapidly evolving field like Artificial Intelligence.

Author action: In the revised version, we have updated references as reviewer’s suggestion at Section 2.

 

Reviewer#2, Concern # 2:

Section 3 mentions "Guidelines on Maritime Traffic Safety Audit in accordance with the Maritime Safety Act in South Korea", explain previously and cite the source.

Author response: I appreciate for comprehensively reviewing the paper. We will explain and cite about "Guidelines on Maritime Traffic Safety Audit in accordance with the Maritime Safety Act in South Korea".

Author action: In the revised version, in response to the reviewer's suggestion, we have explained and cite about the guidelines in Section 3.

 

Reviewer#2, Concern # 3:

In section 4, a detailed description of the dataset is required. how can you access the dataset? What is the size, number of records in the dataset? What type of information does the dataset contain?

Author response: I greatly appreciate your thorough review of the paper. As suggested, we will make mentioned about detailed description of the dataset.

Author action: In the revised version, we have mentioned about detailed description of the dataset as reviewer’s suggestion at Section 4 and Table 2 is added.

 

Reviewer#2, Concern # 4:

In the subsection "4.4 Training and Experiment" the design of the LSTM is described and later it is mentioned that in figure 6 the configuration is shown. However, in figure 6 only the 3 LSTM layers are shown, the input-output layers as well as the hidden layers are not illustrated, nor is it mentioned whether in each LSTM layer there are additional parameters ? Or is it a "base LSTM"? Also, in the Hyperparameters table it is not mentioned that "Optimizer" was used.

Author response: Many thanks to your comprehensively reviewing the paper. We will mention about the input-output layer, as well as hidden layer and also update about parameter including optimizer.

Author action: In the revised version, we have mentioned input-output layer, as well as hidden layer and also update about parameter including optimizer as reviewer’s suggestion at Section 4.4  and Table 3.

 

Reviewer#2, Concern # 5:

In the Results section, the RMSE and MAE metrics should be included.

Author response: I appreciate for comprehensively reviewing the paper. We will include the RMSE and MAE in the result section. These metrics help quantify the differences between the predicted values and the actual values, which are critical in determining the accuracy of the model's predictions.

Author action: In the revised version, we have added RMSE and MAE as reviewer’s suggestion at Section 5.

 

Reviewer#2, Concern # 6:

Section 5 mentions "Algorithm validation was performed using additional data that had not been used in the training and validation processes for the model under scenario (h). This data belongs to 20% of the dataset? It was previously mentioned that it is used an 80/20 arrangement for training and testing Clarify that information in the paragraph.

Author response: Many thanks to your comprehensively reviewing the paper. We will update the description clearly about the arrangement for training and testing data. The description about the arrangement for training and testing not clear enough to understand. Algorithm validation was performed using additional data that does not belong to either 80% or 20%. We will update the description clearly about the arrangement for training and testing.

Author action: In the revised version, we have updated the description clearly about the arrangement for training and testing data as reviewer’s suggestion at Section 4.4.

 

Author Response File: Author Response.docx

Reviewer 3 Report

 

Dear Authors,

I have reviewed the manuscript “An efficient Feature Augmentation and LSTM based method to predict Maritime Traffic Condition”, Manuscript ID: applsci-2223864 that has been submitted for publication in the: Applied Sciences (ISSN 2076-3417), and I have identified a series of aspects that in my opinion must be addressed in order to bring a benefit to the manuscript.

The article under review will be improved if the authors address the following aspects in the text of the manuscript:

1.     It is preferable that the abstract contain numerical results confirming the contribution of the research.

2.     LSTMs work very well for some problems, but some of the drawbacks are:

·       LSTMs take longer to train.

·       LSTMs require more memory to train.

·       LSTMs are easy to overfit.

·       Dropout is much harder to implement in LSTMs.

·       LSTMs are sensitive to different random weight initializations.

How were these problems overcome and why was LSTM chosen?

3.     Authors should use validation measures other than those used.

4.     The results were not compared with other researchers and the preference of the proposed was proven

5.     The references need to be updated for the years 2022 and 2023, as this field has been recently raised.

 https://doi.org/10.3390/su14137734

 

6.     The authors should provide more details in the conclusion about the future work that will be done later.

Author Response

Reviewer#3, Concern # 1:  

It is preferable that the abstract contain numerical results confirming the contribution of the research.

Author response:  Thanks for comprehensively reviewing the paper. We will update abstract including numerical result. Numerical results provide concrete evidence of the findings of the research, and they help to support the claims made in the abstract. They provide a clear and concise way to convey the significance of the research, and they allow others to quickly assess the importance of the work. Additionally, numerical results are often more memorable and impactful than qualitative results, and they can be used to help communicate the findings of the research to a wider audience.

Author action: In the revised version, we update abstract including numerical result as reviewer’s suggestion at Abstract.

 

Reviewer#3, Concern # 2:

LSTMs work very well for some problems, but some of the drawbacks are:

• LSTMs take longer to train.
• LSTMs require more memory to train.
• LSTMs are easy to overfit.
• Dropout is much harder to implement in LSTMs.
• LSTMs are sensitive to different random weight initializations.

How were these problems overcome and why was LSTM chosen?

Author response: I appreciate for comprehensively reviewing the paper. We will update about what is the drawbacks of LSTM, and the reasons for choosing LSTM, and how to overcome the drawbacks. LSTMs have become a popular choice for time series prediction tasks, despite the drawbacks mentioned. The reasons for choosing LSTMs are as follows:

• LSTMs have the ability to capture long-term dependencies, which is especially important in time series tasks. This is because LSTMs have a mechanism called a memory cell that can store information over a long period of time, and they have the ability to selectively read, write, and forget information based on the input.
• LSTMs have proven to be effective in tasks where traditional Recurrent Neural Networks (RNNs) have struggled, such as in handling vanishing gradients.

 

To overcome the drawbacks of LSTMs, we used several techniques and modifications to the original LSTM architecture as follows:

• LSTM training time can be reduced by using more efficient optimization algorithms such as Adam, Adagrad and Adadelta, and memory usage can be reduced by using gradient clipping to avoid exploding gradients. Adam is used for optimization and gradient clipping techniques were used in this paper.
• Overfitting problem was prevented by using dropout as a regularization technique, and Glorot uniform initialization was used as a weight initialization technique to make LSTM less sensitive in this paper.

Author action: In the revised version, in response to the reviewer's suggestion, we have updated the information regarding the drawbacks of LSTM models, the reasons for choosing LSTM, and strategies for overcoming these drawbacks in Section 4.4 and Table 2.

 

Reviewer#3, Concern # 3:

Authors should use validation measures other than those used.

Author response: I appreciate for comprehensively reviewing the paper. We will include the RMSE and MAE in the result section. These metrics help quantify the differences between the predicted values and the actual values, which are critical in determining the accuracy of the model's predictions.

Author action: In the revised version, we have added RMSE and MAE as reviewer’s suggestion at Section 5.

 

Reviewer#3, Concern # 4:

The results were not compared with other researchers and the preference of the proposed was proven.

Author response: Thanks for comprehensively reviewing the paper. We will update result including compared with other researchers. The comparison of results with other studies is important in evaluating the performance of the proposed method.

Author action: In the revised version, we have added comparison of results with other studies as reviewer’s suggestion at Section 5 and, add the table 8, Figure 9.

 

Reviewer#3, Concern # 5:

The references need to be updated for the years 2022 and 2023, as this field has been recently raised.

Author response: I greatly appreciate your thorough review of the paper. As suggested, we will make updates to the references including recommended papers (https://doi.org/10.3390/ su14137734). It is important to keep references up-to-date, especially in rapidly changing fields such as Artificial Intelligence and related fields. This helps ensure that the information used in a study is accurate and relevant to the current state of the field.

Author action: In the revised version, we have updated references as reviewer’s suggestion at Section 2.

 

Reviewer#3, Concern # 6:

The authors should provide more details in the conclusion about the future work that will be done later.

Author response:  Thank you for your insightful suggestion regarding future work. We will update conclusion including about the future works.

Author action: In the revised version, we have mentioned about extending the applications with proposals for further development in the paper as reviewer’s suggestion at Conclusion.

Author Response File: Author Response.docx

Reviewer 4 Report

 

 

1.   The paper proposed an algorithm that predicts future maritime traffic conditions based on past data to improve the performance of autonomous ships. Although the research area is very interesting, the aim of the paper is not mention properly in the abstract.

2.   What do you mean by securing the dataset? How did you do that?

3.   In the preprocessing steps, did you perform interpolation for the missing data?

4.   Why feature augmentation is applied? Is it for dataset balancing or only for solving the overfitting problem?

5.   Construction diagram of the algorithm is not clear in fig (1) especially after feature augmentation process. Please redraw it.

6.   A detail of LSTM configuration model is required.

7.   Did you try the ratio 7:3 for the training and validation data? And compare with 8:2 results.

8.   The results need a fair comparison with other related works that you have used as references.

 

Comments for author File: Comments.docx

Author Response

Reviewer#4, Concern # 1:  

The paper proposed an algorithm that predicts future maritime traffic conditions based on past data to improve the performance of autonomous ships. Although the research area is very interesting, the aim of the paper is not mention properly in the abstract.

Author response:  Thanks for comprehensively reviewing the paper. We will update abstract including the aim of the paper. The abstract of the paper should be revised to properly mention the aim of the proposed algorithm. The paper aims to predict future maritime traffic conditions based on past data to improve the performance of autonomous ships. This aim should be clearly stated in the abstract to provide readers with a clear understanding of the research focus and goals of the paper.

Author action: In the revised version, we update abstract including the aim of the paper as reviewer’s suggestion at Abstract.

 

Reviewer#4, Concern # 2:

What do you mean by securing the dataset? How did you do that?

Author response: Thank you for taking the valuable time to review our paper. We will change the phrase "securing the dataset" to "obtaining the dataset". We apologize for the confusion caused by the use of the phrase "securing the dataset" What we intended to convey was the process of obtaining the dataset. The dataset was obtained through a purchase from a company specializing in the collection of AIS data.

Author action: In the revised version, we have revised the phrase "securing the dataset" to "obtaining the dataset" at Section 4.1 and 4.2.

 

Reviewer#4, Concern # 3:

In the preprocessing steps, did you perform interpolation for the missing data?

Author response: Thank you for thoroughly reviewing the paper. We utilized linear interpolation to handle missing data, however, it was not explicitly stated in the paper. We will ensure to include this in the revised version.

Author action: In the revised version, we have mentioned about process of handling the missing data as reviewer’s suggestion at 4.2.

 

 

Reviewer#4, Concern # 4:

Why feature augmentation is applied? Is it for dataset balancing or only for solving the overfitting problem?

Author response: I appreciate for comprehensively reviewing the paper. Feature argumentation is applied to solve both problems of dataset balancing and overfitting problem. Feature augmentation is a technique used in machine learning to increase the size of the dataset. It is achieved by generating new instances of the data using transformations or operations that preserve the underlying structure of the data. These new instances are combined with the original data to create a larger dataset that can be used to train a machine learning model.

Author action: In the revised version, we have addressed the reviewer's suggestion by explaining the purpose of applying feature augmentation in Section 4.3.

 

Reviewer#4, Concern # 5:

Construction diagram of the algorithm is not clear in fig (1) especially after feature augmentation process. Please redraw it.

Author response: I am grateful for your comprehensive review of the paper. In response to your feedback, we will redraw the flowchart to clearly address the issue of the loop created by inaccuracies in the process flow specially after feature augmentation process.

Author action: In the revised version, we have redrawn the as reviewer’s suggestion at Section 4.1.

 

Reviewer#4, Concern # 6:

A detail of LSTM configuration model is required.

Author response: Thank you for taking the time to carefully review the paper. We have noted your concern about the insufficient explanation of the LSTM configuration and will provide a more detailed description in the revised version.

Author action: In the revised version, we have updated the detail of LSTM configuration as reviewer’s suggestion at Section 4.4.

 

Reviewer#4, Concern # 7:

Did you try the ratio 7:3 for the training and validation data? And compare with 8:2 results.

Author response: I appreciate for comprehensively reviewing the paper. In response to your suggestion, we conducted an evaluation by dividing the data into 7:3 for training and validation and compared the results with the previous 8:2 division. However, we did not observe significant differences in trend of result.

Author action: In the revised version, we have included the results of our comparison between the use of 7:3 and 8:2 ratios for training and validation data at Section 4.4.

 

Reviewer#4, Concern # 8:

The results need a fair comparison with other related works that you have used as references.

Author response: Thanks for comprehensively reviewing the paper. We will update result including compared with other researchers. The comparison of results with other studies is important in evaluating the performance of the proposed method.

Author action: In the revised version, we have added comparison of results with other studies as reviewer’s suggestion at Section 5 and, add the table 8, Figure 9.

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

Accepted in present form