Deep Learning-Based Time-Series Analysis for Detecting Anomalies in Internet of Things

Round 1

Reviewer 1 Report

Anomaly detection on time series data is an integral part of the context of the Internet of Things. This paper investigates the performance of deep learning-based models including recurrent neural network-based Bidirectional LSTM (BI-LSTM) and Long Short-Term Memory (LSTM), CNN-based Temporal Convolutional (TCN), and CuDNN-LSTM, a fast LSTM implementation supported by CuDNN.

1) Why does this paper only focus on the Sing Stage Single Point attack?  I think more experiments should be added.

2) Why do the LSTM and CudNN-LSTM use different model architectures? Table 1 shows that the CudNN-LSTM is better than LSTM in different model architectures. I think for comparison the model architecture should be identical.

3) How to choose the timestamps? Table 1 shows that short timestamps get a better result.  Can you explain it?

Author Response

Response to Reviewer 1 Comments

 

 

Anomaly detection on time series data is an integral part of the context of the Internet of Things. This paper investigates the performance of deep learning-based models including recurrent neural network-based Bidirectional LSTM (BI-LSTM) and Long Short-Term Memory (LSTM), CNN-based Temporal Convolutional (TCN), and CuDNN-LSTM, a fast LSTM implementation supported by CuDNN.

 

Point 1: Why does this paper only focus on the Sing Stage Single Point attack?  I think more experiments should be added.

 

Author’s Response 1: The authors appreciate the reviewer’s comment. The reason to focus on Single Stage Single Point Attack is because of the limitation in the number of attacks in other types of attacks. The chosen type of attack in this dataset includes several anomalies, whereas another type of attacks has only a few anomalies making hard to study them properly. However, as per the suggestion, we have conducted additional experiments with other types of attacks and reported the result in the paper. More specifically, we performed additional experiments for 1) Single Stage Multi-Point Attacks, 2) Multi Stage Single-Point Attacks, and 3) Multi Stage Multi-Point Attacks types of attacks.

 

 

Author’s Action 1: Section “6.2. Analysis of Different Types of Attacks” contains the additional experiments result and outcomes from other attacks.

 

Point 2: Why do the LSTM and CudNN-LSTM use different model architectures? Table 1 shows that the CudNN-LSTM is better than LSTM in different model architectures. I think for comparison the model architecture should be identical.

 

Author’s Response 2: The authors thank for this insightful comment. As we described in background section of CuDNN LSTM, this type of implementation of LSTM model does not have any dropout layer.  As per the suggestion of the reviewer, we have built additional models and conducted additional experiments based on added three models (i.e TCN, LSTM, BI-LSTM) without Dropout layer for the experiments, the results have been added in Table 1.

 

Author’s Action 2: Additional models are built and additional experiments are conducted. The results of the models are updated in Table 1 along with the architecture in Figure 4 of the architecture. Section 4.4 Model Architecture has been updated as per the changes in the models. The experiments of other attacks scenarios using updated models have been incorporated as suggested in the reviewer’s comment 1.

 

 

Point 3: How to choose the timestamps? Table 1 shows that short timestamps get a better result.  Can you explain it?

 

Author’s Response 3:  The authors appreciate the reviewers' feedback. The selection of e proper timestamp needs additional experiment to study its affect on the performance. Therefore, we performed additional analysis and experiments and reported the results the figures and tables.

 

Author’s Action 3: Section 6.3 “The Effect of Timestamp on Training Time and Performance” has been added to explain the affect of various values for the timestamp parameter.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

I am recommending accept with minor, but you need to make some corrections. 

- Your topic is: 'Deep Learning-based Time Series Analysis for Detecting Anomalies in Internet of Things', but what IoT devices? IoT is a term for a variety of systems and devices, which one specifically did you investigate? 

- Your article doesn't have the discussion and conclusion chapters, these are two separate chapters that are present in most articles. 

- You don't introduce or explain your tables and figures, this is a easy task, but quite necessary, without it, all these tables and figures just look dumped out of context

- there is a recent study on the related topic of ‘Algorithms for Artificial Intelligence on Low Memory Devices’ - see: https://ieeexplore.ieee.org/document/9502714 It would be interesting to see a few sentences review and comparison of your work in relations to these recent studies in related topics.

- some of the references are not in the required format e.g., Vailshery, L.S. Internet of things spending worldwide 2023, 2022.

Thats enough comments, I hope you find my feedback useful and I am looking forward to reading the updated version. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The paper has been revised according to my comments. I think it can be accepted.