Toward an Efficient Automatic Self-Augmentation Labeling Tool for Intrusion Detection Based on a Semi-Supervised Approach

Round 1

Reviewer 1 Report

 

In this work, the authors apply machine learning techniques to intrusion detection. They propose a semi-supervised approach called self automatic labeling. The method is tested against NSL-KDD dataset ISCX dataset. Experimental results are comparing to that of probabilistic graphical model method and various-widths clustering method. Comparing to the other two methods, the proposed method provides good accuracy but bad runtime performance for the reported data. The topic is interesting, but there are several issues that should be addressed.

 

Section 2.1

Line 102: What is DPNN? I checked the reference and it stands for “density peaks nearest neighbors.” It would be better to include the full name in the manuscript.

Line 109-113: There is a long sentence “Among the machine learning… IDS problems.” It seems to be grammatically incorrect.

 

Section 3.1

General question: The ensemble voting method is well-explained. However, it would be helpful to also provide some short description of the supervised classifiers C_i used in the experiments.

 

Line 240: Could the authors explain why Eq. (1) is not written in terms of argmax (like Eq. (2))?

 

Section 4.5

Fig.6, Fig.7, Fig.8, Fig.9: The scale of y-axis is different and missing for KNNVWC method.

 

Section 4.5.4

Table 5: What is the unit of runtime?

Line 546: There is a broken sentence “From Table.5 it 546 can be observed that.”

Line 548: “Second,When…” The “W” should be lower-case.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Results and illustrative material must be revised. It requires special attention.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

How to improve the classification accuracy of large-scale data using limited sample data is always the target of various semi-supervised learning models. In this manuscript, the Self Automatic Labeling approach, called SAL, is introduced for using a small labelled network traffic data to potentially detect most types of attacks in intrusion detection. The authors claimed that the proposed SAL approach has a better performance in overall accuracy, class balance accuracy, and F-measure by contrast to PGM and KNNVWC approaches.

overall, this research is innovative, the quality of presentation, however, needs to be improved. It is suggested to revise the content according to the following comments.

1.     The content to introduce the selected datasets is too much, as the datasets is not the core in this research.

2.     What do the TP, TN short for in equation 10, 11. More explanations for both variable and equations are required.

3.     Duplicated content in line 490.

4.     Title mistake of fig 11 (a)

5.     More introductions for fig 6-9, and fig 11 are required. Such as why the shapes like that, why the SAL has a better performance at normal class.

6.     The “most up-to-date” in line 424 is inappropriate, due to the PGM and KNNVWC are proposed in 2010 and 2015, respectively. The semi-supervised models, such as 3VMs, VAT(Virtual Adversarial Training)，MixMatch UDA(Unsupervised Data Augmentation), might be considered to select as a comparison in the manuscript. Anyway. the reason for selecting the PGM and KNNVWC should be given more.

7.     Table 4 and fig 10 have little significance in the manuscript.

8.     It seems unnecessary to use 2 datasets in the experiments. Using one dataset is enough to indicate that most types of attacks can be detected by using a small labeled network traffic data.

9.     It is necessary to further explain how the datasets are used in the experiment, such as the proportion of training and test data, iteration times, etc.

10.  How the Zero-Day attacks (mentioned in the abstract) is evaluated in the experiments?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The concerns in my previous report are addressed appropriately. I have a few minor suggestions:

 

1.     Page 14, line 460: The programming language, operation system and hardware for the experiments are mentioned. It would be nice to present some more details. For example, how many physical cores are there in the CPU? Is the program threaded? 2.6GHz is not enough for readers to identify the CPU being used. Is there some specific machine learning package being used in the python code? The software packages should also be cited.

2.     Page 18, line 564: “…and stable rate detection rate…” It seems like some revision would be required.

3.     Page 18, line 567: “…upon reason- able request…” Might be “reasonable”?

Author Response

Please see the attachment

Author Response File: Author Response.pdf