Detection of COVID-19 Cases Based on Deep Learning with X-ray Images
Round 1
Reviewer 1 Report
The article "Detection of COVID-19 cases Based on Deep Learning with X-ray Images" proposed a COVID-19 X-ray image detection model based on the multi-head self-attention mechanism and residual neural network. Overall, according to my opinion, the article is well organized and well written. The article will be a good addition in “ Deep Learning in Big Data, Image, and Signal Processing in the Modern Digital Age”. The authors of the manuscript are suggested to make minor revisions following the below points.
The following are some of the minor observations that the manuscript needs editing:
Line 2: add “identifying key features in medical images…”
Line 50: should read, “To solve the problem of the smaller medical image data samples and low detection accuracy, this paper……”
After Line 39: Need a paragraph (5 lines max) before line 40 highlighting briefly some of the literature from the past and current AI-assisted diagnosis and treatment technology in detecting diseases. And authors need to highlight how their work is novel or different compared to others in a similar research field.
Line 133-138: Not needed and the last 2 sentences in the paragraph are repeated from the previous one.
Table 1: is the image set taken from male or female or combined? It would be interesting to see if there is any significant differences.
Figure 9: the thickness of the outlines of the boxes in the figure looks blur or is un-even thickness. Please correct it and make it look smooth.
Line 520: I believe “Settings” should be “settings”.
Figure 12: what is the legend representing here. Please label the legend.
Line 692: please include a colon after “Visualization of attention mechanisms” instead of a full stop. Please do it for the rest of points in this sub-section, like line 699 and 704.
General query: I am curious to know if you have a complete case study showing clearly with qualitative images (not quantitative data and results which you have now) differentiating between COVID-19 pneumonia, common pneumonia, and normal lungs using MHSA-ResNet detection model and residual neural network on X-ray Images. If you have the qualitative images, please put them in a supplementary section.
Author Response
Response to Reviewer 1 Comments
Dear Editor and reviewers,
Thanks for your letter and the detailed comments on our manuscript entitled "Detection of COVID-19 cases Based on Deep Learning with X-ray Images". The valuable advice that you provide for us definitely helps us greatly improve the quality of the paper. We have studied the comments carefully and have made a revision of the manuscript shown in the following table.
Point 1: Line 2: add “identifying key features in medical images…”
Response 1: According to your suggestion, we have transferred”... identifying medical images through deep learning can quickly and accurately screen infection cases. ” to” ... identifying key features in medical images through deep learning, infection cases can be screened quickly and accurately.”
Point 2: Line 50: should read, “To solve the problem of the smaller medical image data samples and low detection accuracy, this paper……”
Response 2: According to your suggestion, we have transferred” To solve the problem of the medical image data sample is small, and the detection accuracy is not high... ” to” To solve the problem of the smaller medical image data samples and low detection accuracy...”
Point 3: After Line 39: Need a paragraph (5 lines max) before line 40 highlighting some of the literature briefly from the past and current AI-assisted diagnosis and treatment technology in detecting diseases. And authors need to highlight how their work is novel or different compared to others in a similar research field.
Response 3: According to your suggestion, we have removed ”However, the application of AI-assisted diagnosis and treatment technology in detecting COVID-19 is unsatisfactory.” In addition, we have added a description of AI-assisted diagnosis and treatment technology in the related work section.
Point 4: Line 133-138: Not needed and the last 2 sentences in the paragraph are repeated from the previous one.
Response 4: This paragraph is mainly a summary of the previous one, so we keep it. We have removed the redundancy:” which can effectively detect medical images and identify positive cases of COVID-19”. And we deleted the repetitive part at the beginning of 2.1:” At present, most of the methods of applying deep learning technology to medical image detection are completed based on a convolutional neural network. A convolutional neural network uses many labeled data sets to train the model to achieve the ideal detection effect.”
Point 5: Table 1: is the image set taken from male or female or combined? It would be interesting to see if there are any significant differences.
Response 5: Thanks for this proposal. Aiming at the problem of less medical image data and low detection accuracy, this paper is devoted to the innovation of feature extraction and neural network model. In the subsequent research, we will consider collecting male and female medical image samples and discuss the differences between them.
Point 6: Figure 9: the thickness of the outlines of the boxes in the figure looks blur or is un-even thickness. Please correct it and make it look smooth.
Response 6: We have redrawn Figure 9 according to your suggestion.
Point 7: Line 520: I believe “Settings” should be “settings”.
Response 7: According to your suggestion, we have transferred” Therefore, the size of the convolution kernel and pooling method is consistent with the Settings of the ResNet residual network.” to” Therefore, the size of the convolution kernel and pooling method is consistent with the settings of the ResNet residual network."
Point 8: Figure 12: what is the legend representing here. Please label the legend.
Response 8: According to your suggestion, we have added the illustrations:” Figure12. Confusion matrix on the test set. The rows represent the predictions, and the columns represent the true labels. Both predicted, and true label scores have three categories: "0" for COVID-19, "1" for normal lungs, and "2" for common pneumonia. On the right is a legend that visualizes quantities as their corresponding colors, with darker colors having smaller values and lighter colors having larger values."
Point 9: Line 692: Please include a colon after "Visualization of attention mechanisms" instead of a full stop. Please do it for the rest of the points in this sub-section, like lines 699 and 704.
Response 9: According to your proposal, this place is really not standard. We have modified the opening sentences of these three paragraphs into complete sentence respectively. We have transferred” Visualization of attention mechanisms” to” Realize the visualization of attention mechanism." We have transferred”Extensions to detection capabilities” to” Realize the extension of detection function." We have transferred”Improvement of model” to” Implement model improvements.”
Point 10: General query: I am curious to know if you have a complete case study showing clearly with qualitative images (not quantitative data and results which you have now) differentiating between COVID-19 pneumonia, common pneumonia, and normal lungs using MHSA-ResNet detection model and residual neural network on X-ray Images. If you have the qualitative images, please put them in a supplementary section.
Response 10: Thank you for your suggestion. We don’t have the qualitative images at present.
Following your valuable advice we make an improvement of the scheme. We will appreciate if you give more comments to our paper and we will revise the manuscript carefully until it meets the standard. At last we want to thank you again for both the editor’s and your precise advice and concern of our paper. It will help us to improve the quality of our paper and give an important guiding significance to our researches. We are looking forward your reply.
Sincerely yours,
Zhiqiang Wang, Ke Zhang, Bingyan Wang
Author Response File: 
Author Response.docx
Reviewer 2 Report
The manuscript titled "Detection of COVID-19 cases Based on Deep Learning with X-ray Images" uses deep-learning based approaches, a feature extraction scheme that integrates grayscale co-occurrence matrix with dimensionality reduction algorithms including PCA and T-SNE, and a multi-head self-attention based mechanism and the residual network X-ray image detection model of COVID-19, to classify COVID-19 and normal (healthy) chest X-ray images. My few suggestions about this manuscript are listed below.
1. Authors should slightly improve the Introduction section. Currently, it appears bit short. Although at the end of this section specific work is listed nicely. However, for the benefit of the research community, Introduction section should be ended with contributions listed in bullet form.
2. At the end of Introduction section, please put a table of few acronyms that are widely used in this manuscript.
3. In section 3, many of the equations and evaluation parameters are listed. However, the actual flow of the proposed method is missing. It is very hard for the reader to follow the used algorithm. Therefore, I recommend to put a pseudo code of used algorithm.
4. I recommend to remove the basic details, such as the PCA.
5. Please improve the appearances of the figures. For instance, in print version, Fig. 13 does not dictate that where is normal and where is covid patient. So use different lines patterns to distinguish graphs in print versin. This comment is applicable to all such figures.
6. Please make the transparent blocks in figures, such as Fig. 8. In print version, this colored image does not appear appropriate.
7. Please thoroughly revise the English used in this paper. Few sentences span even over 4 lines, such as in Abstract and in various other places in the manuscript.
Author Response
Response to Reviewer 2 Comments
Dear Editor and reviewers,
Thanks for your letter and the detailed comments on our manuscript entitled "Detection of COVID-19 cases Based on Deep Learning with X-ray Images". The valuable advice that you provide for us definitely helps us greatly improve the quality of the paper. We have studied the comments carefully and have made a revision of the manuscript shown in the following table.
Point 1: Authors should slightly improve the Introduction section. Currently, it appears bit short. Although at the end of this section specific work is listed nicely. However, for the benefit of the research community, Introduction section should be ended with contributions listed in bullet form.
Response 1: We have already broken down the work of this paper at the end of the introduction, and listing contributions at the end may cause duplication. So we put “The specific work is as follows:” into“The main contribution includes the following:”
Point 2: At the end of Introduction section, please put a table of few acronyms that are widely used in this manuscript.
Response 2: In accordance with the official format, we have listed Abbreviations at the end of the body of the text.
Point 3: In section 3, many of the equations and evaluation parameters are listed. However, the actual flow of the proposed method is missing. It is very hard for the reader to follow the used algorithm. Therefore, I recommend to put a pseudo code of used algorithm.
Response 3: According to your suggestion, we have added some algorithms and put them in Table 4, Table 5, and Table 6.
Point 4: I recommend to remove the basic details, such as the PCA.
Response 4: In order to improve the efficiency, we use PCA and T-SNE in the feature extraction stage to reduce the dimension and visualize, and the experiment proves that it does achieve a certain effect.
Point 5: Please improve the appearances of the figures. For instance, in print version, Fig. 13 does not dictate that where is normal and where is covid patient. So use different lines patterns to distinguish graphs in print versin. This comment is applicable to all such figures.
Response 5: The following is a description of Figure 6, figure 12, and figure 13. The meanings of the three colors in Figure 6 were previously described in the main text:” Figure 6 depicts the data results visualized by PCA and T-SNE dimensionality reduction. The left figure shows the effect after PCA dimensionality reduction, and the right figure shows the visualization effect of T-SNE. In the figure, red represents the X-ray images of COVID-19, brown is the X-ray images of normal lungs, and purple is the X-ray images of common pneumonia.” In Figure 12, we supplement the figure legend in the main text:” ......and the visual confusion matrix is shown in Figure 12. In the figure, the rows represent the predictions, and the columns represent the true labels. Both predicted, and true label scores have three categories: "0" for COVID-19, "1" for normal lungs, and "2" for common pneumonia. On the right is a legend that visualizes quantities as their corresponding colors, with darker colors having smaller values and lighter colors having larger values.” In Figure 13, we supplement the figure legend in the main text:” ...... is shown in Figure 13. In the figure, the blue curve represents COVID-19, the orange curve represents normal lungs, and the green curve represents common pneumonia.”
Point 6: Please make the transparent blocks in figures, such as Fig. 8. In the print version, this colored image does not appear appropriate.
Response 6: According to your suggestion, we have changed the background of the module with text in Figure 8 to be transparent.
Point 7: Please thoroughly revise the English used in this paper. Few sentences span even over 4 lines, such as in Abstract and in various other places in the manuscript.
Response 7: We have revised the English expression questions in the context of the full text. Some sentences may have multiple juxtapositions that make them too long, so we changed them as much as possible. Other long sentences we split to varying degrees. For example, we put long sentences," Therefore, the use of AI-assisted diagnosis and treatment technology can share the pressure of medical workers, help them diagnose COVID-19 quickly and accurately, achieve early detection, early isolation, and early treatment, and reduce the spread of COVID-19 and the risk of infection of social personnel." into two sentences“ Therefore, the use of AI-assisted diagnosis and treatment technology can share the pressure of medical workers and help them diagnose COVID-19 quickly and accurately.” and“ This technology can achieve early detection, early isolation, and early treatment, so as to reduce the spread of COVID-19 and the risk of infection of social personnel."
Following your valuable advice we make an improvement of the scheme. We will appreciate if you give more comments to our paper and we will revise the manuscript carefully until it meets the standard. At last we want to thank you again for both the editor’s and your precise advice and concern of our paper. It will help us to improve the quality of our paper and give an important guiding significance to our researches. We are looking forward your reply.
Sincerely yours,
Zhiqiang Wang, Ke Zhang, Bingyan Wang
Author Response File: Author Response.docx
Reviewer 3 Report
The manuscript reports the deep-learning assisted diagnosis of Covid-19, by training the model with X-ray images acquired from Covid-19/other pneumonia/healthy lungs.
Despite the topic being relevant to the scientific community and matching the journal's purpose, the methods already reported in the literature are quite relevant and more advanced with respect to this manuscript.
In the referee’s opinion, the experimental procedure seems to be rigorous. However, it is unclear what the added value to the scientific community is. Plus, the manuscript does not match the quality standards for publication in Electronics.
Some remarks are reported in the following:
1. The sentence in Page 1/26 Line 7-10 is just repetition of Line 3-7.
2. Page 1/26 Line 19-20 ‘As of April 2022, COVID-19 has infected more than 400 million people worldwide and killed more than 6 million.’, the data source must be cited.
3. Fig.1 is acquired from WHO database, which is protected by ‘CC BY 4.0’.
4. Page 2/26 Line 32-34 ‘If the system environment is good enough, the computer can learn much faster than humans, and the learning model can be copied, significantly improving efficiency.’ The learning process of a computer is different from a human being. The statement is not rigorous.
5. Page 2/26 Line 32-34 ‘However, the application of AI-assisted diagnosis and treatment technology in detect- 40 ing COVID-19 is unsatisfactory.’ Unsatisfactory in terms of what? Try to be more specific.
6. Page 3/26 Line 91-92 ‘At present, researchers at home and abroad have done a lot of research on image detection of COVID-19,’ Where is ‘at home’?
7. Wrong way of making citations, in general.
8. Acronyms need to be defined the first time when they appear.
9. Page 3/26 Line 112-113 ‘Compared with VGG, the detection effect of this method is much better.’ Which method?
10. Page 6/26 Line 192 ‘The Kaggle dataset is …’ The database is not acknowledged with citation.
11. Page 6/26 Line 211 ‘DICOM, the abbreviation of medical mathematical imaging and communication,’ Again, repetition of Line 210.
12. Fig.3 Error in the y-axis label.
Comments for author File: 
Comments.pdf
Author Response
Response to Reviewer 3 Comments
Dear reviewer,
Thanks for your letter and the detailed comments on our manuscript entitled "Detection of COVID-19 cases Based on Deep Learning with X-ray Images".
Our manuscript is written according to the official template and requirements. I would like to introduce to you the innovation and contribution of our manuscript. In our manuscript, in order to effectively extract the features of medical X-ray images, we first propose a texture feature extraction and selection scheme based on a gray level co-occurrence matrix for X-ray images. Firstly, we extract texture features from X-ray images based on the gray co-occurrence matrix and then implement feature selection by PCA and T-SNE algorithm. Through ablation experiments, the proposed feature extraction scheme improves the accuracy by more than 20 percentage points compared with the scheme using only a gray co-occurrence matrix. In order to improve the detection accuracy, we also designed a novel COVID-19 X-ray image detection model based on a multi-head self-attention mechanism and residual network, namely MHSA-ResNet. The experimental results show that, compared with the residual network, the number of training parameters is reduced by about 15%, the training speed is increased by about 19%, and the model accuracy is improved by 9.48%. In order to further verify the effectiveness of the detection scheme, we designed a comparative evaluation scheme on the neural network model and carried out a large number of experiments on open-source medical image datasets. Compared with the latest detection methods, such as long short-term memory neural network, twin network, and convolutional neural network, the experimental results show that the accuracy of the MHSA-ResNet detection model reaches 95.52% and the accuracy reaches 96.02%, which proves the effectiveness of our proposed scheme.
The valuable advice that you provide for us definitely helps us significantly improve the quality of the paper. We have studied the comments carefully and have made a revision of the manuscript shown in the following table.
Point 1: The sentence in Page 1/26 Line 7-10 is just repetition of Line 3-7
Response 1: The original intention here is to summarize the method of this paper in the form of total - points. Maybe the summary sentence is written too specific, which causes the repetition of the sub-sentence. Part of the umbrella sentence has been deleted:“ a feature extraction scheme combining grayscale co-occurrence matrix with dimensionality reduction algorithms including PCA and T-SNE, and a multi-head self-attention-based mechanism and the residual network X-ray image detection model of COVID-19".
Point 2: Page 1/26 Line 19-20 ‘As of April 2022, COVID-19 has infected more than 400 million people worldwide and killed more than 6 million.’, the data source must be cited.
Response 2: We've changed this part to:" As shown in Figure 1, as of March 25, 2019, COVID-19 had caused more than 400 million infections and more than 6 million deaths worldwide, according to WHO statistics."
Point 3: Fig.1 is acquired from WHO database, which is protected by ‘CC BY 4.0’.
Response 3: The description of Figure 1 has been changed to“ Figure 1. Number of COVID-19 cases according to WHO (as of March 25, 2019). This figure is acquired from WHO database, which is protected by ‘CC BY 4.0’."
Point 4: Page 2/26 Line 32-34 ‘If the system environment is good enough, the computer can learn much faster than humans, and the learning model can be copied, significantly improving efficiency.’ The learning process of a computer is different from a human being. The statement is not rigorous.
Response 4: In accordance with your suggestion, we have deleted“ If the system environment is good enough, the computer can learn much faster than humans, and the learning model can be copied, significantly improving efficiency."
Point 5: Page 2/26 Line 32-34 ‘However, the application of AI-assisted diagnosis and treatment technology in detecting COVID-19 is unsatisfactory.’ Unsatisfactory in terms of what? Try to be more specific.
Response 5: The sentence you said is not found in the tex document and may have been deleted.
Point 6: Page 3/26 Line 91-92 ‘At present, researchers at home and abroad have done a lot of research on image detection of COVID-19,’ Where is ‘at home’?
Response 6: We've changed this place to“ At present, researchers have done a lot of research on image detection of COVID-19, ......"
Point 7: Wrong way of making citations, in general.
Response 7: Our citations are written in accordance with the citation style guide provided by the MDPI. We have revised and improved the references in references 5, 6, 25, etc.
Point 8: Acronyms need to be defined the first time when they appear.
Response 8: We have defined the first abbreviations, such as PCA and T-SHE, in the summary section:“ Considering that the gray co-occurrence matrix contains a large number of features, we use principal components analysis (PCA) and t-distributed stochastic neighbor embedding (T-SNE) algorithms to complete feature dimensionality reduction and visualize the data." MHSA-ResNet, VGG, CT, DICOM, PNG, OpenCV, GLCM., ReLU, RMSProp, LSTM, and Grad-CAM were also defined when they first appeared.
Point 9: Page 3/26 Line 112-113 ‘Compared with VGG, the detection effect of this method is much better.’ Which method?
Response 9: We have added the content and results of this method in this section:" The authors collected chest X-ray images of COVID-19 positive patients and normal patients from the publicly available Kaggle dataset. The authors found through experiments that the accuracy rate of the Bayesian convolutional neural network was 92.9%. Compared with VGG, the detection effect of this method is much better."
Point 10: Page 6/26 Line 192 ‘The Kaggle dataset is …’ The database is not acknowledged with citation.
Response 10: This dataset is the Kaggle dataset collated in reference 22. We've changed this part to" We also use image datasets collated in the Covid-Net literature\cite{22,23}, where the Kaggle dataset is composed of research institutions such as universities and hospitals from places such as Qatar. The researchers created a database of chest X-ray images used to identify positive cases of COVID-19, which is still being updated."
Point 11: Page 6/26 Line 211 'DICOM, the abbreviation of medical, mathematical imaging and communication,' Again, repetition of Line 210.
Response 11: This place does repeat. We've changed it to:" The X-ray medical image format is usually stored in DICOM (Digital Imaging and Communications in Medicine) format. DICOM is an international standard for medical imaging and related information......"
Point 12: Fig.3 Error in the y-axis label.
Response 12: This is a gray histogram. Gray histogram reflects the gray distribution law in the image. It describes the number of pixels per gray level. So the vertical axis "# of pixels" represents the number of pixels per pixel on the horizontal axis. We have supplemented the relevant parts in the main text:" The gray scale is shown in Figure 3. The horizontal axis represents the gray level, which is set to 256, and the vertical axis represents the number of times each gray level occurs."
Following your valuable advice we make an improvement of the scheme. We will appreciate if you give more comments to our paper and we will revise the manuscript carefully until it meets the standard. At last we want to thank you again for both the editor’s and your precise advice and concern of our paper. It will help us to improve the quality of our paper and give an important guiding significance to our researches. We are looking forward your reply.
Sincerely yours,
Zhiqiang Wang, Ke Zhang, Bingyan Wang
Author Response File: Author Response.docx
Round 2
Reviewer 3 Report
All the concerns have been properly addressed. One more comment: the legend in Fig.6 (left) is barely visible, please revise.
Author Response
Response to Reviewer 3 Comments
Dear reviewer,
Thanks for your letter and the detailed comments on our manuscript entitled "Detection of COVID-19 cases Based on Deep Learning with X-ray Images". The valuable advice that you provide for us definitely helps us significantly improve the quality of the paper. We have studied the comments carefully and have made a revision of the manuscript shown as follows:
Point 1: the legend in Fig.6 (left) is barely visible, please revise.
Response 1: We have changed the legend on the left of Figure 6. Please see it in the new manuscript.
Following your valuable advice we make an improvement of the scheme. We will appreciate if you give more comments to our paper and we will revise the manuscript carefully until it meets the standard. At last we want to thank you again for both the editor’s and your precise advice and concern of our paper. It will help us to improve the quality of our paper and give an important guiding significance to our researches. We are looking forward your reply.
Sincerely yours,
Zhiqiang Wang, Ke Zhang, Bingyan Wang
Author Response File: Author Response.docx
