Intelligent Damage Assessment for Post-Earthquake Buildings Using Computer Vision and Augmented Reality
Round 1
Reviewer 1 Report
The paper proposes a method to integrate Computer Vision (CV) and Augmented Reality (AR) to improve the efficiency and objectivity of post-earthquake surveys. However, the proposed method still remains a visual inspection with all the related limitations (superficial survey, no testing of any material properties etc.). English is not sufficiently good and should be improved, especially in the introduction section. Some suggestions are as follows:
Lines 16-17: please, define acronyms when they first appear in the manuscript (e.g. CV, AR, CNN etc.), abstract included.
Lines 31-32: please, consider replacing “… relied on the destroying information…” with “..based on information about the destruction occurred…”
Line 36: please, consider adding “Visual inspection by expert surveyors ...” to “Visual inspection...”
Lines 37-38: the sentence “…the results are often needed to be artificial calculation, which is not suitable for mass assessment and real-time assessment.” Is not clear. Please, reformulate it or explain more explicitly that are the problems of post-earthquakes visual inspection results.
Lines 38-39: also the sentence “Computer vision technology (CV) is an integrated technology” is quite poor and unclear. Maybe you can try with “Computer vision (CV) is a technology that integrates the management of acquired data from digital vision devices with computer hardware and software resources”. Otherwise, please, explain more explicitly what you mean.
Lines 40-41. Do you mean “…to calculate indirect damage information” or “…to indirectly calculate damage information”? Please, make it clearer.
Lines 41-42. What do you mean by “…the scientific of damage assessment…”? is it the objectivity of observations about damage detection and quantification? Again, please, explain more explicitly what you mean.
Line 45. The paper here stresses the fact that the proposed method remains a visual inspection (though improved by recent advances in integrated CV and AR) with all the related limitations (superficial inspection, no use of specific structural instruments, no testing of mechanical properties of the materials etc.). For more completeness, here the authors could mention the existence of other methods capable of testing the material properties on the basis of non-destructive and contact methods, like sonic and ultrasonic testing of structural members. An example can be found here:
· Polimeno, M. R., Roselli, I., Luprano, V., Mongelli, M., Tatì, A., De Canio, G. A non-destructive testing methodology for damage assessment of reinforced concrete buildings after seismic events. Engineering Structures 2018, 163, pp. 122-136, DOI: 10.1016/j.engstruct.2018.02.053
Figure 1. Some labels are not defined and/or remain unclear. What are the “system operating rules”? is the “worker” a human surveyor? What does “assessment grade division basis” mean?
In the following of the manuscript further corrections and improvements of the English are needed. This reviewer suggests considering an overall linguistic revision.
Lines 103-104: in this context the authors should mention the recent developments in the use of drones and other unmanned aerial vehicles (UAV). By now, a vast literature is available about the use of such technologies for rescue and survey purposes after natural catastrophes, including damage assessment after earthquakes. A few examples are as follows:
· Levine, N.M.; Spencer, B.F.Jr. Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework. Sensors 2022, 22, 873. https://doi.org/10.3390/s22030873
· Mongelli, M.; De Canio, G.; Roselli, I.; Malena, M.; Nacuzi, A.; de Felice, G. 3D Photogrammetric reconstruction by drone scanning for FE analysis and crack pattern mapping of the “Bridge of the Towers”, Spoleto. Key Engineering Materials 2017, 747, pp 423-430, ISSN: 1013-9826, DOI: 10.4028/www.scientific.net/KEM.747.423
Lines 223-225: what are the cited "Technical specifications for post-earthquake urgent assessment and repair of buildings", 2017? Related reference [25] should be more specific and detailed…
Lines 373-377: here some numbers in the text seemingly appear as possible typos or badly written references. Please, check and correct.
Lines 378-381: Figure 9 needs more explanation, and caption is too concise. Is it an example of what? Why 20 images? What are the differences between such 20 images? What’s the need of 20 images instead of only one? Etc.
Line 390: caption of Figure 10 is very unclear. Please, reformulate it, preferably by describing the content of the photo…
Figure 13 is very bad, unclear and unnecessary. What was this intended for? Anyway, the caption should be more explanatory, especially since the labels in the images are in Chinese!
Lines 429-430: Please, explain why images augmentation gave 4980 patches. If 227x227 is the patches size in pixels, state it more explicitly. Moreover, provide explanation of what patches are, what they are meant for, and reasons of their size.
Table 3 is not comprehensible. Please, consider changing the caption in “Classification of images collected in the laboratory experiment”. What are the numbers in the table? If they indicate the number of pictures used, state it more explicitly. In this case, for example, this reviewer suggests replacing the table columns headers as “Concrete state classes”, “Initial images”, “Processed patches”, “Training patches” and “Test patches” respectively.
Figure 14. what are the three green circles in “fully connected” box? What is the “Softmax” green rectangle? Is the final square image on the right a kind of output result? Please, explicitly clarify these doubts through more explanatory figure labels or in the caption.
Line 455. The validation process is not described much and remains quite vague. More details should be provided about how the results were validated. In general, validation is achieved by comparison with independent measurements or estimates obtained through a more consolidated and accurate method. What is such method in this study?
Figures 15-16. Please, provide labels to x and y axes.
Line 518. The authors state that the proposed method can evaluate the damage class in “real time”, but nowhere in the manuscript the computational time of the processes was assessed or discussed.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This paper presents research about an intelligent damage assessment approach for post-earthquake buildings damage evaluation using computer vision and augmented reality. In general, the paper seems to be interesting and within the scope of the Sustainability Journal. The Authors validated the accuracy of the method with numerical examples. However, there are some major revisions to be addressed by the Authors before proceeding further. Required revisions are listed as follows:
1. In the Abstract Section, please declare the meaning of the acronyms “CV”, “AR”, and “CNN”. Please keep in mind that acronyms must be declared before they are used in the text.
2. Please improve the Abstract section by declaring the main findings of this research. This may be declared by the end of the Abstract Section, this will make the paper stronger.
3. By the beginning of the Introduction Section, please increase the discussion about more methods to evaluate the damage and risk or reliability of structures considering the performance of them under the action of ground motions. In this sense, please consider the following papers in such a discussion:
a. https://doi.org/10.1002/(SICI)1096-9845(199701)26:1<79::AID-EQE624>3.0.CO;2-Y
b. https://doi.org/10.1016/j.engstruct.2004.05.017
c. https://doi.org/10.1016/j.engstruct.2014.09.047
d. https://doi.org/10.1016/j.engstruct.2017.03.076
e. https://doi.org/10.1016/j.strusafe.2018.12.002
4. In the Introduction Section, line 65, please declare the meaning of the acronym “IDEFO”.
5. Figures 1 and 2 must be moved to other section of the paper, these type of flowcharts are generally introduced by the end of the methodology section. Please consider moving such Figures from the Introduction Section to another one. In general terms, it is not very common to present these flowcharts in the Introduction Sections of papers. Please revise this.
6. In Section 2.1, lines 97-98, please change “Federal Emergency Administration of the United States (FEMA)” to “Federal Management Emergency Administration (FEMA) of the United States”.
7. In Section 2.1, line 99, please change “to satisfied” to “to satisfy”.
8. A list of acronyms must be provided by the end or at the beginning of the paper. There are a lot of acronyms in the body of the manuscript.
9. In Section 3, line 192, please change “twice times” to simply “twice”.
10. Reference “[25]” is only declared as “Technical Specification for Post-earthquake Urgent Assessment and Repair of Buildings. 2017.”. More information must be provided to locate such a document. Readers of the paper may want to explore more about this technical document.
11. It seems that the study presented in this paper is based only in Reinforced Concrete (RC) beams. Can the methodology be used for other structural elements (columns, slabs, etc.) or materials (steel, etc.)?
12. Please delete the black frame of Figure 5.
13. Please include the proper citation of each of the equations presented in the paper.
14. Please delete the black frame of Figure 6.
15. Please delete the black frame of Figure 8.
16. In Section 4.4, line 374, please change “damage34” to “damage”.
17. In Section 4.4, line 376, please change “methods36” to “methods”.
18. In Section 4.4, line 376, please change “CNN35” to “CNN”.
19. What is illustrated in Figure 9? Is a RC beam? If so, please declare it in the text.
20. As far as I understand, what is illustrated in Figure 10 is a RC beam. If so, could you please provide a sketch of the boundary conditions of the beam? In other words, the support conditions of the beam must be illustrated in the paper.
21. Figure 13 a and b are in Chinese, please change them to English.
22. Figures 15 and 16 are unitless? If so, please ignore this comment, if not, please include the units of vertical and horizontal axes, respectively, of such Figures.
23. It seems that the method presented in this paper is quite accurate. However, please provide more information about the effectiveness of the method in terms of computational time.
24. How far is this method from being included in guidelines of codes for the seismic retrofit of structures? What else must be done before this can be achieved?
25. In the References Section, please include the DOI of every cited paper.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
This paper presented an interesting study on the intelligent damage assessment of building structures subjected to strong earthquakes. Though the computer vision (CV) based method has been widely used in many practical cases, but the application of augmented reality (AR) technique has hardly been reported. This paper applied AR to the data collection, enriching the data required for damage assessment. In general, this paper is well structured. However, the following issues shall be addressed before publication:
Major Comments
1. What are the meanings of “damage degree” and “AR data” in figure 1? The first time that “AR data” occurred is in Line 184 “AR can supplement the virtual data, which is called AR data.” You have not explained the “AR data” clearly, please supply some description of “AR data”.
2. There are some grammatical errors, please revise.
One example is in Line 99:
“FEMA154 is proposed to satisfied the assessment needs of fast, reliable and simple calculation of damaged structures.” should be “FEMA154 is proposed to satisfy the assessment needs of a fast, reliable and simple calculation of damaged structures.”
3. In Section 4.1, you mentioned “The fused information includes the type information and the quantity information”. However, I think the information is obtained after data analysis and integration. Therefore, I think you should change “fusion information” to “fusion data”. The original “fusion data” is the input of damage assessment.
4. The format of the formula needs to be modified. Such as formula (3) in Line 292 is not centered, the formula (1) and (2) are not shown completely.
5. The expression in Line 348 “Convolutional Neural Networks (CNN) as the deep learning algorithm to complete intelligent damage assessment” should be “CNN as the deep learning algorithm to complete intelligent damage assessment”.
Minor Comments
I strongly recommend that the authors utilize some language editing or at least pay more attention of the format of the paper.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
The article proposes a method combining CV and AR technology for post-earthquake damage assessment, which is very interesting and innovative. This intelligent damage assessment of post-earthquake is very inspiring for seismic buildings retrofitting. However, there are still some problems in the article:
1. The “Discussion” part of the article is missing, some results of this article should be shown in “Discussion”. If you put “Discussion” in “Conclusion”, please explain it.
2. Some abbreviations are not fully spelled when first used. (Line 151: This method used modified Faster R-CNN…)
3. Some figures exist Chinese expression, and the picture that never exist in the paper are in some figures. Such as Fig.3, the picture of “Damage Assessment” is in Chinese and only occurs one time. Please revise these figures.
4. The Section 4.2 involves: Information Collection and Display of Post-earthquake Buildings”, the differences between “Information Collection” and “Information Display” are not mentioned in this paper. Please add some description about the differences between “Information Collection” and “Information Display”.
To sum up, this paper can be a reference for damage assessment of post-earthquake, although there some problems in this paper, we think this paper is suitable for publishing after revising.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
After revision by authors the manuscript still needs a few limited corrections. The following points still need correction:
point 14. At line 397 and in Figure 8 caption the authors use the term "sample" but they probably mean "example". Please, clarify or correct. Moreover, at line 399, please, replace "The figure 8 shown that," with "The figure 8 shows that".
point 16. The authors response is acceptable, but they should add it to the manuscript. For example, they could add at line 443 as follows: "Due to the still immature development of the system, the visualization of the information collection system is not of very good quality yet. Besides, as this is a Chinese software and the technology, only Chinese language is supported at present."
point 17. Also the authors response to this point is acceptable, but they did not add this explanation to the manuscript. Please, add it to the manuscript at line 453.
point 18. The header of the second columns should be corrected. Please, replace “Initial image” with “Initial images”. Plural here is important.
point 19. Again the authors response to this point is acceptable, but they did not add this explanation to the manuscript. Please, add it to the manuscript at line 463.
point 20. The authors should add explicitly their definition of validation (i.e as a comparison to CNN results etc.) to the manuscript at line 477.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 2 Report
The Authors have addressed all the recommendations of the Reviewers. The paper is now in a more suitable form to be published.
Author Response
Thank you for your advice.
