Enhancing Recovery of Structural Health Monitoring Data Using CNN Combined with GRU

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This study combines CNN and GRU networks, achieving better data reconstruction results compared to using a single CNN or GRU network. The proposed approach is feasible but still immature in model training, as it does not follow the basic common standard for dividing datasets in contemporary neural network model training: training set, validation set, and test set. In addition to these, there are other methods for splitting datasets to avoid data confusion and data leakage, which can falsely enhance the model's test performance. The authors can refer to neural network training paradigms, re-divide the datasets, and conduct training and testing. It is believed that similar conclusions will be reached, but with greater persuasiveness.

Here are the questions directed to the authors:

1. Could you replace Figure 3 with an architecture diagram that clearly illustrates the specific layer architecture and model parameters of the CNN-GRU model used in the paper?

2. The image in Figure 6 is too blurry. Can you provide a high-resolution version?

3. In Section 3.3, the model training did not follow the standard division of training set, validation set, and test set. How can you ensure that the model does not overfit? Please provide a reasonable explanation.

4. According to Figure 7, the model trained in this paper only uses the training set and validation set, without an independent test set. This means all data are involved in training. Please explain this approach. It is recommended to retrain using a reasonable dataset division method to avoid errors caused by data confusion, thus better evaluating the model and strengthening the credibility of the conclusions.

5. The readability of Figure 11 is poor. It is recommended to increase the image resolution and enlarge the font size.

Author Response

Comment 1: Could you replace Figure 3 with an architecture diagram that clearly illustrates the specific layer architecture and model parameters of the CNN-GRU model used in the paper?

Response 1: Thank you for your comment. Figure 3 illustrates the implementation steps to help readers visualize the process. These steps include data preparation, passing the data through layers, and obtaining the output. The specific parameters of the network are presented in the case study.

Comment 2: The image in Figure 6 is too blurry. Can you provide a high-resolution version?

Response 2: Thank you for your comment. The authors revised changed the figure with a high-resolution

Comment 3: In Section 3.3, the model training did not follow the standard division of training set, validation set, and test set. How can you ensure that the model does not overfit? Please provide a reasonable explanation.

Response 3: Thank you for your comment. In practice, data is typically split into 70/15/15 proportions, where 70% of the data is used for training, 15% for validation, and 15% for testing. In this study, testing is conducted using a new dataset (the full dataset includes about 40 minutes of measurement, with the first 20 minutes used for training and validation, and the remaining 20 minutes for testing). This approach is used because a large amount of data is necessary for modal analysis. The results from the testing through modal analysis show that the recovered data still maintains accurate results, indicating that no overfitting occurred. The authors provided more detail from line 237-240

Comment 4: According to Figure 7, the model trained in this paper only uses the training set and validation set, without an independent test set. This means all data are involved in training. Please explain this approach. It is recommended to retrain using a reasonable dataset division method to avoid errors caused by data confusion, thus better evaluating the model and strengthening the credibility of the conclusions.

Response 4: Thank you for your comment. Please see the explain above

Comment 5: The readability of Figure 11 is poor. It is recommended to increase the image resolution and enlarge the font size.

Response 5: Thank you for your comment. The authors revised changed the figure with a high-resolution and enlarge the font size.

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript proposes a method to reliably reconstruct time series data with errors or missing values in structural health monitoring using a CNN-GRU model. The method performs well on real bridge monitoring data, effectively handling transient vibrations caused by vehicles passing over the bridge. However, the manuscript lacks a detailed description of how the proposed method handles missing or abnormal data, which affects the comprehensive evaluation of the method's performance. Below are my specific comments:

1.      L40 "These sensors are permanently installed on the bridge structure and collect critical signals such as vibrations, displacements, deformations, or tilting.... [5–8]", It is unclear whether the ellipsis (....) is intentional or a typo. Please check and correct the sentence for clarity and proper punctuation.

2.      There is inconsistency in the use of singular and plural forms of terms in L16 "Convolutional Neural Network (CNN)" and Line 112: "Convolutional Neural Network (CNNs)", others like CNN, RNN, GRU, and LSTM have the same error. Please standardize the terminology throughout the manuscript to ensure consistency.

3.      The description of CNN in Section 2 should be optimized to reflect the specific choices made in this study. For example, CNN activation functions include ReLU, Sigmoid, Softmax, etc., and pooling layers include MaxPooling, AveragePooling, etc. These choices should be made based on the specific requirements of the task.

4.      References 36-38 in Section 2.1 are primarily focused on image processing using CNN, while this study deals with time series data. Similarly, references 40 and 42 for GRU are not entirely relevant.

5.      L231 The acronym PCB in "PCB accelerometer sensors" should be expanded to its full term on first use.

6.      The inconsistency in font styles and sizes in the figures affects readability. Please ensure that all figures use a consistent font style and size.

7.      Figure 7, subplot (b): The y-axis label and units are missing. Similar issues are present in Figures 8 and 10.

8.      Figure 9 presents four modes of the bridge in 3D. However, there is a discrepancy with the earlier statement in L252: "One sensor will be assumed……", it is unclear why more than one point differs from the original position in Figure 9. Additionally, the error rate in Table 1 needs clarification. The Modal Assurance Criterion is typically used to assess the similarity between two modal vectors. Please specify which two modal vectors are being compared and how the error rate is calculated.

9.      Section 3.3 uses the same dataset as the previous sections. However, it is unclear how problematic observation sequences are defined, and how error points are identified. The manuscript lacks a detailed analysis of all time series, including the identification and labeling of missing and abnormal values. It is essential to provide a clear definition of problematic sequences and error points, along with a thorough analysis of all time series. The method for identifying and correcting missing and abnormal values should also be described in detail.

Author Response

Comment 1: L40 "These sensors are permanently installed on the bridge structure and collect critical signals such as vibrations, displacements, deformations, or tilting.... [5–8]", It is unclear whether the ellipsis (....) is intentional or a typo. Please check and correct the sentence for clarity and proper punctuation.

Response 1: Thank you for your comment. The authors revised: replace (…) by “and among others”

Comment 2: There is inconsistency in the use of singular and plural forms of terms in L16 "Convolutional Neural Network (CNN)" and Line 112: "Convolutional Neural Network (CNNs)", others like CNN, RNN, GRU, and LSTM have the same error. Please standardize the terminology throughout the manuscript to ensure consistency.

Response 2: Thank you for your comment. The authors revised in the new version

Comment 3: The description of CNN in Section 2 should be optimized to reflect the specific choices made in this study. For example, CNN activation functions include ReLU, Sigmoid, Softmax, etc., and pooling layers include MaxPooling, AveragePooling, etc. These choices should be made based on the specific requirements of the task.

Response 3: Thank you for your comment. The authors revised in the new version. Please see in section 2 from line 128-130 and 131-135

Comment 4: References 36-38 in Section 2.1 are primarily focused on image processing using CNN, while this study deals with time series data. Similarly, references 40 and 42 for GRU are not entirely relevant.

Response 4: Thank you for your comment. These references are used to cite formulas that have been cited in the article.

Comment 5: L231 The acronym PCB in "PCB accelerometer sensors" should be expanded to its full term on first use.

Response 5: Thank you for your comment. PCB is the name of the sensor. The authors added more information in line 232, added reference relate to this sensor

Comment 6: The inconsistency in font styles and sizes in the figures affects readability. Please ensure that all figures use a consistent font style and size.

Response 6: Thank you for your comment. The authors revised the figures

Comment 7: Figure 7, subplot (b): The y-axis label and units are missing. Similar issues are present in Figures 8 and 10.

Response 7: Thank you for your comment. The authors revised the figures

Comment 8: Figure 9 presents four modes of the bridge in 3D. However, there is a discrepancy with the earlier statement in L252: "One sensor will be assumed……", it is unclear why more than one point differs from the original position in Figure 9. Additionally, the error rate in Table 1 needs clarification. The Modal Assurance Criterion is typically used to assess the similarity between two modal vectors. Please specify which two modal vectors are being compared and how the error rate is calculated.

Response 8: Thank you for your comment. In this study, each data location will be used to recover the data using the proposed method. The changes in each training are the input and output of the proposed network. The authors revised in table 1. The MAC value ensures the mode shape similarity between the real data and the recovered data.

Comment 9: Section 3.3 uses the same dataset as the previous sections. However, it is unclear how problematic observation sequences are defined, and how error points are identified. The manuscript lacks a detailed analysis of all time series, including the identification and labeling of missing and abnormal values. It is essential to provide a clear definition of problematic sequences and error points, along with a thorough analysis of all time series. The method for identifying and correcting missing and abnormal values should also be described in detail.

Response 9: Thank you for your comment. The authors revised. Please see the line 265-266 and 363-364

 

Reviewer 3 Report

Comments and Suggestions for Authors

The reviewed paper is devoted to increasing the efficiency of bridge structural health monitoring systems by using artificial intelligence (AI) tools. The main idea of the study is the use AI tools to address the problem of data corruption during long-term monitoring. The primary innovation of the study is the suggestion to use of a combination of two AI models (CNN and GRU) for data recovery. It is significant to note that a substantial part of the paper is dedicated to a case study of tool implementation on the existing bridge, underscoring the practical relevance of this research. Based on the reference list and introduction paragraphs, it is evident that the authors have conducted a fairly comprehensive review of the latest studies and existing tools. The structure of the paper ensures understanding and logical coherence. In addition, the terminology used is quite simple and allows not only specialists to understand the essence of the study.

However, there are several comments on the conceptual ideas for the authors:
1. It is essential to clarify whether employing the term "recovery" accurately reflects the actions undertaken in this context. It appears that the output data may be better described as "predictive values" or "predictive indicators" at specific points based on other (serviceable) physical signals. This terminological distinction is crucial due to varying reliability values of the physical signal data and "predictive values".
2. In connection with what was written in comment 1, it would be beneficial to specify in the paper the main task for which the proposed method could be applicable (i.e., what purpose the bridge monitoring system serves). Based on the submitted paper, the application of the method seems promising for addressing tasks related to determining quantitative parameters of bridge loading modes (such as the number and amplitudes of strain ranges, statistics of frequencies, and others). However, it appears there are some limitations regarding whether the suggested method can effectively "recover" anomalies in signals (related to some damages, for example). Additionally, it seems obvious that anomalies will be qualitatively detected using serviceable sensors located nearby, which would already indicate potential damages.

In addition, there are several comments on specific part of the paper text:
3. It is recommended to include a detailed description of the parameters of data recording. Specifically, information about the duration of monitoring records, sensor frequency, and the total volume of analyzed data would be useful.
4. Clarification regarding how two measuring networks comprising 8 sensors each (totaling 16 sensors) were positioned across 14 measurement points is recommended (see Line 235).
5. It is recommended to specify what exact data was input into the first CNN network. At the basic level of understanding neural networks, it appears that CNNs typically process raster formats (graphics), while monitoring signals are generally represented as time-dependent data.

In summary, this paper presents itself as a high-quality contribution within an innovative research field characterized by clear methodology and well-supported conclusions.

Author Response

Comment 1: It is essential to clarify whether employing the term "recovery" accurately reflects the actions undertaken in this context. It appears that the output data may be better described as "predictive values" or "predictive indicators" at specific points based on other (serviceable) physical signals. This terminological distinction is crucial due to varying reliability values of the physical signal data and "predictive values".

Response 1: Thank you for your comment. In this study, the authors use the term recovery because in fact, the study serves to reconstruct the lost data of the faulty sensors. The term “prediction” also has the same effect in this case, but it is on a broader scale. The prediction can be done even without the need for data from the old sensor but from a location where the model is wanted. Therefore, the authors use the term “recovery”
Comment 2: In connection with what was written in comment 1, it would be beneficial to specify in the paper the main task for which the proposed method could be applicable (i.e., what purpose the bridge monitoring system serves). Based on the submitted paper, the application of the method seems promising for addressing tasks related to determining quantitative parameters of bridge loading modes (such as the number and amplitudes of strain ranges, statistics of frequencies, and others). However, it appears there are some limitations regarding whether the suggested method can effectively "recover" anomalies in signals (related to some damages, for example). Additionally, it seems obvious that anomalies will be qualitatively detected using serviceable sensors located nearby, which would already indicate potential damages.

Response 2: Thank you for your comment. The authors revised and added more information from line 108-112
Comment 3: It is recommended to include a detailed description of the parameters of data recording. Specifically, information about the duration of monitoring records, sensor frequency, and the total volume of analyzed data would be useful.

Response 3: Thank you for your comment. The authors revised and added more information from line 244-247
Comment 4: Clarification regarding how two measuring networks comprising 8 sensors each (totaling 16 sensors) were positioned across 14 measurement points is recommended (see Line 235).

Response 4: Thank you for your comment. The authors revised and added more information from line 247-250

Comment 5. It is recommended to specify what exact data was input into the first CNN network. At the basic level of understanding neural networks, it appears that CNNs typically process raster formats (graphics), while monitoring signals are generally represented as time-dependent data.

Response 5: Thank you for your comment. The authors revised and added more information from line 192-194

 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The paper is accepted without any further changes.

Author Response

The authors thank you for your suggestions, which helped improve the manuscript.

 

Reviewer 2 Report

Comments and Suggestions for Authors

1.       In Equation (1), the symbol "∙" between "X(i+m, j+n)" and "W(m, n)" is used. Could the authors please clarify whether this symbol represents a multiplication operation?

2.       There is an inconsistency in the abbreviation of "Gated Recurrent Units (GRU)". In L16, it is written as "Gated Recurrent Units(GRU)", while in Line 146, it is written as "Gated Recurrent Units (GRUs)".

3.       In Figure 2, the reset gate part on the left side should be connected to the hidden state input.

4.       In Figure 9, there is still confusion regarding the statement "One sensor will be assumed to malfunction, and its values will be set to zero." It is unclear whether this refers to setting the values of the malfunctioning sensor to zero in the input data or in the output data. Based on Figure 8, it seems more likely that the input data values are being set to zero. Additionally, it is not clear what data are used as input to the anomaly detection model—data from other sensors or additional data sources. Please clarify these points in the manuscript to avoid ambiguity.

Author Response

Comment 1: In Equation (1), the symbol "∙" between "X(i+m, j+n)" and "W(m, n)" is used. Could the authors please clarify whether this symbol represents a multiplication operation?

Response 1: Thank you for your comment. The authors revised from line 126-127

Comment 2: There is an inconsistency in the abbreviation of "Gated Recurrent Units (GRU)". In L16, it is written as "Gated Recurrent Units(GRU)", while in Line 146, it is written as "Gated Recurrent Units (GRUs)".

Response 2: Thank you for your comment. The authors revised: Gated Recurrent Units(GRUs) in line 16-17

Comment 3: In Figure 2, the reset gate part on the left side should be connected to the hidden state input.

Response 3: Thank you for your comment. The authors revised Figure 2

Comment 4: In Figure 9, there is still confusion regarding the statement "One sensor will be assumed to malfunction, and its values will be set to zero." It is unclear whether this refers to setting the values of the malfunctioning sensor to zero in the input data or in the output data. Based on Figure 8, it seems more likely that the input data values are being set to zero. Additionally, it is not clear what data are used as input to the anomaly detection model—data from other sensors or additional data sources. Please clarify these points in the manuscript to avoid ambiguity.

Response 4: Thank you for your comment. The authors added more information. Please see from line 274-279

The input data are from the normal sensors; the output data are from the sensor that we assume is faulty.

Figure 8 shows the actual data and the recovered data for accuracy comparison.