A Comparison of Deep Learning Algorithms for Anomaly Detection in Discrete Mechanical Systems
Abstract
:1. Introduction
2. System Description
Mathematical Model
3. Experimental Campaign
- Four TE triaxial capacitive MEMS accelerometers, one per storey;
- A PCB piezotronics impact hammer;
- A National Instruments c-DAQ.
4. Network Architecture and Training
4.1. Pre-Processing
4.2. Training and Test
4.3. Autoencoder
- Separable convolutional 1D layer: This layer applies 1D-convolutional windows separately to every channel. Then, it mixes the channel by point-wise multiplication. Indeed, the application of convolutional layers is proven to be particularly efficient in the analysis of time series [35].
- MaxPooling layer: A limitation of convolutional layers is that they greatly increase the number of parameters in the output tensors compared to the input ones; when many filters are involved, the magnitude of the tensors grows exponentially. For this reason, a pooling layer usually follows a convolutional one. Its purpose is to sub-sample the feature map by retaining only the most attractive information extracted by the convolutional layer. There are many possible pooling functions, but in this work, the MaxPooling function is adopted, which takes only the max value out of a predefined sub-matrix.
- Dropout layer: A common problem in the development of an ANN is overfitting; this occurs when a model learns from a particular random feature in the training data so that it is able to perfectly manage that set, but these learned concepts may not apply to new data, leading to poor performance. Dropout is a form of regularization, i.e., an approach that makes the network more robust in the training phase by forcing the network to learn general and recurrent patterns. During training, if a tensor passes through a dropout layer, some of its values are randomly dropped according to the dropout probability, i.e., the fraction of input’s elements whose value is set to zero. During testing, no values become zero, but the output is scaled by a factor equal to the dropout probability. Sometimes, the values are adjusted by the same fraction only in training to leave the test and prediction phases untouched. Some guidelines to manage the dropout layer can be found in [36].
- Dense layer: This is the simplest and most straight-forward type of layer that can be used. It is used to define the latent space. All the neurons in a dense layer are connected to all the neurons in the previous layer. Every connection is characterized by a weight, which multiplies the input value. The dense layer defines a bias b and an activation function f. If x is the input tensor, z is the output tensor, and W is the weights tensor, the mathematical equation of a dense layer is:
- Transposed convolutional 1D layer: This is a type of convolutional layer that can be used to increase the spatial resolution of an input tensor while maintaining a connectivity pattern that is compatible with some convolutional layer. It can be thought of as an operation that takes an input tensor and produces an output tensor with a larger spatial resolution. This operation is also called deconvolution.
4.4. Physics-Informed Neural Network
5. Results
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
SHM | Structural health monitoring |
RNN | Recurrent neural network |
NN | Neural network |
PINN | Physics-informed neural network |
CAE | Convolutional autoencoder |
MAE | Mean absolute error |
EMA | Experimental modal analysis |
FRF | Frequency response function |
PINN-CAE | Physics-informed convolutional autoencoder |
DD-CAE | Data-driven convolutional autoencoder |
References
- Song, G.; Wang, C.; Wang, B. Structural health monitoring (SHM) of civil structures. Appl. Sci. 2017, 7, 789. [Google Scholar] [CrossRef]
- Li, H.N.; Ren, L.; Jia, Z.G.; Yi, T.H.; Li, D.S. State-of-the-art in structural health monitoring of large and complex civil infrastructures. J. Civ. Struct. Health Monit. 2016, 6, 3–16. [Google Scholar] [CrossRef]
- Montalvao, D.; Maia, N.M.M.; Ribeiro, A.M.R. A review of vibration-based structural health monitoring with special emphasis on composite materials. Shock Vib. Dig. 2006, 38, 295–324. [Google Scholar] [CrossRef]
- Farrar, C.R.; Worden, K. Structural Health Monitoring: A Machine Learning Perspective; John Wiley & Sons: Hoboken, NJ, USA, 2012. [Google Scholar]
- Muc, A. Fuzzy approach in modeling static and fatigue strength of composite materials and structures. Neurocomputing 2020, 393, 156–164. [Google Scholar] [CrossRef]
- Wu, X.; Ghaboussi, J.; Garrett, J., Jr. Use of neural networks in detection of structural damage. Comput. Struct. 1992, 42, 649–659. [Google Scholar] [CrossRef]
- Salawu, O. Detection of structural damage through changes in frequency: A review. Eng. Struct. 1997, 19, 718–723. [Google Scholar] [CrossRef]
- Goyal, D.; Pabla, B. The vibration monitoring methods and signal processing techniques for structural health monitoring: A review. Arch. Comput. Methods Eng. 2016, 23, 585–594. [Google Scholar] [CrossRef]
- Fritzen, C.P. Vibration-based structural health monitoring–concepts and applications. Key Eng. Mater. 2005, 293, 3–20. [Google Scholar] [CrossRef]
- Magalhães, F.; Cunha, Á.; Caetano, E. Vibration based structural health monitoring of an arch bridge: From automated OMA to damage detection. Mech. Syst. Signal Process. 2012, 28, 212–228. [Google Scholar] [CrossRef]
- Deraemaeker, A.; Reynders, E.; De Roeck, G.; Kullaa, J. Vibration-based structural health monitoring using output-only measurements under changing environment. Mech. Syst. Signal Process. 2008, 22, 34–56. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Raissi, M.; Perdikaris, P.; Karniadakis, G.E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 2019, 378, 686–707. [Google Scholar] [CrossRef]
- Rastin, Z.; Ghodrati Amiri, G.; Darvishan, E. Unsupervised structural damage detection technique based on a deep convolutional autoencoder. Shock Vib. 2021, 2021, 6658575. [Google Scholar] [CrossRef]
- Rosafalco, L.; Manzoni, A.; Corigliano, A.; Mariani, S. A time series autoencoder for load identification via dimensionality reduction of sensor recordings. Eng. Proc. 2021, 2, 34. [Google Scholar]
- Ma, X.; Lin, Y.; Nie, Z.; Ma, H. Structural damage identification based on unsupervised feature-extraction via Variational Auto-encoder. Measurement 2020, 160, 107811. [Google Scholar] [CrossRef]
- Jian, X.; Zhong, H.; Xia, Y.; Sun, L. Faulty data detection and classification for bridge structural health monitoring via statistical and deep-learning approach. Struct. Control Health Monit. 2021, 28, e2824. [Google Scholar] [CrossRef]
- Do, J.S.; Kareem, A.B.; Hur, J.W. LSTM-Autoencoder for Vibration Anomaly Detection in Vertical Carousel Storage and Retrieval System (VCSRS). Sensors 2023, 23, 1009. [Google Scholar] [CrossRef]
- Finotti, R.P.; Barbosa, F.d.S.; Cury, A.A.; Pimentel, R.L. Novelty Detection Using Sparse Auto-Encoders to Characterize Structural Vibration Responses. Arab. J. Sci. Eng. 2022, 47, 13049–13062. [Google Scholar] [CrossRef]
- Jimenez Martinez, M.; Alfaro Ponce, M. Effects of synthetic data applied to artificial neural networks for fatigue life prediction in nodular cast iron. J. Braz. Soc. Mech. Sci. Eng. 2021, 43, 10. [Google Scholar] [CrossRef]
- Raissi, M.; Wang, Z.; Triantafyllou, M.S.; Karniadakis, G.E. Deep learning of vortex-induced vibrations. J. Fluid Mech. 2019, 861, 119–137. [Google Scholar] [CrossRef]
- Cai, S.; Mao, Z.; Wang, Z.; Yin, M.; Karniadakis, G.E. Physics-informed neural networks (PINNs) for fluid mechanics: A review. Acta Mech. Sin. 2021, 37, 1727–1738. [Google Scholar] [CrossRef]
- Yucesan, Y.; Viana, F. A Physics-informed Neural Network for Wind Turbine Main Bearing Fatigue. Int. J. Progn. Health Manag. 2020, 11, 17. [Google Scholar]
- Huang, B.; Wang, J. Applications of Physics-Informed Neural Networks in Power Systems—A Review. IEEE Trans. Power Syst. 2023, 38, 572–588. [Google Scholar] [CrossRef]
- Di Carlo, S.; Benedetti, L.; Di Gialleonardo, E. Teaching by active learning: A laboratory experience on fundamentals of vibrations. Int. J. Mech. Eng. Educ. 2022, 50, 869–882. [Google Scholar] [CrossRef]
- Cheli, F.; Diana, G. Advanced Dynamics of Mechanical Systems; Springer: Berlin/Heidelberg, Germany, 2015; Volume 2020. [Google Scholar]
- Ghali, A.; Neville, A.M.; Brown, T.G. Structural Analysis; CRC Press: Boca Raton, FL, USA, 2017. [Google Scholar] [CrossRef]
- Liu, M.; Gorman, D.G. Formulation of Rayleigh damping and its extensions. Comput. Struct. 1995, 57, 277–285. [Google Scholar] [CrossRef]
- Schwarz, B.J.; Richardson, M.H. Experimental modal analysis. CSI Reliab. Week 1999, 35, 1–12. [Google Scholar]
- Cunha, Á.; Caetano, E. Experimental modal analysis of civil engineering structures. Sound Vib. 2006, 40. [Google Scholar]
- Hajela, P.; Soeiro, F. Structural damage detection based on static and modal analysis. AIAA J. 1990, 28, 1110–1115. [Google Scholar] [CrossRef]
- Bono, F.; Radicioni, L.; Cinquemani, S. A novel approach for quality control of automated production lines working under highly inconsistent conditions. Eng. Appl. Artif. Intell. 2023, 122, 106149. [Google Scholar] [CrossRef]
- Zhang, Y. A better autoencoder for image: Convolutional autoencoder. In Proceedings of the ICONIP17-DCEC, 2018, Guangzhou, China, 14–18 October 2017; Available online: http://users.cecs.anu.edu.au/Tom.Gedeon/conf/ABCs2018/paper/ABCs2018_paper_58.pdf (accessed on 23 March 2017).
- Li, Z.; Liu, F.; Yang, W.; Peng, S.; Zhou, J. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. 2021, 33, 6999–7019. [Google Scholar] [CrossRef]
- Zhang, C.; Song, D.; Chen, Y.; Feng, X.; Lumezanu, C.; Cheng, W.; Ni, J.; Zong, B.; Chen, H.; Chawla, N.V. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI Conference on Artificial intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33, pp. 1409–1416. [Google Scholar]
- Srivastava, N. Improving neural networks with dropout. Univ. Tor. 2013, 182, 7. [Google Scholar]
- Qi, J.; Du, J.; Siniscalchi, S.M.; Ma, X.; Lee, C.H. On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression. IEEE Signal Process. Lett. 2020, 27, 1485–1489. [Google Scholar] [CrossRef]
- Bono, F.M.; Radicioni, L.; Cinquemani, S.; Benedetti, L.; Cazzulani, G.; Somaschini, C.; Belloli, M. A Deep Learning Approach to Detect Failures in Bridges Based on the Coherence of Signals. Future Internet 2023, 15, 119. [Google Scholar] [CrossRef]
- Karniadakis, G.E.; Kevrekidis, I.G.; Lu, L.; Perdikaris, P.; Wang, S.; Yang, L. Physics-informed machine learning. Nat. Rev. Phys. 2021, 3, 422–440. [Google Scholar] [CrossRef]
- Cuomo, S.; Di Cola, V.S.; Giampaolo, F.; Rozza, G.; Raissi, M.; Piccialli, F. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. J. Sci. Comput. 2022, 92, 88. [Google Scholar] [CrossRef]
- Davis, P.J.; Rabinowitz, P. Methods of Numerical Integration; Courier Corporation: North Chelmsford, MA, USA, 2007. [Google Scholar]
- Brandt, A.; Brincker, R. Integrating time signals in frequency domain—Comparison with time domain integration. Measurement 2014, 58, 511–519. [Google Scholar] [CrossRef]
- Doebling, S.W.; Farrar, C.R.; Prime, M.B.; Shevitz, D.W. Damage Identification and Health Monitoring of Structural and Mechanical Systems from Changes in Their Vibration Characteristics: A Literature Review; U.S. Department of Energy Office of Scientific and Technical Information: Washington, WA, USA, 1996. [Google Scholar]
Storeys | |
---|---|
Area | mm2 |
Thickness | 20 mm |
Mass | 2.26 kg |
Pillars | |
Area | mm2 |
Length | 180 mm |
Thickness | Negligible |
Mass | Negligible |
0.03 | |
0.028 |
Mode | Numerical Model (Hz) | Experimental Model (Hz) |
---|---|---|
1 | 0.79 | 0.75 |
2 | 2.51 | 2.41 |
3 | 3.88 | 3.74 |
4 | 5.01 | 5.04 |
Damage Percentage | Length |
---|---|
180.0 mm | |
186.5 mm | |
194.0 mm | |
203.0 mm | |
213.5 mm | |
227.0 mm | |
244.0 mm |
Damage | Accuracy | |
---|---|---|
Percentage | DD-CAE | PINN-CAE |
−10% | 33.19% | 79.43% |
−20% | 40.20% | 82.81% |
−30% | 52.24% | 87.22% |
−40% | 65.11% | 92.03% |
−50% | 84.61% | 100% |
−60% | 100% | 100% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bono, F.M.; Radicioni, L.; Cinquemani, S.; Bombaci, G. A Comparison of Deep Learning Algorithms for Anomaly Detection in Discrete Mechanical Systems. Appl. Sci. 2023, 13, 5683. https://doi.org/10.3390/app13095683
Bono FM, Radicioni L, Cinquemani S, Bombaci G. A Comparison of Deep Learning Algorithms for Anomaly Detection in Discrete Mechanical Systems. Applied Sciences. 2023; 13(9):5683. https://doi.org/10.3390/app13095683
Chicago/Turabian StyleBono, Francesco Morgan, Luca Radicioni, Simone Cinquemani, and Gianluca Bombaci. 2023. "A Comparison of Deep Learning Algorithms for Anomaly Detection in Discrete Mechanical Systems" Applied Sciences 13, no. 9: 5683. https://doi.org/10.3390/app13095683
APA StyleBono, F. M., Radicioni, L., Cinquemani, S., & Bombaci, G. (2023). A Comparison of Deep Learning Algorithms for Anomaly Detection in Discrete Mechanical Systems. Applied Sciences, 13(9), 5683. https://doi.org/10.3390/app13095683