Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework
Abstract
:1. Introduction
2. Methodology
2.1. Pre-Earthquake
2.1.1. Preliminary Building Assessment
2.1.2. BIM Development
2.1.3. UAV Survey and 3D Reconstruction
2.2. Post-Earthquake
2.2.1. BIM-Guided Automatic Image Selection by Component
2.2.2. BIM-Guided Component Identification in 2D Images
2.2.3. Damage Detection in 3D Point Clouds
3. Example 1: BIM-Based Digital Twin Development for a Reinforced Concrete Moment Frame Building
3.1. Building Description
3.2. Preliminary Building Assessment
3.3. BIM Development, UAV Survey and 3D Reconstruction
3.4. Results and Discussion
3.4.1. BIM-Guided Image Selection by Component
3.4.2. BIM-Guided Component Identification
4. Example 2: 3D Change Detection for a Synthetic Earthquake-Damaged Masonry Veneer Wall
4.1. Graphics Model Description
4.2. Point Cloud Generation and Pre-Processing
4.3. Results and Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Applied Technology Council (ATC). ATC 20 Procedures for Postearthquake Safety Evaluation of Buildings; Applied Technology Council (ATC): Redwood City, CA, USA, 1989. [Google Scholar]
- Applied Technology Council (ATC). ATC 20-2 Addendum to the ATC-20 Postearthquake Building Safety Evaluation Procedures; Applied Technology Council (ATC): Redwood City, CA, USA, 1995. [Google Scholar]
- Earthquake Engineering Research Institute (EERI). EERI Special Earthquake Report: The M 6.3 Christchurch, New Zealand, Earthquake of February 22, 2011; EERI: Oakland, CA, USA, 2011. [Google Scholar]
- Chock, G. ATC-20 Post-Earthquake Building Safety Evaluations Performed after the October 15, 2006 Hawaii Earthquakes, Summary and Recommendations for Improvements (Updated). Available online: https://www.eeri.org/images/archived/wp-content/uploads/United%20States/SEAOH_EarthquakeSafetyEvalRec.pdf (accessed on 5 May 2020).
- City and County of San Francisco. BORP Guidelines for Engineers. Available online: https://sfdbi.org/borp (accessed on 5 May 2020).
- The Building Occupancy Resumption Program (BORP). Available online: https://www.cityofberkeley.info/BORP/ (accessed on 27 October 2021).
- Lorenz, T.R. 20 Years after Northridge Earthquake Be Ready with Glendale’s Back to Business Program. Available online: https://www.glendaleca.gov/Home/Components/News/News/8/2129?arch=1 (accessed on 27 October 2021).
- Spencer, B.F.; Hoskere, V.; Narazaki, Y. Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring. Engineering 2019, 5, 199–222. [Google Scholar] [CrossRef]
- Narazaki, Y.; Hoskere, V.; Yoshida, K.; Spencer, B.F.; Fujino, Y. Synthetic environments for vision-based structural condition assessment of Japanese high-speed railway viaducts. Mech. Syst. Signal Process. 2021, 160, 107850. [Google Scholar] [CrossRef]
- Zhu, Z.; Brilakis, I. Concrete column recognition in images and videos. J. Comput. Civ. Eng. 2010, 24, 478–487. [Google Scholar] [CrossRef]
- Zhu, Z.; German, S.; Brilakis, I. Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation. Autom. Constr. 2011, 20, 874–883. [Google Scholar] [CrossRef]
- German, S.; Brilakis, I.; Desroches, R. Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments. Adv. Eng. Inform. 2012, 26, 846–858. [Google Scholar] [CrossRef]
- German, S.; Jeon, J.S.; Zhu, Z.; Bearman, C.; Brilakis, I.; Desroches, R.; Lowes, L. Machine vision-enhanced postearthquake inspection. J. Comput. Civ. Eng. 2013, 27, 622–634. [Google Scholar] [CrossRef]
- Paal, S.G.; Jeon, J.S.; Brilakis, I.; DesRoches, R. Automated damage index estimation of reinforced concrete columns for post-earthquake evaluations. J. Struct. Eng. 2015, 141, 04014228. [Google Scholar] [CrossRef] [Green Version]
- Narazaki, Y.; Hoskere, V.; Hoang, T.A.; Fujino, Y.; Sakurai, A.; Spencer, B.F. Vision-based automated bridge component recognition with high-level scene consistency. Comput.-Aided Civ. Infrastruct. Eng. 2020, 35, 465–482. [Google Scholar] [CrossRef]
- Hoskere, V.; Narazaki, Y.; Hoang, T.; Spencer, B.F. Vision-based Structural Inspection using Multiscale Deep Convolutional Neural Networks. In Proceedings of the 3rd Huixian International Forum on Earthquake Engineering for Young Researchers, Urbana, IL, USA, 11–12 August 2017. [Google Scholar]
- Hoskere, V.; Narazaki, Y.; Hoang, T.A.; Spencer, B.F. Towards Automated Post-Earthquake Inspections with Deep Learning-based Condition-Aware Models. In Proceedings of the 7th World Conference on Structural Control and Monitoring, Qingdao, China, 22–25 July 2018; Volume 10. [Google Scholar]
- Hoskere, V.; Narazaki, Y.; Hoang, T.A.; Spencer, B.F. MaDnet: Multi-task semantic segmentation of multiple types of structural materials and damage in images of civil infrastructure. J. Civ. Struct. Health Monit. 2020, 10, 757–773. [Google Scholar] [CrossRef]
- Yeum, C.M.; Dyke, S.J.; Ramirez, J. Visual data classification in post-event building reconnaissance. Eng. Struct. 2018, 155, 16–24. [Google Scholar] [CrossRef]
- Hüthwohl, P.; Lu, R.; Brilakis, I. Multi-classifier for reinforced concrete bridge defects. Autom. Constr. 2019, 105, 102824. [Google Scholar] [CrossRef]
- Mitrani-Reiser, J.; Wu, S.; Beck, J.L. Virtual Inspector and its application to immediate pre-event and post-event earthquake loss and safety assessment of buildings. Nat. Hazards 2016, 81, 1861–1878. [Google Scholar] [CrossRef]
- Burton, H.V.; Sreekumar, S.; Sharma, M.; Sun, H. Estimating aftershock collapse vulnerability using mainshock intensity, structural response and physical damage indicators. Struct. Saf. 2017, 68, 85–96. [Google Scholar] [CrossRef]
- Burton, H.V.; Deierlein, G.G. Integrating visual damage simulation, virtual inspection, and collapse capacity to evaluate post-earthquake structural safety of buildings. Earthq. Eng. Struct. Dyn. 2018, 47, 294–310. [Google Scholar] [CrossRef]
- Zhang, Y.; Burton, H.V.; Sun, H.; Shokrabadi, M. A machine learning framework for assessing post-earthquake structural safety. Struct. Saf. 2018, 72, 1–16. [Google Scholar] [CrossRef]
- Zhang, Y.; Burton, H.V. Pattern recognition approach to assess the residual structural capacity of damaged tall buildings. Struct. Saf. 2019, 78, 12–22. [Google Scholar] [CrossRef]
- Golparvar-Fard, M.; Peña-Mora, F.; Savarese, S. Integrated sequential as-built and as-planned representation with D 4AR tools in support of decision-making tasks in the AEC/FM industry. J. Constr. Eng. Manag. 2011, 137, 1099–1116. [Google Scholar] [CrossRef]
- Karsch, K.; Golparvar-Fard, M.; Forsyth, D. ConstructAide: Analyzing and visualizing construction sites through photographs and building models. ACM Trans. Graph. 2014, 33, 1–11. [Google Scholar] [CrossRef]
- Golparvar-Fard, M.; Peña-Mora, F.; Savarese, S. Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models. J. Comput. Civ. Eng. 2015, 29, 04014025. [Google Scholar] [CrossRef]
- Han, K.K.; Golparvar-Fard, M. Potential of big visual data and building information modeling for construction performance analytics: An exploratory study. Autom. Constr. 2017, 73, 184–198. [Google Scholar] [CrossRef] [Green Version]
- Glaessgen, E.; Stargel, D. The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles. In Proceedings of the 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, Honolulu, HI, USA, 23–26 April 2012. [Google Scholar] [CrossRef] [Green Version]
- Greengard, S. Digital Twins Grow Up. Available online: https://cacm.acm.org/news/238642-digital-twins-grow-up/fulltext (accessed on 28 October 2021).
- Boschert, S.; Rosen, R. Digital twin—The simulation aspect. In Mechatronic Futures; Springer: Berlin/Heidelberg, Germany, 2016; pp. 59–74. [Google Scholar] [CrossRef]
- Hughes, A.J.; Barthorpe, R.J.; Dervilis, N.; Farrar, C.R.; Worden, K. A probabilistic risk-based decision framework for structural health monitoring. Mech. Syst. Signal Process. 2021, 150, 107339. [Google Scholar] [CrossRef]
- Zhu, Y.C.; Wagg, D.; Cross, E.; Barthorpe, R. Real-Time Digital Twin Updating Strategy Based on Structural Health Monitoring Systems. In Model Validation and Uncertainty Quantification, Volume 3; Springer: Berlin/Heidelberg, Germany, 2020; pp. 55–64. [Google Scholar]
- Gardner, P.; Dal Borgo, M.; Ruffini, V.; Zhu, Y.; Hughes, A. Towards the Development of a Digital Twin for Structural Dynamics Applications. In Model Validation and Uncertainty Quantification, Volume 3; Springer: Berlin/Heidelberg, Germany, 2020; pp. 165–179. [Google Scholar]
- Angjeliu, G.; Coronelli, D.; Cardani, G. Development of the simulation model for Digital Twin applications in historical masonry buildings: The integration between numerical and experimental reality. Comput. Struct. 2020, 238, 106282. [Google Scholar] [CrossRef]
- Lin, K.; Xu, Y.L.; Lu, X.; Guan, Z.; Li, J. Digital twin-based collapse fragility assessment of a long-span cable-stayed bridge under strong earthquakes. Autom. Constr. 2021, 123, 103547. [Google Scholar] [CrossRef]
- Applied Technology Council (ATC). FEMA P-58-1 Seismic Performance Assessment of Buildings Volume 1—Methodology, 2nd ed.; Number September; Federal Emergency Management Agency: Redwood City, CA, USA, 2018.
- United States General Services Administration. Level of Detail. Available online: https://www.gsa.gov/real-estate/design-construction/3d4d-building-information-modeling/guidelines-for-bim-software/document-guides/level-of-detail (accessed on 5 January 2022).
- Revit 2019. Available online: https://www.autodesk.com/ (accessed on 3 October 2021).
- ISO 16739; Industry Foundation Classes (IFC) for Data Sharing in the Construction and Facility Management Industries. ISO: Geneva, Switzerland, 2013.
- Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision, 2nd ed.; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar] [CrossRef] [Green Version]
- Furukawa, Y.; Hernández, C. Multi-View Stereo: A Tutorial. In Foundations and Trends in Computer Graphics and Vision; Now Publishers, Inc.: Boston, MA, USA, 2013; Volume 9. [Google Scholar]
- Morgenthal, G.; Hallermann, N.; Kersten, J.; Taraben, J.; Debus, P.; Helmrich, M.; Rodehorst, V. Framework for automated UAS-based structural condition assessment of bridges. Autom. Constr. 2019, 97, 77–95. [Google Scholar] [CrossRef]
- Ibrahim, A.; Golparvar-Fard, M.; Bretl, T.; El-Rayes, K. Model-driven visual data capture on construction sites: Method and metrics of success. In Proceedings of the ASCE International Workshop on Computing in Civil Engineering 2017, Seattle, WA, USA, 25–27 June 2017; pp. 109–116. [Google Scholar] [CrossRef]
- Ibrahim, A.; Golparvar-Fard, M.; El-Rayes, K. Metrics and methods for evaluating model-driven reality capture plans. Comput.-Aided Civ. Infrastruct. Eng. 2022, 37, 55–72. [Google Scholar] [CrossRef]
- Krijnen, T. IfcOpenShell 0.6.0. Available online: http://ifcopenshell.org/ (accessed on 27 April 2021).
- Zhou, Q.Y.; Park, J.; Koltun, V. Open3D: A Modern Library for 3D Data Processing. arXiv 2018, arXiv:1801.09847. [Google Scholar]
- Stent, S.; Gherardi, R.; Stenger, B.; Soga, K.; Cipolla, R. Visual change detection on tunnel linings. Mach. Vis. Appl. 2016, 27, 319–330. [Google Scholar] [CrossRef]
- Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive Foreground Extraction Using Iterated Graph Cuts. In ACM SIGGRAPH 2004 Papers; Association for Computing Machinery: New York, NY, USA, 2004; pp. 309–314. [Google Scholar] [CrossRef]
- Boykov, Y.; Kolmogorov, V. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 2004, 26, 1124–1137. [Google Scholar] [CrossRef] [Green Version]
- OpenCV 4.0.1. Available online: https://opencv.org/ (accessed on 27 October 2021).
- Ghahremani, K.; Khaloo, A.; Mohamadi, S.; Lattanzi, D. Damage Detection and Finite-Element Model Updating of Structural Components through Point Cloud Analysis. J. Aerosp. Eng. 2018, 31, 1–12. [Google Scholar] [CrossRef]
- Khaloo, A.; Lattanzi, D.; Jachimowicz, A.; Devaney, C. Utilizing UAV and 3D computer vision for visual inspection of a large gravity dam. Front. Built Environ. 2018, 4, 31. [Google Scholar] [CrossRef] [Green Version]
- Google Maps. Available online: https://www.google.com/maps/ (accessed on 2 July 2020).
- ASCE/SEI 41-13; Seismic Evaluation and Retrofit of Existing Buildings. ASCE Standard, American Society of Civil Engineers: Reston, VA, USA, 2014.
- Applied Technology Council (ATC). FEMA P695 Quantification of Seismic Performance Factors, 1st ed.; Federal Emergency Management Agency: Redwood City, CA, USA, 2009.
- Bogdanoff, J.L.; Goldberg, J.E.; Bernard, M.C. Response of a simple structure to a random earthquake-type disturbance. Bull. Seismol. Soc. Am. 1961, 51, 293–310. [Google Scholar] [CrossRef]
- Kubo, T.; Penzien, J. Time and Frequency Domain Analyses of Three-Dimensional Ground Motions, San Fernando Earthquake; Technical Report; Earthquake Engineering Research Center, University of California: Berkeley, CA, USA, 1976. [Google Scholar]
- Aslani, H. Probabilistic Earthquake Loss Estimation and Loss Disaggregation in Buildings. Ph.D. Thesis, Stanford University, Stanford, CA, USA, 2005. [Google Scholar]
- Reality Capture 1.1. Epic Games. Available online: https://www.capturingreality.com/ (accessed on 17 February 2021).
- CloudCompare 2.11.1. Available online: http://www.cloudcompare.org/ (accessed on 27 April 2021).
- Khaloo, A.; Lattanzi, D. Hierarchical dense structure-from-motion reconstructions for infrastructure condition assessment. J. Comput. Civ. Eng. 2017, 31, 04016047. [Google Scholar] [CrossRef]
- Dutta, S.C.; Mukhopadhyay, P.; Goswami, K. Augmenting strength of collapsed unreinforced masonry junctions: Principal damage feature of walls damaged by moderate indian earthquakes. Nat. Hazards Rev. 2013, 14, 281–285. [Google Scholar] [CrossRef]
- The Blender Foundation. Blender 2.93.4. Available online: https://www.blender.org/ (accessed on 2 December 2021).
- TurboSquid. Available online: https://www.turbosquid.com/ (accessed on 27 April 2021).
- Bhattacharya, S.; Nayak, S.; Dutta, S.C. A critical review of retrofitting methods for unreinforced masonry structures. Int. J. Disaster Risk Reduct. 2014, 7, 51–67. [Google Scholar] [CrossRef] [Green Version]
- Giaretton, M.; Dizhur, D.; da Porto, F.; Ingham, J.M. Construction Details and Observed Earthquake Performance of Unreinforced Clay Brick Masonry Cavity-walls. Structures 2016, 6, 159–169. [Google Scholar] [CrossRef]
- Jafari, B.; Khaloo, A.; Lattanzi, D. Deformation tracking in 3D point clouds via statistical sampling of direct cloud-to-cloud distances. J. Nondestruct. Eval. 2017, 36, 1–10. [Google Scholar] [CrossRef]
- Hoskere, V.; Narazaki, Y.; Spencer, B.F. Physics-Based Graphics Models in 3D Synthetic Environments as Autonomous Vision-Based Inspection Testbeds. Sensors 2022, 22, 532. [Google Scholar] [CrossRef]
- Hoskere, V.; Narazaki, Y.; Spencer, B.F.; Smith, M.D. Deep learning-based damage detection of miter gates using synthetic imagery from computer graphics. In Proceedings of the 12th International Workshop on Structural Health Monitoring: Enabling Intelligent Life-Cycle Health Management for Industry Internet of Things (IIOT), Stanford, CA, USA, 10–12 September 2019; Volume 2, pp. 3073–3080. [Google Scholar]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Levine, N.M.; Spencer, B.F., Jr. Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework. Sensors 2022, 22, 873. https://doi.org/10.3390/s22030873
Levine NM, Spencer BF Jr. Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework. Sensors. 2022; 22(3):873. https://doi.org/10.3390/s22030873
Chicago/Turabian StyleLevine, Nathaniel M., and Billie F. Spencer, Jr. 2022. "Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework" Sensors 22, no. 3: 873. https://doi.org/10.3390/s22030873
APA StyleLevine, N. M., & Spencer, B. F., Jr. (2022). Post-Earthquake Building Evaluation Using UAVs: A BIM-Based Digital Twin Framework. Sensors, 22(3), 873. https://doi.org/10.3390/s22030873