Construction Jobsite Image Classification Using an Edge Computing Framework
Abstract
:1. Introduction
2. Literature Review
2.1. Edge Computing
2.2. Image Classification
2.3. Construction Site Image Classification Applications
3. Methodology
3.1. Overview
3.2. Classification Model Development
3.3. Edge Environment Setup
4. Case Study
4.1. Material Classification
4.2. Safety Detection: Identifying Boards with Nails
5. Limitations
6. Recommendations for Future Research
7. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Hamledari, H.; Fischer, M. Construction Payment Automation Using Blockchain-Enabled Smart Contracts and Robotic Reality Capture Technologies. Autom. Constr. 2021, 132, 103926. [Google Scholar] [CrossRef]
- Akhavian, R.; Behzadan, A.H. Smartphone-Based Construction Workers’ Activity Recognition and Classification. Autom. Constr. 2016, 71, 198–209. [Google Scholar] [CrossRef]
- Nath, N.D.; Chaspari, T.; Behzadan, A.H. Single- and Multi-Label Classification of Construction Objects Using Deep Transfer Learning Methods. J. Inf. Technol. Constr. 2019, 24, 511–526. [Google Scholar] [CrossRef]
- Jiang, Y.; Bai, Y.; Han, S. Determining Ground Elevations Covered by Vegetation on Construction Sites Using Drone-Based Orthoimage and Convolutional Neural Network. J. Comput. Civ. Eng. 2020, 34, 04020049. [Google Scholar] [CrossRef]
- Ham, Y.; Kamari, M. Automated Content-Based Filtering for Enhanced Vision-Based Documentation in Construction toward Exploiting Big Visual Data from Drones. Autom. Constr. 2019, 105, 102831. [Google Scholar] [CrossRef]
- Luo, H.; Liu, J.; Fang, W.; Love, P.E.D.; Yu, Q.; Lu, Z. Real-Time Smart Video Surveillance to Manage Safety: A Case Study of a Transport Mega-Project. Adv. Eng. Inform. 2020, 45, 101100. [Google Scholar] [CrossRef]
- Chen, C.; Zhu, Z.; Hammad, A. Automated Excavators Activity Recognition and Productivity Analysis from Construction Site Surveillance Videos. Autom. Constr. 2020, 110, 103045. [Google Scholar] [CrossRef]
- Asadi, K.; Kalkunte Suresh, A.; Ender, A.; Gotad, S.; Maniyar, S.; Anand, S.; Noghabaei, M.; Han, K.; Lobaton, E.; Wu, T. An Integrated UGV-UAV System for Construction Site Data Collection. Autom. Constr. 2020, 112, 103068. [Google Scholar] [CrossRef]
- Kim, H.; Seong, J.; Jung, H.-J. Real-Time Struck-By Hazards Detection System for Small- and Medium-Sized Construction Sites Based on Computer Vision Using Far-Field Surveillance Videos. J. Comput. Civ. Eng. 2023, 37, 04023028. [Google Scholar] [CrossRef]
- Lim, Y.G.; Wu, J.; Goh, Y.M.; Tian, J.; Gan, V. Automated Classification of “Cluttered” Construction Housekeeping Images through Supervised and Self-Supervised Feature Representation Learning. Autom. Constr. 2023, 156, 105095. [Google Scholar] [CrossRef]
- Rabbi, A.B.K.; Jeelani, I. AI Integration in Construction Safety: Current State, Challenges, and Future Opportunities in Text, Vision, and Audio Based Applications. Autom. Constr. 2024, 164, 105443. [Google Scholar] [CrossRef]
- Kim, J.; Hwang, J.; Jeong, I.; Chi, S.; Seo, J.O.; Kim, J. Generalized Vision-Based Framework for Construction Productivity Analysis Using a Standard Classification System. Autom. Constr. 2024, 165, 105504. [Google Scholar] [CrossRef]
- Sherafat, B.; Ahn, C.R.; Akhavian, R.; Behzadan, A.H.; Golparvar-Fard, M.; Kim, H.; Lee, Y.-C.; Rashidi, A.; Azar, E.R. Automated Methods for Activity Recognition of Construction Workers and Equipment: State-of-the-Art Review. J. Constr. Eng. Manag. 2020, 146, 03120002. [Google Scholar] [CrossRef]
- Song, L.; Mohammed, T.; Stayshich, D.; Eldin, N. A Cost Effective Material Tracking and Locating Solution for Material Laydown Yard. Procedia Eng. 2015, 123, 538–545. [Google Scholar] [CrossRef]
- Teizer, J. Status Quo and Open Challenges in Vision-Based Sensing and Tracking of Temporary Resources on Infrastructure Construction Sites. Adv. Eng. Inform. 2015, 29, 225–238. [Google Scholar] [CrossRef]
- Lee, D.; Lee, S. Digital Twin for Supply Chain Coordination in Modular Construction. Appl. Sci. 2021, 11, 5909. [Google Scholar] [CrossRef]
- Han, K.K.; Golparvar-Fard, M. Appearance-Based Material Classification for Monitoring of Operation-Level Construction Progress Using 4D BIM and Site Photologs. Autom. Constr. 2015, 53, 44–57. [Google Scholar] [CrossRef]
- Paneru, S.; Jeelani, I. Computer Vision Applications in Construction: Current State, Opportunities & Challenges. Autom. Constr. 2021, 132, 103940. [Google Scholar]
- Chen, Z.; Chen, J.; Shen, F.; Lee, Y. Collaborative Mobile-Cloud Computing for Civil Infrastructure Condition Inspection. J. Comput. Civ. Eng. 2015, 29, 04014066. [Google Scholar] [CrossRef]
- Tan, Y.; Yi, W.; Chen, P.; Zou, Y. An Adaptive Crack Inspection Method for Building Surface Based on BIM, UAV and Edge Computing. Autom. Constr. 2024, 157, 105161. [Google Scholar] [CrossRef]
- Kochovski, P.; Stankovski, V. Supporting Smart Construction with Dependable Edge Computing Infrastructures and Applications. Autom. Constr. 2018, 85, 182–192. [Google Scholar] [CrossRef]
- Abner, M.; Wong, P.K.Y.; Cheng, J.C.P. Battery Lifespan Enhancement Strategies for Edge Computing-Enabled Wireless Bluetooth Mesh Sensor Network for Structural Health Monitoring. Autom. Constr. 2022, 140, 104355. [Google Scholar] [CrossRef]
- Asp, A.; Sydorov, Y.; Keskikastari, M.; Valkama, M.; Niemelä, J. Impact of Modern Construction Materials on Radio Signal Propagation: Practical Measurements and Network Planning Aspects. In Proceedings of the 2014 IEEE 79th Vehicular Technology Conference (VTC Spring), Seoul, Republic of Korea, 18–21 May 2014; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2014; pp. 1–7. [Google Scholar]
- Satyanarayanan, M. The Emergence of Edge Computing. Computer 2017, 50, 30–39. [Google Scholar] [CrossRef]
- Ali, O.; Ishak, M.K.; Bhatti, M.K.L.; Khan, I.; Kim, K.-I. A Comprehensive Review of Internet of Things: Technology Stack, Middlewares, and Fog/Edge Computing Interface. Sensors 2022, 22, 995. [Google Scholar] [CrossRef] [PubMed]
- Zhou, Z.; Chen, X.; Li, E.; Zeng, L.; Luo, K.; Zhang, J. Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing. Proc. IEEE 2019, 107, 1738–1762. [Google Scholar]
- Russo, T.; Ribeiro, R.R.; Araghi, A.; Lameiras, R.d.M.; Granja, J.; Azenha, M. Continuous Monitoring of Elastic Modulus of Mortars Using a Single-Board Computer and Cost-Effective Components. Buildings 2023, 13, 1117. [Google Scholar] [CrossRef]
- Chen, C.; Gu, H.; Lian, S.; Zhao, Y.; Xiao, B. Investigation of Edge Computing in Computer Vision-Based Construction Resource Detection. Buildings 2022, 12, 2167. [Google Scholar] [CrossRef]
- Salim, D.D.; Mulder, H.M.; Burry, J.R. Form Fostering: A Novel Design Approach for Interacting with Parametric Models in the Embodied Virtuality. J. Inf. Technol. Constr. (ITcon) 2011, 16, 135–150. [Google Scholar]
- Li, K.; Zhao, J.; Hu, J.; Chen, Y. Dynamic Energy Efficient Task Offloading and Resource Allocation for NOMA-Enabled IoT in Smart Buildings and Environment. Build. Environ. 2022, 226, 109513. [Google Scholar] [CrossRef]
- Li, D.; Li, B.; Qin, N.; Jing, X.; Du, C.; Wan, C. The Research of NOMA-MEC Network Based on Untrusted Relay-Assisted Transmission in Power Internet of Things. In Proceedings of the 2020 2nd International Conference on Civil Engineering, Environment Resources and Energy Materials, Changsha, China, 18–20 September 2020; IOP Conference Series: Earth and Environmental Science. IOP Publishing Ltd.: Bristol, UK, 2021; Volume 634. [Google Scholar]
- Nvidia NVIDIA Jetson Nano Product Design Guide. 2020. Available online: https://docs-download.oss-cn-shanghai.aliyuncs.com/%E5%8E%82%E5%95%86%E8%B5%84%E6%96%99/%E8%8B%B1%E4%BC%9F%E8%BE%BE/%E8%8B%B1%E4%BC%9F%E8%BE%BE%E7%A1%AC%E4%BB%B6/Jetson%20Nano-Z/Jetson%20Nano-Z%20Product%20Design%20Guide%20DG-09502-001v2.1.pdf (accessed on 20 September 2024).
- Blanco, J.L.; Mullin, A.; Pandya, K.; Sridhar, M. The New Age of Engineering and Construction Technology; McKinsey & Company: Chicago, IL, USA, 2017. [Google Scholar]
- Hazelrigg, G.A. Honing Your Proposal Writing Skills. Available online: https://www.soest.hawaii.edu/GG/FACULTY/smithkonter/GG_711/proposal_tips/NSF_ProposalWriting_Tips.pdf (accessed on 19 September 2024).
- Lu, S.; Lu, J.; An, K.; Wang, X.; He, Q. Edge Computing on IoT for Machine Signal Processing and Fault Diagnosis: A Review. IEEE Internet Things J. 2023, 10, 11093–11116. [Google Scholar] [CrossRef]
- Kalyani, Y.; Collier, R. A Systematic Survey on the Role of Cloud, Fog, and Edge Computing Combination in Smart Agriculture. Sensors 2021, 21, 5922. [Google Scholar] [CrossRef]
- Moore, G.E. Cramming More Components onto Integrated Circuits. Proc. IEEE 1998, 86, 82–85. [Google Scholar]
- Dilley, J.; Maggs, B.; Parikh, J.; Prokop, H.; Sitaraman, R.; Weihl, B. Globally Distributed Content Delivery. IEEE Internet Comput. 2002, 6, 50–58. [Google Scholar] [CrossRef]
- Noble, B.D.; Satyanarayanan, M.; Narayanan, D.; Tilton, J.E.; Flinn, J.; Walker, K.R. Agile Application-Aware Adaptation for Mobility. In Proceedings of the 16th ACM Symposium on Operating System Principles, St. Malo, France, 5–8 October 2001; 1997; pp. 276–287. [Google Scholar]
- Satyanarayanan, M. Pervasive Computing: Vision and Challenges. IEEE Pers. Commun. 2001, 8, 10–17. [Google Scholar] [CrossRef]
- Rowstron, A.; Druschel, P. Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems. In Proceedings of the 18th IFIP/ACM International Conference on Distributed Systems Platforms (Middleware 2001), Heidelberg, Germany, 12–16 November 2001; pp. 329–350. [Google Scholar]
- Amazon Elastic Compute Cloud 2006. Available online: https://aws.amazon.com/about-aws/whats-new/2006/08/24/announcing-amazon-elastic-compute-cloud-amazon-ec2---beta/ (accessed on 20 September 2024).
- Satyanarayanan, M.; Bahl, P.; Cáceres, R.; Davies, N. The Case for VM-Based Cloudlets in Mobile Computing. IEEE Pervasive Comput. 2009, 8, 14–23. [Google Scholar] [CrossRef]
- Bonomi, F.; Milito, R.; Zhu, J.; Addepalli, S. Fog Computing and Its Role in the Internet of Things. In Proceedings of the MCC ’12: First Edition of the MCC Workshop on Mobile Cloud Computing, Helsinki, Finland, 17 August 2012; pp. 13–16. [Google Scholar]
- Task Committee on Application of Small Computers in Construction of the Construction Division. Application of Small Computers in Construction. J. Constr. Eng. Manag. 1985, 111, 173–189. [Google Scholar] [CrossRef]
- Kim, C.; Park, T.; Lim, H.; Kim, H. On-Site Construction Management Using Mobile Computing Technology. Autom. Constr. 2013, 35, 415–423. [Google Scholar] [CrossRef]
- Chen, Z.; Chen, J. Mobile Imaging and Computing for Intelligent Structural Damage Inspection. Adv. Civ. Eng. 2014, 2014, 483729. [Google Scholar] [CrossRef]
- Zhang, H.; Chi, S.; Yang, J.; Nepal, M.; Moon, S. Development of a Safety Inspection Framework on Construction Sites Using Mobile Computing. J. Manag. Eng. 2017, 33, 04016048. [Google Scholar] [CrossRef]
- B’Far, R.; Fielding, R.T. Mobile Computing Principles; Cambridge University Press: Cambridge, UK, 2004; ISBN 9780521817332. [Google Scholar]
- Chen, Y.; Kamara, J.M. A Framework for Using Mobile Computing for Information Management on Construction Sites. Autom. Constr. 2011, 20, 776–788. [Google Scholar] [CrossRef]
- Fang, Y.; Cho, Y.K.; Zhang, S.; Perez, E. Case Study of BIM and Cloud–Enabled Real-Time RFID Indoor Localization for Construction Management Applications. J. Constr. Eng. Manag. 2016, 142, 05016003. [Google Scholar] [CrossRef]
- Zhao, X. A Scientometric Review of Global BIM Research: Analysis and Visualization. Autom. Constr. 2017, 80, 37–47. [Google Scholar] [CrossRef]
- Chen, Y. A Conceptual Framework for a BIM-Based Collaboration Platform Supported by Mobile Computing. Appl. Mech. Mater. 2011, 94–96, 2144–2148. [Google Scholar] [CrossRef]
- Chi, H.L.; Kang, S.C.; Wang, X. Research Trends and Opportunities of Augmented Reality Applications in Architecture, Engineering, and Construction. Autom. Constr. 2013, 33, 116–122. [Google Scholar] [CrossRef]
- Chen, K. Enhancing Construction Safety Management through Edge Computing: Framework and Scenarios. J. Inf. Technol. Constr. 2020, 25, 438–451. [Google Scholar] [CrossRef]
- Ye, X.W.; Li, Z.X.; Jin, T. Smartphone-Based Structural Crack Detection Using Pruned Fully Convolutional Networks and Edge Computing. Smart Struct. Syst. 2022, 29, 141–151. [Google Scholar] [CrossRef]
- Żarski, M.; Wójcik, B.; Książek, K.; Miszczak, J.A. Finicky Transfer Learning—A Method of Pruning Convolutional Neural Networks for Cracks Classification on Edge Devices. Comput.-Aided Civ. Infrastruct. Eng. 2022, 37, 500–515. [Google Scholar] [CrossRef]
- Kochovski, P.; Stankovski, V. Building Applications for Smart and Safe Construction with the DECENTER Fog Computing and Brokerage Platform. Autom. Constr. 2021, 124, 103562. [Google Scholar] [CrossRef]
- Proietti, M.; Bianchi, F.; Marini, A.; Menculini, L.; Termite, L.F.; Garinei, A.; Biondi, L.; Marconi, M. Edge Intelligence with Deep Learning in Greenhouse Management. In Proceedings of the International Conference on Smart Cities and Green ICT Systems, SMARTGREENS—Proceedings, Online, 28–30 April 2021; Science and Technology Publications, Lda: Setúbal, Portugal, 2021; pp. 180–187. [Google Scholar]
- Alsalemi, A.; Himeur, Y.; Bensaali, F.; Amira, A. An Innovative Edge-Based Internet of Energy Solution for Promoting Energy Saving in Buildings. Sustain. Cities Soc. 2022, 78, 103571. [Google Scholar] [CrossRef]
- Costin, A.; McNair, J. IoT and Edge Computing in the Construction Site. In Buildings and Semantics; CRC Press: Boca Raton, FL, USA, 2022; pp. 223–237. [Google Scholar]
- Bae, H.; Golparvar-Fard, M.; White, J. Image-Based Localization and Content Authoring in Structure-from-Motion Point Cloud Models for Real-Time Field Reporting Applications. J. Comput. Civ. Eng. 2014, 29, B4014008. [Google Scholar] [CrossRef]
- Wei, Y.; Akinci, B. A Vision and Learning-Based Indoor Localization and Semantic Mapping Framework for Facility Operations and Management. Autom. Constr. 2019, 107, 102915. [Google Scholar] [CrossRef]
- Feng, C.; Kamat, V.R. Plane Registration Leveraged by Global Constraints for Context-Aware AEC Applications. Comput.-Aided Civ. Infrastruct. Eng. 2013, 28, 325–343. [Google Scholar] [CrossRef]
- Zakiyudin, M.Z.; Fathi, M.S.; Rambat, S.; Mohd Tobi, S.U.; Kasim, N.; Ahmad Latiffi, A. The Potential of Context-Aware Computing for Building Maintenance Management Systems. Appl. Mech. Mater. 2013, 405–408, 3505–3508. [Google Scholar] [CrossRef]
- Li, M.; Feng, X.; Han, Y.; Liu, X. Mobile Augmented Reality-Based Visualization Framework for Lifecycle O&M Support of Urban Underground Pipe Networks. Tunn. Undergr. Space Technol. 2023, 136, 105069. [Google Scholar] [CrossRef]
- Hayden, S.; Ames, D.P.; Turner, D.; Keene, T.; Andrus, D. Mobile, Low-Cost, and Large-Scale Immersive Data Visualization Environment for Civil Engineering Applications. J. Comput. Civ. Eng. 2015, 29, 05014011. [Google Scholar] [CrossRef]
- Li, Q.; Wang, X.; Wang, P.; Zhang, W.; Yin, J. FARDA: A Fog-Based Anonymous Reward Data Aggregation Security Scheme in Smart Buildings. Build. Environ. 2022, 225, 109578. [Google Scholar] [CrossRef]
- Jararweh, Y.; Otoum, S.; Ridhawi, I. Al Trustworthy and Sustainable Smart City Services at the Edge. Sustain. Cities Soc. 2020, 62, 102394. [Google Scholar] [CrossRef]
- Sun, Y.; Liu, J.; Bashir, A.K.; Tariq, U.; Liu, W.; Chen, K.; Alshehri, M.D. E-CIS: Edge-Based Classifier Identification Scheme in Green & Sustainable IoT Smart City. Sustain. Cities Soc. 2021, 75, 103312. [Google Scholar] [CrossRef]
- Xiong, F.; Xu, C.; Ren, W.; Zheng, R.; Gong, P.; Ren, Y. A Blockchain-Based Edge Collaborative Detection Scheme for Construction Internet of Things. Autom. Constr. 2022, 134, 104066. [Google Scholar] [CrossRef]
- Zou, H.; Zhou, Y.; Jiang, H.; Chien, S.C.; Xie, L.; Spanos, C.J. WinLight: A WiFi-Based Occupancy-Driven Lighting Control System for Smart Building. Energy Build 2018, 158, 924–938. [Google Scholar] [CrossRef]
- Williams, G.; Gheisari, M.; Chen, P.-J.; Irizarry, J. BIM2MAR: An Efficient BIM Translation to Mobile Augmented Reality Applications. J. Manag. Eng. 2015, 31, A4014009. [Google Scholar] [CrossRef]
- Fenais, A.; Ariaratnam, S.T.; Ayer, S.K.; Smilovsky, N. Integrating Geographic Information Systems and Augmented Reality for Mapping Underground Utilities. Infrastructures 2019, 4, 60. [Google Scholar] [CrossRef]
- Behzadan, A.H.; Kamat, V.R. Integrated Information Modeling and Visual Simulation of Engineering Operations Using Dynamic Augmented Reality Scene Graphs. J. Inf. Technol. Constr. (ITcon) 2011, 16, 259–278. [Google Scholar]
- Lowe, D.G. Object Recognition from Local Scale-Invariant Features. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; IEEE: Piscataway, NJ, USA, 1999; pp. 1150–1157. [Google Scholar]
- Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 20–25 June 2005; pp. 886–893. [Google Scholar]
- Lecun, Y.; Bottou, E.; Bengio, Y.; Haffner, P. Gradient-Based Learning Applied to Document Recognition. Proc. IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2014. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; IEEE Computer Society: Washington, DC, USA, 2016; pp. 770–778. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; Li, F.-F. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; IEEE: Piscataway, NJ, USA, 2009. [Google Scholar]
- Pan, S.J.; Yang, Q. A Survey on Transfer Learning. IEEE Trans. Knowl. Data Eng. 2010, 22, 1345–1359. [Google Scholar] [CrossRef]
- Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Zhang, X.; Zhou, X.; Lin, M.; Sun, J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; IEEE Computer Society: Washington, DC, USA, 2018; pp. 6848–6856. [Google Scholar]
- Tan, M.; Le, Q.V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv 2019, arXiv:1905.11946. [Google Scholar]
- Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258. [Google Scholar]
- Raza, R.; Zulfiqar, F.; Khan, M.O.; Arif, M.; Alvi, A.; Iftikhar, M.A.; Alam, T. Lung-EffNet: Lung Cancer Classification Using EfficientNet from CT-Scan Images. Eng. Appl. Artif. Intell. 2023, 126, 106902. [Google Scholar] [CrossRef]
- Dimitrov, A.; Golparvar-Fard, M. Vision-Based Material Recognition for Automated Monitoring of Construction Progress and Generating Building Information Modeling from Unordered Site Image Collections. Adv. Eng. Inform. 2014, 28, 37–49. [Google Scholar] [CrossRef]
- Gong, J.; Caldas, C.H. Computer Vision-Based Video Interpretation Model for Automated Productivity Analysis of Construction Operations. J. Comput. Civ. Eng. 2010, 24, 252–263. [Google Scholar] [CrossRef]
- Xiahou, X.; Li, Z.; Xia, J.; Zhou, Z.; Li, Q. A Feature-Level Fusion-Based Multimodal Analysis of Recognition and Classification of Awkward Working Postures in Construction. J. Constr. Eng. Manag. 2023, 149, 04023138. [Google Scholar] [CrossRef]
- Chen, G.; Liu, M.; Zhang, Y.; Wang, Z.; Hsiang, S.M.; He, C. Using Images to Detect, Plan, Analyze, and Coordinate a Smart Contract in Construction. J. Manag. Eng. 2023, 39, 04023002. [Google Scholar] [CrossRef]
- Luo, X.; Li, H.; Cao, D.; Dai, F.; Seo, J.; Lee, S. Recognizing Diverse Construction Activities in Site Images via Relevance Networks of Construction-Related Objects Detected by Convolutional Neural Networks. J. Comput. Civ. Eng. 2018, 32, 04018012. [Google Scholar] [CrossRef]
- Mneymneh, B.E.; Abbas, M.; Khoury, H. Vision-Based Framework for Intelligent Monitoring of Hardhat Wearing on Construction Sites. J. Comput. Civ. Eng. 2019, 33, 04018066. [Google Scholar] [CrossRef]
- Chen, J.; Lu, W.; Lou, J. Automatic Concrete Defect Detection and Reconstruction by Aligning Aerial Images onto Semantic-Rich Building Information Model. Comput.-Aided Civ. Infrastruct. Eng. 2023, 38, 1079–1098. [Google Scholar] [CrossRef]
- Alsakka, F.; Assaf, S.; El-Chami, I.; Al-Hussein, M. Computer Vision Applications in Offsite Construction. Autom. Constr. 2023, 154, 104980. [Google Scholar] [CrossRef]
- Shin, Y.; Choi, Y.; Won, J.; Hong, T.; Koo, C. A New Benchmark Model for the Automated Detection and Classification of a Wide Range of Heavy Construction Equipment. J. Manag. Eng. 2024, 40, 04023069. [Google Scholar] [CrossRef]
- Akhavian, R.; Behzadan, A.H. Construction Equipment Activity Recognition for Simulation Input Modeling Using Mobile Sensors and Machine Learning Classifiers. Adv. Eng. Inform. 2015, 29, 867–877. [Google Scholar] [CrossRef]
- Jacob, B.; Kligys, S.; Chen, B.; Zhu, M.; Tang, M.; Howard, A.; Adam, H.; Kalenichenko, D. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. arXiv 2017. [Google Scholar] [CrossRef]
- Xiao, X.; Chen, C.; Skitmore, M.; Li, H.; Deng, Y. Exploring Edge Computing for Sustainable CV-Based Worker Detection in Construction Site Monitoring: Performance and Feasibility Analysis. Buildings 2024, 14, 2299. [Google Scholar] [CrossRef]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.-C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 4510–4520. [Google Scholar]
- Chen, G.; Alsharef, A.; Pedraza, M.; Albert, A.; Jaselskis, E. A Novel Edge Computing Framework for Construction Nail Detection under Conditions of Constrained Computing Resources. In Proceedings of the 2024 ASCE International Conference on Computing in Civil Engineering, Pittsburgh, PE, USA, 28–31 July 2024; ASCE: Reston, VA, USA, 2024. [Google Scholar]
- Google Coral Coral-USB-Accelerator-Datasheet 2019. Available online: https://cdn-reichelt.de/documents/datenblatt/A300/CORAL-USB-ACCELERATOR-DATASHEET.pdf (accessed on 20 September 2024).
- U.S Department of Labor National Census of Fatal Occupational Injuries in 2022. 2022. Available online: https://www.bls.gov/news.release/pdf/cfoi.pdf (accessed on 21 March 2024).
- OSHA. Safety and Health Regulations for Construction—General Safety and Health Provisions. Available online: https://www.osha.gov/laws-regs/regulations/standardnumber/1926/1926.25 (accessed on 7 January 2024).
- American Plywood Association Building for High-Wind Resilience in Light-Frame Wood Construction (Design Guidance). 2021. Available online: https://osb.westfraser.com/wp-content/uploads/2018/08/M310E-2018-Building-in-High-Wind.pdf (accessed on 20 September 2024).
- Google Post-Training Quantization|TensorFlow Lite. Available online: https://www.tensorflow.org/lite/performance/post_training_quantization (accessed on 15 May 2024).
- Chen, G.; Alsharef, A.; Jaselskis, E. Integrating Sensor-Empowered Federated Learning and Smart Contracts for Automatic Flood Risk Management. In Proceedings of the 2024 ASCE International Conference on Computing in Civil Engineering, Pittsburgh, PE, USA, 28 July 2024; ASCE: Reston, VA, USA, 2024. [Google Scholar]
Product | Function | Price (USD) |
---|---|---|
CanaKit Raspberry Pi 4 8 GB Starter Kit—8 GB RAM | Single Board of Computer | 159.99 |
Arducam for Raspberry Pi Camera Module | Camera | 12.99 |
Vilros 8 Inch 1024 × 768 Screen | Screen | 79.99 |
Vilros 2.4 GHz Mini Wireless Keyboard with Touchpad Mouse-USB Receiver | Keyboard with Touchpad Mouse | 14.99 |
Vilros Mini Bluetooth Speaker | Speaker | 8.99 |
Power Ridge Portable Power Bank with AC Outlet, 100 W 26,270 mAh | Battery | 69.99 |
Coral USB Accelerator | Google Edge TPU ML accelerator coprocessor | 83.90 |
SUM | 430.84 |
Model | Accuracy | Class | Precision | Recall | F1 Score | Support | Model Size |
---|---|---|---|---|---|---|---|
MobileNetV1 | 1 | Plywood | 1.00 | 1.00 | 1.00 | 259 | 27.5 MB |
OSB | 1.00 | 1.00 | 1.00 | 322 | |||
MobileNetV2 | 1 | Plywood | 1.00 | 1.00 | 1.00 | 259 | 14.3 MB |
OSB | 1.00 | 1.00 | 1.00 | 322 | |||
MobileNetV3Small | 0.91 | Plywood | 0.89 | 0.91 | 0.90 | 259 | 9.7 MB |
OSB | 0.93 | 0.91 | 0.92 | 322 | |||
MobileNetV3Large | 0.89 | Plywood | 0.81 | 1.00 | 0.90 | 259 | 26.5 MB |
OSB | 1.00 | 0.82 | 0.90 | 322 |
Model | Accuracy | Class | Precision | Recall | F1 Score | Support | Model Size |
---|---|---|---|---|---|---|---|
MobileNetV1 | 0.90 | Boards | 0.98 | 0.83 | 0.90 | 581 | 17.4 MB |
Boards with nails | 0.83 | 0.98 | 0.90 | 472 | |||
MobileNetV2 | 0.87 | Boards | 0.97 | 0.79 | 0.87 | 581 | 15.1 MB |
Boards with nails | 0.79 | 0.97 | 0.87 | 472 | |||
MobileNetV3Small | 0.73 | Boards | 0.92 | 0.57 | 0.70 | 581 | 9.4 MB |
Boards with nails | 0.64 | 0.94 | 0.76 | 472 | |||
MobileNetV3Large | 0.83 | Boards | 0.80 | 0.92 | 0.85 | 581 | 19.2 MB |
Boards with nails | 0.88 | 0.71 | 0.79 | 472 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, G.; Alsharef, A.; Jaselskis, E. Construction Jobsite Image Classification Using an Edge Computing Framework. Sensors 2024, 24, 6603. https://doi.org/10.3390/s24206603
Chen G, Alsharef A, Jaselskis E. Construction Jobsite Image Classification Using an Edge Computing Framework. Sensors. 2024; 24(20):6603. https://doi.org/10.3390/s24206603
Chicago/Turabian StyleChen, Gongfan, Abdullah Alsharef, and Edward Jaselskis. 2024. "Construction Jobsite Image Classification Using an Edge Computing Framework" Sensors 24, no. 20: 6603. https://doi.org/10.3390/s24206603
APA StyleChen, G., Alsharef, A., & Jaselskis, E. (2024). Construction Jobsite Image Classification Using an Edge Computing Framework. Sensors, 24(20), 6603. https://doi.org/10.3390/s24206603