Deep Learning-Based PC Member Crack Detection and Quality Inspection Support Technology for the Precise Construction of OSC Projects

Abstract

Recently, the construction industry has benefited from the increased application of smart construction led by the core technologies of the fourth industrial revolution, such as BIM, AI, modular construction, and AR/VR, which enhance productivity and work efficiency. In addition, the importance of “Off-Site Construction (OSC)”, a factory-based production method, is being highlighted as modular construction increases in the domestic construction market as a means of productivity enhancement. The problem with OSC construction is that the quality inspection of Precast Concrete (PC) members produced at the factory and brought to the construction site is not carried out accurately and systematically. Due to the shortage of quality inspection manpower, a lot of time and money is wasted on inspecting PC members on-site, compromising inspection efficiency and accuracy. In this study, the major inspection items to be checked during the quality inspection are classified based on the existing PC member quality inspection checklist and PC construction specifications. Based on the major inspection items, the items to which AI technology can be applied (for automatic quality inspection) were identified. Additionally, the research was conducted focusing on the detection of cracks, which are one of the major types of defects in PC members. However, accurate detection of cracks is difficult since the inspection mostly relies on a visual check coupled with subjective experience. To automate the detection of cracks for PC members, video images of cracks and non-cracks on the surface were collected and used for image training and recognition using Convolutional Neural Network (CNN) and object detection, one of the deep learning technologies commonly applied in the field of image object recognition. Detected cracks were classified according to set thresholds (crack width and length), and finally, an automated PC member crack detection system that enables automatic crack detection based on mobile and web servers using deep learning and imaging technologies was proposed. This study is expected to enable more accurate and efficient on-site PC member quality inspection. Through the smart PC member quality inspection system proposed in this study, the time required for each phase of the existing PC member quality inspection work was reduced. This led to a reduction of 13 min of total work time, thereby improving work efficiency and convenience. Since quality inspection information can be stored and managed in the system database, human errors can be reduced while managing the quality of OSC work systematically and accurately. It is expected that through optimizing and upgrading our proposed system, quality work for the precise construction of OSC projects can be ensured. At the same time, systematic and accurate quality management of OSC projects is achievable through inspection data. In addition, the smart quality inspection system is expected to establish a smart work environment that enables efficient and accurate quality inspection practices if applied to various construction activities other than the OSC projects.

1. Introduction

Recently, the construction industry has benefited from the increased application of smart construction led by the core technologies of the fourth industrial revolution, such as Building Information Modeling (BIM), Artificial Intelligence (AI), modular construction, and Augmented Reality (AR)/Virtual Reality (VR), which enhance productivity and work efficiency. In addition, the importance of “Off-Site Construction (OSC)”, a factory-based production method, is being highlighted as modular construction increases in the domestic construction market as a means of productivity enhancement [1,2,3,4].

OSC involves factory-producing framework members, components, and pre-assembly parts in advance and transporting them to the building construction site where they will be assembled and installed [5,6,7]. Precast concrete (PC) members produced at the factory and brought to the site require a thorough quality check since damage or defects may occur during transportation. However, although accurate quality inspection is performed on PC members prior to leaving the factory, the same level of quality checking is lacking at the site where the members are transported to. The existing on-site inspection of PC members relies on visual checking using one’s naked eye and is limited to sample inspection due to a shortage of qualified manpower, thus compromising the accuracy of quality inspection as well as inspection efficiency.

In order to enhance the efficiency and accuracy of member damage or defect detection, research on automated defect detection that uses Convolutional Neural Network (CNN), one of the deep learning models, and object detection techniques applied to image recognition for concrete cracks or surface defects of electronic components, has gained traction recently [8,9,10,11]. Additionally, several studies have focused on automated quality inspection systems using mobile devices, such as smartphones and tablet PCs, that support field workers in conducting efficient quality checks and management of structures and construction work [12,13,14,15].

With the goal of improving the accuracy and efficiency of quality inspection practices, this study proposes an AI-based smart PC member quality inspection system that minimizes reliance on the inspector’s subjective experience in conducting quality inspection work. Through the proposed system, it is possible to reduce the inspection preparation time as well as the overall inspection work time while securing the work quality of the OSC project, thereby building a smart work environment.

2. Research Methodology

In this study, the research focused on the detection of cracks, which are one of the major types of defects in PC members. Moreover, accurate detection of cracks is difficult since the inspection mostly relies on a visual check coupled with subjective experience. With the research goal of developing a smart quality inspection system, the study proceeded as follows.

First, an extensive literature review was conducted to explore and analyze recent research trends related to quality defect detection for the target objects using AI technology and automated quality control practices using IT technology. Second, the major quality inspection items were derived by analyzing the on-site quality inspection process of PC members, which is commonly based on a field quality inspection checklist and specifications. Among the identified major inspection items, the items to which AI technology can be applied were classified. Third, to automate the crack detection process of PC members, an automated crack detection process based on AI technology and mobile and web server-based crack detection system was designed and developed. Finally, the developed system was validated through the on-site application of the automated crack detection system for PC members. The steps involved in the quality inspection system design, development, and verification are summarized in Figure 1 below.

3. Literature Review

3.1. Research Trends on Automation of Quality Defect Detection Using AI Technology

Conventional quality inspection of structures or members consumes a lot of work time and manpower while lacking objectivity and credibility since the inspection is carried out based on the inspector’s visual and subjective experience. In order to solve this problem, deep learning has been frequently applied in recent days to enhance the efficiency and accuracy of quality inspection on various objects, such as concrete structures, including roads, bridges, tunnels, electronic products, etc.

Kim et al. [16] conducted a study to detect cracks in concrete foundations by applying deep learning and image processing techniques, whereas Lee et al. [17] retrained Inception-v3, one of the deep learning neural network models, using concrete crack photos, and conducted a study to recognize and visualize cracks in photos of concrete using the retrained model. By applying deep learning and image processing technology, Jung et al. [18] conducted a study on an algorithm that not only enables users to recognize concrete cracks but also to check information on the width and length of cracks. Hassan et al. [19] conducted a study on a system that detects damage to sewer pipes and classifies damage types using CNN. Choi et al. [20,21] researched defect inspection and detection on the surface of electronic components, which commonly poses high detection difficulty and insufficient training data, also using CNN.

As a result of analyzing the research trends related to AI-based defect detection automation, research on automating recognition and detection of defects on existing structures or exterior members is on the rise. However, little research has been conducted on the automation of defect detection for individual PC members transported to the OSC site and used before the structure is completed.

3.2. Research Trends on Automation of Quality Inspection and Management Using IT Technology

Inspection activities conducted on existing construction projects or buildings, such as quality checks or safety inspections, can be a hassle for field workers who have to be familiar with various inspection information in advance, such as design drawings, specifications, and inspection items. In addition, there are problems associated with the inconvenience of having to carry hard copies of inspection-related documents, errors that may occur due to handwriting inspection items, and concerns about the loss of checklists. In order to solve the inefficiency and human errors of these tasks, several recent studies have focused on automation of quality detection that enhances efficiency and accuracy by using mobile devices, such as smartphones or tablet PCs when performing an on-site quality inspection.

Yoon et al. [22] researched establishing and developing a quality inspection system with an accompanying prototype for temporary construction projects, an electronic work support system designed to improve access to information during quality inspection of temporary constructions, and also to ensure systematic quality inspection through automation of inspection tasks. Similarly, to improve the quality inspection efficiency of temporary constructions, Choi et al. [23] focused on developing a rule-based temporary construction quality inspection system linked to BIM that can automate related tasks and systematically store and manage various quality inspection information.

In order to utilize mobile devices for work during building safety inspections, Ko et al. [24] developed a prototype application applicable to regular building inspections by deriving problems encountered and core requirement functions in the existing safety inspection practice. Oh et al. [12] collected information on quality inspection and defect management of apartment buildings in real-time by using a personal digital assistant (PDA) and the web and developed an apartment quality inspection and defect management system, which supports history management for each household and efficient work processing between related entities.

4. Analysis of PC Member Quality Inspection Process and Major Inspection Items

4.1. Analysis of On-Site Quality Inspection Process of PC Members

With the goal of developing an automated PC member crack detection system, this study conducted interviews with PC member production companies and experts to understand and to analyze the overall PC member production process and quality inspection flow as well as the PC member quality inspection process currently performed at OSC sites.

Factory-manufactured PC members that passed the quality inspection are shipped to each site by means of transportation. PC members transported to the site are moved to the storage yard and go through the unloading procedure. Since defects may occur during transportation, loading, and unloading, a sampling inspection based on a visual check is performed to find any defects after unloading. When defects in a member are found, a return or repair plan is reviewed after consulting with the person in charge of the site. Figure 2 shows the life cycle of PC members from the production factory to the construction site.

As shown in Figure 3, the on-site quality inspection of PC members is performed by the following procedure. First, a quality inspection checklist and any needed tools for quality inspection are prepared. Second, a sample PC member to be inspected is selected. Third, visual inspection is performed on the sample member based on the quality inspection checklist. Fourth, after completing the inspection, the quality inspection checklist is filled according to the inspection result. Finally, after the final review of the inspection results, the quality inspection process is completed.

However, problems are commonly encountered during this on-site quality inspection process due to the extensive time it takes to prepare the quality inspection checklist and any needed inspection tools and to perform inspection based on one’s visual and experience. Moreover, the inspection accuracy can be diminished due to human errors, such as errors due to handwriting checklists and the possibility of loss during checklist storage.

4.2. Analysis of Major Quality Inspection Items and Defect Types of PC Members

Based on PC construction standard specifications and interviews with experts, the major quality inspection items of PC members are classified into crack, damage, deformation, reinforcement and accessories condition, and exterior surface condition, which are visually inspected, as summarized in

Table 1. Major quality inspection items, inspection methods, and criteria.

Inspection Items	Inspection Method	Inspection Criteria
Crack	Visual check or actual measurement	According to construction specifications
Damage	Visual check	Class 1: Structural Hazard (Damage on hook part, damage over 200 mm of joint part) Class 2: Hazardous to waterproofing (Damage of the edge of the bottom plate, etc.) Class 3: Damage other than Class 1 and 2
Deformation	Visual check	Torsion/Bending: ±5 mm
Reinforcement and accessories condition	Visual check	The type and quantity of reinforcement and accessories must be in accordance with the member production drawing and installed in the correct position
Exposed surface condition	Visual check	Must be flat and normal appearance, have no defects due to honeycomb, marks, air bubbles, etc., no signs of lack of rebar coating

. In addition, the major defect types of PC members are classified as discoloration, crack, damage, and bubble, as shown in Figure 4.

In order to classify the major items to be checked when inspecting the quality of PC members, companies specializing in PC were consulted to collect and analyze the PC member quality inspection checklist, as shown in Figure 5. For PC members produced at the factory, quality inspection is performed based on the primary inspection items on the quality inspection checklist. After the inspection is completed, the members clear of defects are brought to the construction site by means of a heavy transport loader with the inspection checklist attached.

Since defects can occur to members delivered to the site during loading/unloading or transportation, re-inspection is made by the field workers. In the case of on-site quality inspection, the inspection is made based on the secondary quality inspection items. Specifically, workers perform inspections focusing on the construction project name, member identification number, manufacturing date, product inspection mark, cracks, damage, and deformation according to the PC construction standard specifications.

5. Crack Detection Automation Algorithm Design and System Development for PC Members

5.1. Deep Learning-Based Crack Detection Automation Process and Algorithm Design

For the automation of PC member crack detection, image training and recognition were performed using CNN and object detection, one of the deep learning technologies commonly applied in the field of image object recognition [25]. CNN consists of a convolution layer, a pooling layer, and a fully connected layer. To train the CNN model and carry out CNN-based image training, a defect image (i.e., crack defection) dataset was created by collecting images from web sources and by taking actual photos of target images. With the field application in mind, taking photos of target defect images was carried out under conditions similar to the actual inspection environment, where the distance between the target image and the camera (Galaxy S20 Ultra) lens was maintained at about 0.1 m, and the angle was kept constant at 90 degrees. After collecting image data, they were classified 1:1 for crack and non-crack images, respectively, so as to build an image dataset for deep learning. Then, using Labelme, data labeling was performed by segmenting the cracks within the crack images collected for training. Figure 6 below visually shows this process of building an image dataset.

As shown in Figure 7, the CNN-based image training and defect detection process proceeded as follows [26,27,28].

As shown in Figure 8, the quality inspection starts with selecting the target PC member to be inspected, and the video images of the surface of the selected member are recorded using a mobile device-based camera. The recorded video data is sent to a web server where cracks are automatically recognized through CNN-based image training and recognition. After crack recognition, the operator arbitrarily sets the values of the length and width of the crack to be detected, the crack is classified according to the user setting, and finally, quantified values for the crack length and width are generated. In addition, the detected crack information is stored and managed through a web server database.

The Single Shot Multi-box Detector 300 (SSD300) algorithm, one of the deep learning-based object detection models, was used to design an automated algorithm for detecting surface cracks in PC members delivered on-site [29]. As shown in Figure 9, the working principle of the SSD algorithm is as follows.

First, the training image is set to 300 × 300 resolution and input to the SSD model; second, it passes through the Visual Geometry Group (VGG) model, and feature maps with different scales are extracted from the image; third, the image goes through convolution by passing through several feature maps with different grid cell scales; finally, defect detection is performed by merging the entire feature map and training the SSD network through the loss function.

As for defect detection, the SSD algorithm is faster than Faster R-CNN, another object detection model, and is more accurate than YOLO [30,31].

Table 2. Comparison of Deep Learning-based Object Detection Model Performance.

Method	mAP	FPS	Batch Size	Boxes	Input Resolution
Faster R-CNN	73.2	7	1	6000	1000 × 600
Fast YOLO	52.7	155	1	98	448 × 448
YOLO	66.4	21	1	98	448 × 448
SSD 300	74.3	46	1	8732	300 × 300

compares the performance of different deep learning-based object detection models.

The automated crack detection algorithm consists of a total of four stages: data training stage, image processing stage, crack feature information extraction stage, and output stage. Through defect detection, members with no cracks are processed normally, whereas members found with cracks go through the process where the crack location is marked with a bounding box, and the crack size is quantified by measuring the width and length. Quantified crack information is then used to determine whether the crack exceeds the defect limit or not, and the defect level is classified based on this determination. Finally, the defect information is recorded and stored in the database.

Figure 10 shows the crack image analysis and detection process when crack image data is transmitted to the server of the PC member crack automated detection system. When analyzing the image, the image is segmented in units of patches, where the crack detection process occurs based on the segmented images. This process is followed by the image coupling and crack image adjustment of the detected crack location, and the noise is removed through the process of separating the crack from the background by means of the image brightness conversion. Based on the noise-removed crack image, the algorithm for measuring the length and width of the crack is applied, and the final crack measurement value is generated through CNN-based regression analysis.

5.2. Smart Quality Inspection Process Based on Automated Crack Detection

The on-site smart quality inspection process (‘To-Be’ Process in Figure 11 below) using CNN-based automated crack detection is as follows.

First, a mobile device, such as a smartphone or tablet PC capable of capturing video images of PC members brought to the site and recording inspection results is prepared; second, after selecting and setting the target PC member for quality inspection, the mobile device-based camera records surface images of the PC member to be inspected; third, after the captured video image is transmitted to the web server, the crack detection information is automatically produced, enabling the operator to grasp this information in real-time through the mobile device. Moreover, the same crack detection information on PC members is available through the field office computer in real-time. Lastly, the crack detection information is reviewed and recorded on the inspection checklist, which completes the on-site quality inspection process.

Through this automated crack detection-based quality inspection process, the accuracy and efficiency of the PC member quality inspection process can be enhanced. Likewise, the web server-based database storage and management enables more efficient and convenient management of present and historical inspection information.

5.3. Building an Automated Crack Detection System for PC Member

The conceptual diagram of the automated PC member crack detection system presented in this study is shown in Figure 12. This system is configured based on mobile devices and web servers but also can connect and communicate with the field office. The video image taken from the mobile device-based camera is used for video analysis and image training through the web server which processes defect (i.e., cracks) detection. The user can arbitrarily set the width and length values of the crack to be detected, and crack detection is carried out based on the set threshold values. The resulting detection information is transmitted in real-time to the user’s mobile device. In addition, the quality inspection-related information of PC members, such as crack detection, measurement, image, and detection information, can be stored on the web server-based database and managed accordingly.

The functional block diagram of the automated PC member crack detection system is shown in Figure 13. The main menu items of the system consist of image capture, image analysis, defect detection, detection information, and measurement information. The video recording function is linked with the camera application of the mobile device, whereas the video image analysis and defect detection functions are linked with the web server. In addition, the detection information and measurement information are linked with the web server, stored, and managed as a database.

6. On-Site Application and Verification of the Automated PC Member Crack Detection System

6.1. Scenario for the System Field Application

The flow diagram of the field application scenario established in this study and organized in Figure 14 consists of three steps. Step 1. Target member image capturing; Step 2. Damage information detection and recording; Step 3. Handling of the defective member. Each of these steps is further explained as follows. First, the system is initiated through the mobile device once the user completes inspection preparation. Second, the user logs in to the automated crack detection system and starts performing the detection task. Third, the target member is set in the system, and the video image of the member surface is captured. The system automatically assigns an individual code at the completion of each video recording of the member. Fourth, crack detection is initiated, and the system performs the deep learning-based automated crack detection process. The system automatically detects cracks based on the image recognition algorithm and quantifies the location and size of the crack if found. Based on the quantified information, the crack size is compared to the set defect threshold, and the defect information is recorded and produced automatically. Once the recording of defect information is completed, the detection result is saved in the database; finally, based on the classified defect level, the user completes an inspection of the member free of defects. However, members found to have crack(s) are returned if the defection is classified as a grade 1 crack or a repair plan is set, or the member is returned if the defection is classified as a grade 2 crack.

6.2. Verification of the System through Field Application

For the verification through field application of the automated PC member crack detection system proposed in this study, a construction site involving PC members was visited, where video images of cracks and non-cracks on the surface of PC members (e.g., PC slab and column) were recorded and collected. The construction site involved PC construction of an underground parking structure for a reconstruction housing project. Specific project information is summarized in

Table 3. Overview of the PC construction site for the system field application.

PC Construction Site Overview
Project Name	Gwangmyeong Reorganization District Housing Reconstruction Project
Project Duration	20 February 2020~28 October 2022
Project Location	S. Korea, Gyeonggi-do, Gwangmyeong-si, Saeteo-ro 45
Construction Method	PC construction
Construction Task	Underground parking structure PC construction (Fabrication and installation)

below.

Using a mobile device camera, 2000 crack and non-crack images each were collected from the PC members (i.e., slabs and columns) at the verification site.

Table 4. Sample images of crack and non-crack taken using a mobile device camera.

Image Object	Sample Image	Number of Images
Crack Image (PC slab and column)		2000
Non-crack Image (PC slab and column)		2000

below shows sample images of crack and non-crack, respectively, from the total of 4000 image data. After data collection, the original crack image data were labeled, and a training dataset was prepared so as to proceed with CNN-based image training.

Table 5. Composition of the image data for training and test.

	Training	Test
Crack data	1600	400
Non-crack data	1600	400

summarizes the composition of the original crack data, the crack data for training, and the original non-crack data used in this study. To proceed with the training, a CNN-based neural network was used, and the training was carried out based on a server with CPU Intel i9-9900k, GPU GeForce RTX 2080 Ti, and 64 GB memory.

As summarized in

Table 6. Recall and precision rates for training and test.

Title	Recall (%)	Precision (%)
Training	78	75
Test	75	71

, crack training showed 78% recall and 75% precision, whereas test showed 75% recall and 71% precision, both cases indicating somewhat lower precision than recall. However, recall and precision for both cases resulted in a high percentage of 70% or more. In addition, by setting the crack width 0.4 mm and the length 3.0 mm as threshold values, only cracks above the threshold values were classified and detected. Moreover, quantitative information on the width and length of cracks was also obtained.

Comparing the crack width and length measured visually with those measured using the automated crack detection system presented in this study revealed that the crack width had an error in the range of 0.01 mm to 0.02 mm in Cracks 1 and 2. As for the crack length, an error of 0.1 mm occurred in Cracks 3 and 4. Specific size measurement values for each crack case are presented in

Table 7. Crack size measurement values for visual check and crack detection system.

	Visual Check		Crack Detection System
	Width (mm)	Length (mm)	Width (mm)	Length (mm)
Crack 1	0.41	8.5	0.42	8.5
Crack 2	0.43	7.5	0.45	7.5
Crack 3	0.42	7.3	0.42	7.2
Crack 4	0.41	10.3	0.41	10.2

. The images of the crack detection result are shown in Figure 15.

Comparing the time taken for each step of the detection process using the existing on-site PC member quality inspection process versus the automated crack detection system, the work time during the inspection preparation phase was reduced by six minutes as it took one minute of preparation time for the automated system compared to the seven minutes of work time for the existing on-site inspection process. The work time during the inspection phase, based on a single PC member, was reduced by five minutes as it took two minutes for the automated system compared to the seven minutes for the existing process. Finally, during the inspection result review phase, the work time was reduced by two minutes as it took three minutes for the existing process versus one minute for the automated system. Adding this reduced time, the total work time was shortened by 13 min, as summarized in

Table 8. Comparison of inspection time between as-is process versus smart quality inspection process.

Task Title	As-Is Quality Inspection Process (min)	Smart Quality Inspection Process (min)
Inspection Preparation	7	1
Inspection	7	2
Review Inspection Result	3	1
Total	17	4

.

7. Conclusions and Future Research

In this study, a mobile and web server-based automated PC member crack detection system that can automatically detect cracks on the exterior of a PC member was proposed. For system construction and field application, the on-site quality inspection items, process, and standards for PC members were analyzed, and a deep learning-based PC member smart quality inspection algorithm was designed.

Additionally, system verification was performed by establishing a smart quality inspection process and scenario for the on-site application of the automated PC member smart quality inspection system. Through this system, it is possible to more efficiently and accurately inspect the quality of the PC members brought to the construction site, compared to the existing on-site inspection that commonly relies on one’s visual check and experience. In addition, the reliability of the quality inspection checklist items is secured through the use of deep learning-based crack detection, whereas the work efficiency and convenience are improved through the inspection information management by means of a web server-based inspection information database.

The main results derived from this study are as follows:

(1)

Derivation of key items for on-site quality inspection of PC members and classification of inspection items applicable to AI;

(2)

Analysis of problems involving the as-is process of on-site quality inspection of PC members and classification of work steps in which IT technology is applicable;

(3)

Design of the smart quality inspection algorithm for PC members and system construction;

(4)

Establishment of the smart quality inspection process and field application scenario for the system field application;

(5)

Reduction of work time compared to as-is quality inspection process through field application of smart quality inspection system;

(6)

Improvement of work accuracy and efficiency through smart quality inspection system-based quality inspection data storage and management;

(7)

Support of OSC precise construction and establishment of systematic OSC project management structure through the application of the smart quality inspection system.

Since OSC projects were the target of the smart system presented in this study, verification, optimization, and improvement of the system are warranted through application on diverse construction sites. Moreover, enhancement of the defect detection accuracy can be achieved by varying deep learning model training and testing based on factors that affect defect detection, such as deep learning model and parameter, image data quantity, and quality.

In future research, based on the automated PC member crack detection system proposed in this study, not only cracks in PC members but also automatic recognition and classification of member defects, such as wear, breakage, and discoloration, can be developed. Moreover, research on automatic reporting of user’s quality inspection checklists based on automated defect detection of the PC members can be conducted. It is considered that research on automatic reporting technology is possible. Furthermore, in connection with various smart construction technologies, such as BIM and platform technology, a study on the automation of quality and history management for the production–transport–construction–maintenance stage of the OSC site is necessary, while the research on automation of quality control for various construction materials, not limited to the PC member, is also considered necessary.