Sensors

Research

23 pages, 7922 KiB

Open AccessArticle

Smart Home Automation-Based Hand Gesture Recognition Using Feature Fusion and Recurrent Neural Network

by Bayan Ibrahimm Alabdullah, Hira Ansar, Naif Al Mudawi, Abdulwahab Alazeb, Abdullah Alshahrani, Saud S. Alotaibi and Ahmad Jalal

Sensors 2023, 23(17), 7523; https://doi.org/10.3390/s23177523 - 30 Aug 2023

Cited by 19 | Viewed by 5822

Abstract

Gestures have been used for nonverbal communication for a long time, but human–computer interaction (HCI) via gestures is becoming more common in the modern era. To obtain a greater recognition rate, the traditional interface comprises various devices, such as gloves, physical controllers, and [...] Read more.

Gestures have been used for nonverbal communication for a long time, but human–computer interaction (HCI) via gestures is becoming more common in the modern era. To obtain a greater recognition rate, the traditional interface comprises various devices, such as gloves, physical controllers, and markers. This study provides a new markerless technique for obtaining gestures without the need for any barriers or pricey hardware. In this paper, dynamic gestures are first converted into frames. The noise is removed, and intensity is adjusted for feature extraction. The hand gesture is first detected through the images, and the skeleton is computed through mathematical computations. From the skeleton, the features are extracted; these features include joint color cloud, neural gas, and directional active model. After that, the features are optimized, and a selective feature set is passed through the classifier recurrent neural network (RNN) to obtain the classification results with higher accuracy. The proposed model is experimentally assessed and trained over three datasets: HaGRI, Egogesture, and Jester. The experimental results for the three datasets provided improved results based on classification, and the proposed system achieved an accuracy of 92.57% over HaGRI, 91.86% over Egogesture, and 91.57% over the Jester dataset, respectively. Also, to check the model liability, the proposed method was tested on the WLASL dataset, attaining 90.43% accuracy. This paper also includes a comparison with other-state-of-the art methods to compare our model with the standard methods of recognition. Our model presented a higher accuracy rate with a markerless approach to save money and time for classifying the gestures for better interaction. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

15 pages, 4111 KiB

Open AccessArticle

Detection and Length Measurement of Cracks Captured in Low Definitions Using Convolutional Neural Networks

by Jin-Young Kim, Man-Woo Park, Nhut Truong Huynh, Changsu Shim and Jong-Woong Park

Sensors 2023, 23(8), 3990; https://doi.org/10.3390/s23083990 - 14 Apr 2023

Cited by 3 | Viewed by 2266

Abstract

Continuous efforts were made in detecting cracks in images. Varied CNN models were developed and tested for detecting or segmenting crack regions. However, most datasets used in previous works contained clearly distinctive crack images. No previous methods were validated on blurry cracks captured [...] Read more.

Continuous efforts were made in detecting cracks in images. Varied CNN models were developed and tested for detecting or segmenting crack regions. However, most datasets used in previous works contained clearly distinctive crack images. No previous methods were validated on blurry cracks captured in low definitions. Therefore, this paper presented a framework of detecting the regions of blurred, indistinct concrete cracks. The framework divides an image into small square patches which are classified into crack or non-crack. Well-known CNN models were employed for the classification and compared with each other with experimental tests. This paper also elaborated on critical factors—the patch size and the way of labeling patches—which had considerable influences on the training performance. Furthermore, a series of post-processes for measuring crack lengths were introduced. The proposed framework was tested on the images of bridge decks containing blurred thin cracks and showed reliable performance comparable to practitioners. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

23 pages, 6913 KiB

Open AccessArticle

Bottleneck Detection in Modular Construction Factories Using Computer Vision

by Roshan Panahi, Joseph Louis, Ankur Podder, Colby Swanson and Shanti Pless

Sensors 2023, 23(8), 3982; https://doi.org/10.3390/s23083982 - 14 Apr 2023

Cited by 8 | Viewed by 2749

Abstract

The construction industry is increasingly adopting off-site and modular construction methods due to the advantages offered in terms of safety, quality, and productivity for construction projects. Despite the advantages promised by this method of construction, modular construction factories still rely on manually-intensive work, [...] Read more.

The construction industry is increasingly adopting off-site and modular construction methods due to the advantages offered in terms of safety, quality, and productivity for construction projects. Despite the advantages promised by this method of construction, modular construction factories still rely on manually-intensive work, which can lead to highly variable cycle times. As a result, these factories experience bottlenecks in production that can reduce productivity and cause delays to modular integrated construction projects. To remedy this effect, computer vision-based methods have been proposed to monitor the progress of work in modular construction factories. However, these methods fail to account for changes in the appearance of the modular units during production, they are difficult to adapt to other stations and factories, and they require a significant amount of annotation effort. Due to these drawbacks, this paper proposes a computer vision-based progress monitoring method that is easy to adapt to different stations and factories and relies only on two image annotations per station. In doing so, the Scale-invariant feature transform (SIFT) method is used to identify the presence of modular units at workstations, and the Mask R-CNN deep learning-based method is used to identify active workstations. This information was synthesized using a near real-time data-driven bottleneck identification method suited for assembly lines in modular construction factories. This framework was successfully validated using 420 h of surveillance videos of a production line in a modular construction factory in the U.S., providing 96% accuracy in identifying the occupancy of the workstations and an F-1 Score of 89% in identifying the state of each station on the production line. The extracted active and inactive durations were successfully used via a data-driven bottleneck detection method to detect bottleneck stations inside a modular construction factory. The implementation of this method in factories can lead to continuous and comprehensive monitoring of the production line and prevent delays by timely identification of bottlenecks. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

25 pages, 4711 KiB

Open AccessArticle

Development of a Knowledge Graph for Automatic Job Hazard Analysis: The Schema

by Sonali Pandithawatta, Seungjun Ahn, Raufdeen Rameezdeen, Christopher W. K. Chow, Nima Gorjian and Tae Wan Kim

Sensors 2023, 23(8), 3893; https://doi.org/10.3390/s23083893 - 11 Apr 2023

Cited by 4 | Viewed by 2927

Abstract

In the current practice, an essential element of safety management systems, Job Hazard Analysis (JHA), is performed manually, relying on the safety personnel’s experiential knowledge and observations. This research was conducted to create a new ontology that comprehensively represents the JHA knowledge domain, [...] Read more.

In the current practice, an essential element of safety management systems, Job Hazard Analysis (JHA), is performed manually, relying on the safety personnel’s experiential knowledge and observations. This research was conducted to create a new ontology that comprehensively represents the JHA knowledge domain, including the implicit knowledge. Specifically, 115 actual JHA documents and interviews with 18 JHA domain experts were analyzed and used as the source of knowledge for creating a new JHA knowledge base, namely the Job Hazard Analysis Knowledge Graph (JHAKG). To ensure the quality of the developed ontology, a systematic approach to ontology development called METHONTOLOGY was used in this process. The case study performed for validation purposes demonstrates that a JHAKG can operate as a knowledge base that answers queries regarding hazards, external factors, level of risks, and appropriate control measures to mitigate risks. As the JHAKG is a database of knowledge representing a large number of actual JHA cases previously developed and also implicit knowledge that has not been formalized in any explicit forms yet, the quality of JHA documents produced from queries to the database is expectedly higher than the ones produced by an individual safety manager in terms of completeness and comprehensiveness. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

21 pages, 6864 KiB

Open AccessArticle

Improved Discriminative Object Localization Algorithm for Safety Management of Indoor Construction

by Jungeun Hwang, Kanghyeok Lee, May Mo Ei Zan, Minseo Jang and Do Hyoung Shin

Sensors 2023, 23(8), 3870; https://doi.org/10.3390/s23083870 - 10 Apr 2023

Cited by 2 | Viewed by 2022

Abstract

Object localization is a sub-field of computer vision-based object recognition technology that identifies object classes and locations. Studies on safety management are still in their infancy, particularly those aimed at lowering occupational fatalities and accidents at indoor construction sites. In comparison to manual [...] Read more.

Object localization is a sub-field of computer vision-based object recognition technology that identifies object classes and locations. Studies on safety management are still in their infancy, particularly those aimed at lowering occupational fatalities and accidents at indoor construction sites. In comparison to manual procedures, this study suggests an improved discriminative object localization (IDOL) algorithm to aid safety managers with visualization to improve indoor construction site safety management. The IDOL algorithm employs Grad-CAM visualization images from the EfficientNet-B7 classification network to automatically identify internal characteristics pertinent to the set of classes evaluated by the network model without the need for further annotation. To evaluate the performance of the presented algorithm in the study, localization accuracy in 2D coordinates and localization error in 3D coordinates of the IDOL algorithm and YOLOv5 object detection model, a leading object detection method in the current research area, are compared. The comparison findings demonstrate that the IDOL algorithm provides a higher localization accuracy with more precise coordinates than the YOLOv5 model over both 2D images and 3D point cloud coordinates. The results of the study indicate that the IDOL algorithm achieved improved localization performance over the existing YOLOv5 object detection model and, thus, is able to assist with visualization of indoor construction sites in order to enhance safety management. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

29 pages, 11653 KiB

Open AccessArticle

Prototyping a Chatbot for Site Managers Using Building Information Modeling (BIM) and Natural Language Understanding (NLU) Techniques

by Will Y. Lin

Sensors 2023, 23(6), 2942; https://doi.org/10.3390/s23062942 - 8 Mar 2023

Cited by 6 | Viewed by 4284

Abstract

Amidst the domestic labor shortage and worldwide pandemic in recent years, there has been an urgent need for a digital means that allows construction site workers, particularly site managers, to obtain information more efficiently in support of their daily managerial tasks. For workers [...] Read more.

Amidst the domestic labor shortage and worldwide pandemic in recent years, there has been an urgent need for a digital means that allows construction site workers, particularly site managers, to obtain information more efficiently in support of their daily managerial tasks. For workers who move around the site, traditional software applications that rely on a form-based interface and require multiple finger movements such as key hits and clicks can be inconvenient and reduce their willingness to use such applications. Conversational AI, also known as a chatbot, can improve the ease of use and usability of a system by providing an intuitive interface for user input. This study presents a demonstrative Natural Language Understanding (NLU) model and prototypes an AI-based chatbot for site managers to inquire about building component dimensions during their daily routines. Building Information Modeling (BIM) techniques are also applied to implement the answering module of the chatbot. The preliminary testing results show that the chatbot can successfully predict the intents and entities behind the inquiries raised by site managers with satisfactory accuracy for both intent prediction and the answer. These results provide site managers with alternative means to retrieve the information they need. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

21 pages, 5397 KiB

Open AccessArticle

Fast Detection of Missing Thin Propagating Cracks during Deep-Learning-Based Concrete Crack/Non-Crack Classification

by Ganesh Kolappan Geetha, Hyun-Jung Yang and Sung-Han Sim

Sensors 2023, 23(3), 1419; https://doi.org/10.3390/s23031419 - 27 Jan 2023

Cited by 10 | Viewed by 2831

Abstract

Existing deep learning (DL) models can detect wider or thicker segments of cracks that occupy multiple pixels in the width direction, but fail to distinguish the thin tail shallow segment or propagating crack occupying fewer pixels. Therefore, in this study, we proposed a [...] Read more.

Existing deep learning (DL) models can detect wider or thicker segments of cracks that occupy multiple pixels in the width direction, but fail to distinguish the thin tail shallow segment or propagating crack occupying fewer pixels. Therefore, in this study, we proposed a scheme for tracking missing thin/propagating crack segments during DL-based crack identification on concrete surfaces in a computationally efficient manner. The proposed scheme employs image processing as a preprocessor and a postprocessor for a 1D DL model. Image-processing-assisted DL as a precursor to DL eliminates labor-intensive labeling and the plane structural background without any distinguishable features during DL training and testing; the model identifies potential crack candidate regions. Iterative differential sliding-window-based local image processing as a postprocessor to DL tracks missing thin cracks on segments classified as cracks. The capability of the proposed method is demonstrated on low-resolution images with cracks of single-pixel width, captured using unmanned aerial vehicles on concrete structures with different surface textures, different scenes with complicated disturbances, and optical variability. Due to the multi-threshold-based image processing, the overall approach is invariant to the choice of initial sensitivity parameters, hyperparameters, and the sequence of neuron arrangement. Further, this technique is a computationally efficient alternative to semantic segmentation that results in pixelated mapping/classification of thin crack regimes, which requires labor-intensive and skilled labeling. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

16 pages, 17843 KiB

Open AccessArticle

GCP-Based Automated Fine Alignment Method for Improving the Accuracy of Coordinate Information on UAV Point Cloud Data

by Yeongjun Choi, Suyeul Park and Seok Kim

Sensors 2022, 22(22), 8735; https://doi.org/10.3390/s22228735 - 11 Nov 2022

Cited by 6 | Viewed by 2180

Abstract

3D point cloud data (PCD) can accurately and efficiently capture the 3D geometric information of a target and exhibits significant potential for construction applications. Although one of the most common approaches for generating PCD is the use of unmanned aerial vehicles (UAV), UAV [...] Read more.

3D point cloud data (PCD) can accurately and efficiently capture the 3D geometric information of a target and exhibits significant potential for construction applications. Although one of the most common approaches for generating PCD is the use of unmanned aerial vehicles (UAV), UAV photogrammetry-based point clouds are erroneous. This study proposes a novel framework for automatically improving the coordinate accuracy of PCD. Image-based deep learning and PCD analysis methods are integrated into a framework that includes the following four phases: GCP (Ground Control Point) detection, GCP global coordinate extraction, transformation matrix estimation, and fine alignment. Two different experiments, as follows, were performed in the case study to validate the proposed framework: (1) experiments on the fine alignment performance of the developed framework, and (2) performance and run time comparison between the fine alignment framework and common registration algorithms such as ICP (Iterative Closest Points). The framework achieved millimeter-level accuracy for each axis. The run time was less than 30 s, which indicated the feasibility of the proposed framework. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

22 pages, 1093 KiB

Open AccessArticle

Building Heating and Cooling Load Prediction Using Ensemble Machine Learning Model

by Rajasekhar Chaganti, Furqan Rustam, Talal Daghriri, Isabel de la Torre Díez, Juan Luis Vidal Mazón, Carmen Lili Rodríguez and Imran Ashraf

Sensors 2022, 22(19), 7692; https://doi.org/10.3390/s22197692 - 10 Oct 2022

Cited by 27 | Viewed by 3501

Abstract

Building energy consumption prediction has become an important research problem within the context of sustainable homes and smart cities. Data-driven approaches have been regarded as the most suitable for integration into smart houses. With the wide deployment of IoT sensors, the data generated [...] Read more.

Building energy consumption prediction has become an important research problem within the context of sustainable homes and smart cities. Data-driven approaches have been regarded as the most suitable for integration into smart houses. With the wide deployment of IoT sensors, the data generated from these sensors can be used for modeling and forecasting energy consumption patterns. Existing studies lag in prediction accuracy and various attributes of buildings are not very well studied. This study follows a data-driven approach in this regard. The novelty of the paper lies in the fact that an ensemble model is proposed, which provides higher performance regarding cooling and heating load prediction. Moreover, the influence of different features on heating and cooling load is investigated. Experiments are performed by considering different features such as glazing area, orientation, height, relative compactness, roof area, surface area, and wall area. Results indicate that relative compactness, surface area, and wall area play a significant role in selecting the appropriate cooling and heating load for a building. The proposed model achieves 0.999

R^{2}

for heating load prediction and 0.997

R^{2}

for cooling load prediction, which is superior to existing state-of-the-art models. The precise prediction of heating and cooling load, can help engineers design energy-efficient buildings, especially in the context of future smart homes. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

19 pages, 7180 KiB

Open AccessArticle

Damage-Detection Approach for Bridges with Multi-Vehicle Loads Using Convolutional Autoencoder

by Kanghyeok Lee, Seunghoo Jeong, Sung-Han Sim and Do Hyoung Shin

Sensors 2022, 22(5), 1839; https://doi.org/10.3390/s22051839 - 25 Feb 2022

Cited by 7 | Viewed by 2313

Abstract

Deep learning has been widely employed in recent studies on bridge-damage detection to improve the performance of damage-detection methods. Unsupervised deep learning can be effectively utilized to increase the applicability of damage-detection approaches. Hence, the authors propose a convolutional-autoencoder (CAE)-based damage-detection approach, which [...] Read more.

Deep learning has been widely employed in recent studies on bridge-damage detection to improve the performance of damage-detection methods. Unsupervised deep learning can be effectively utilized to increase the applicability of damage-detection approaches. Hence, the authors propose a convolutional-autoencoder (CAE)-based damage-detection approach, which is an unsupervised deep-learning network. However, the CAE-based damage-detection approach demonstrates only satisfactory accuracy for prestressed concrete bridges with a single-vehicle load. Therefore, this study was performed to verify whether the CAE-based damage-detection approach can be applied to bridges with multi-vehicle loads, which is a typical scenario. In this study, rigid-frame and reinforced-concrete-slab bridges were modeled and simulated to obtain the behavior data of bridges. A CAE-based damage-detection approach was tested on both bridges. For both bridges, the results demonstrated satisfactory damage-detection accuracy of over 90% and a false-negative rate of less than 1%. These results prove that the CAE-based approach can be successfully applied to various types of bridges with multi-vehicle loads. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

19 pages, 2885 KiB

Open AccessArticle

Heat Balance Calculation and Energy Efficiency Analysis for Building Clusters Based on Psychrometric Chart

by Shihai Yang, Huiling Su, Xun Dou, Mingming Chen and Yixuan Huang

Sensors 2021, 21(22), 7606; https://doi.org/10.3390/s21227606 - 16 Nov 2021

Cited by 3 | Viewed by 3188

Abstract

How to perform accurate calculation of heat balance and quantitative analysis of energy efficiency for building clusters is an urgent problem to be solved to reduce building energy consumption and improve energy utilization efficiency. This article proposes a method for the heat balance [...] Read more.

How to perform accurate calculation of heat balance and quantitative analysis of energy efficiency for building clusters is an urgent problem to be solved to reduce building energy consumption and improve energy utilization efficiency. This article proposes a method for the heat balance calculation and energy efficiency analysis of building clusters based on enthalpy and humidity diagrams and applies it to the energy management of building clusters containing primary return air systems and heating pipe networks. Firstly, the basic structure and energy management principle of building clusters with a primary return air system and a heating pipe network were given, and the heat balance calculation and energy efficiency analysis method based on i-d diagram was proposed to realize the accurate calculation of heat load and the quantification of energy utilization. Secondly, the energy management model of the building cluster with a primary return air system and a heating pipe network was established to efficiently manage the indoor temperature and the heating schedule of ASHP, HN and HI. Finally, the proposed method was validated by calculation examples, and the results showed that the proposed method is beneficial for improving the energy economy and energy efficiency of building clusters. Full article

(This article belongs to the Special Issue AI and Big Data for Smart Construction)

► Show Figures

Figure 1

Journal Menu

Journal Browser

AI and Big Data for Smart Construction

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (11 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI