A Deep Learning-Based Approach to Failure Detection in Mooring (Thin) Lines from Marine Images ^†

Abstract

Mooring systems are incorporated from mooring (thin) lines that are constituted of fiber ropes, steel wires, and chains. Mooring systems are used for the stationary keeping of floating units during the drilling process of oil and gas from offshore deep water, and the unloading of productions to the shuttle storage tanker. However, it is crucial to monitor mooring systems for early-stage failure detection in mooring lines during offshore mooring operations to avoid any unexpected losses, including human injuries, and catastrophic failure. This paper addresses the challenges of mooring line detection, and proposes a deep learning-based approach for the detection of mooring lines from marine images using the bounding box. A convolutional neural network, Inception v3, is used for the detection and classification of thin-line objects from marine images, and it is a pre-trained model with 1000 classes. Furthermore, various testing samples have been evaluated for assessing the performance of the pre-trained proposed model. According to the results, it has been observed that the proposed model obtained the highest accuracy (87.33%) in classifying the mooring line objects from images, but failed to accurately detect mooring lines. Furthermore, in a few highlighted cases, the performance of the model decreased in terms of accuracy due to the misclassification and wrong detection of mooring line objects. Despite this, the proposed study furnishes a potential solution for the detection of failure in mooring lines from marine images.

1. Introduction

Mooring systems are made up of mooring lines, which are commonly made of polyester ropes, chains, and steel wire ropes. These elements are critical in securing offshore boats and stationary items during deep-water mooring operations. The number of mooring lines within each type of mooring system or design varies according to the individual floating units utilized for offshore activities such as drilling or transferring hydrocarbon production to shuttle tankers [1]. Mooring lines are extremely important in the context of offshore maritime activities due to the potentially disastrous repercussions of their failure. These repercussions include significant financial losses, human life losses, production disruptions [2,3], and environmental concerns caused by petroleum leaks. These undesirable occurrences are often caused by mooring line abnormalities, which might include corrosion, wire breakage, and poor maintenance, ultimately leading to mooring line failures [4]. Importantly, when a mooring line fails, it adds tension to the other mooring lines, possibly leading to a disastrous scenario [3,5]. As a result, mooring line maintenance and monitoring techniques must be implemented on a regular basis. These procedures serve the dual function of preventing abnormalities that might jeopardize mooring lines, while also limiting the environmental concerns associated with mooring system failures [6].

Over the last decade, researchers have conducted a variety of studies aiming to discover mooring line abnormalities and create real-time mooring monitoring solutions for the detection of such anomalies. Study [7] proposed a long short-term model with an attention mechanism to monitor the underwater mooring lines fixed at the seafloor to accomplish floating-point storage and offloading (FPSO) operations safely. A deep learning-based approach was proposed to predict mooring line tension using LSTM and reservoir computing (RC) [8]. The study proposed by [9] developed a deep neural network–grid search (DNN-GS) model to identify dynamic mooring line tensions in single-line failure conditions. Moreover, an artificial neural network (ANN)-based model was proposed by [5] to inspect mooring line tension for offshore vessels fixed at the seabed under deep water. The deformation of the illustrated anchor chain induces fatigue failure in the maritime environment under dynamic stress. In this regard, the researchers proposed a study [10] to optimize the design framework with minimal cost for the mooring anchor chain.

Furthermore, the deep learning-based LSTM model was proposed by [11,12] for mooring line load monitoring to perform offshore mooring operations safely. They proposed a Box–Cox transformation (BCT) model to improve the predicting accuracy. Additionally, a global navigation satellite system (GNSS) was proposed by [13] to monitor the mooring systems. The proposed work relies on data that comprise changes in the position of the vessel in the convergence of offshore mooring operations. The study [14] proposed a deep learning model for early stage failure detection in the mooring lines of submerged floating tunnels (SFTs). Damage detection in submerged mooring lines is critical to ensuring the safety of offshore mooring operations. In keeping with this viewpoint, researchers developed a recursive neural network (RNN) architecture based on deep learning principles to detect flaws in underwater catenary mooring lines. This detection approach is based on analyzing a single response generated by offshore platform modeling and the accompanying environmental data [15].

With the increasing expansion of artificial intelligence, several academics have focused on deep learning and machine learning algorithms for detecting and monitoring abnormalities in mooring lines, but these are less accurate and less reliable, and the performance of the models is degraded when implementing the techniques in strong seaway conditions. Hence, there is still room to develop a real-time mooring monitoring system that can obviate unexpected failure in mooring lines and detect anomalies at an early stage during offshore mooring operations. This paper addresses the challenges of mooring line detection and proposes a deep learning-based approach for the detection of mooring lines from marine images using the bounding box. A convolutional neural network, Inception v3, is used for the detection and classification of thin line objects from marine images.

The rest of the paper has been organized as follows. Section 2 describes the methodology used for dataset collection and the framework designed for the detection of mooring lines from images. Section 3 presents the results and furnishes a discussion of the analysis conducted for the proposed model, and compares the results with those of existing studies. Lastly, the conclusion drawn from the research work has been demonstrated in Section 4.

2. Materials and Methods

2.1. Description of Dataset

No publicly available marine images dataset comprising mooring (thin) lines has previously existed. Therefore, standard software “OrcaFlex” was used to create a synthetic dataset of marine images consisting of mooring lines. OrcaFlex, known for its user-friendliness and depth of technical competence, is the top software program in the world for the dynamic study of offshore marine systems [16]. Using this software program, we created eight 3D simulations by incorporating the concept of a spread mooring system; each simulation video lasted for 12 s and featured different positions of floating-point storage and offshore (FPSO) vessels. However, each simulation was extracted with a frame rate of 20 fps (frames per second). Furthermore, standard video-to-image conversion software was utilized to extract the individual frames from each of the extracted video simulations, and the resulting frames were saved in a “PNG” image format for further analysis and preprocessing. Moreover, for each output frame, the image width and height were set to −1, preserving the original dimensions of the simulation. In this way, a total of 150 marine images consisting of mooring lines were collected. Furthermore, the final training dataset was subjected to image augmentation.

2.2. Image Augmentation

Image augmentation is the most commonly used computer vision technique. Image data augmentation is a process used to generalize the performance of the model and enhance the capabilities of the model learning. It is a procedure used to add new images to the dataset and expand the size of the training dataset artificially. The most commonly used image augmentation techniques are position augmentation and color augmentation [17]. However, for the proposed work, the training dataset was randomly augmented 12 times using rotation, horizontal and vertical skew within the range of 20 degrees, and 20% scaling in the height and width of images; lastly, horizontal and vertical flip was performed to enhance the size of training dataset for the proposed deep learning model to further avoid overfitting and enhance the performance of the model. In addition to that, we carried out image rescaling. The dataset images were rescaled to 299 × 299 dimensions and were used to input the proposed deep CNN network called Inception v3.

2.3. Architecture of the Inception v3 Model

An improved version of the GoogLeNet Inception v1 model is Inception v3. This model integrates several network performance optimization approaches, which enhances flexibility. In comparison to Inception v1 and v2, it promises improved efficiency, a deeper network structure, lower processing requirements, and the use of auxiliary classifiers as regularization tools [18].

The Inception v3 architecture, well known for its exceptional performance in the 2014 ImageNet Large Scale Visual Recognition Challenge, underwent early training utilizing a dataset made up of around 1,280,000 images covering 1000 different object categories. The use of convolutional filters of different sizes inside the same layer is made possible by the architecture’s total of 48 deep layers, which facilitates the extraction of information at various scales. The main modifications made to the Inception v3 model include significant improvements in factorization into smaller convolutions, spatial factorization through asymmetric convolutions, use of auxiliary classifiers, and the application of a successful grid size reduction strategy [19].

A pre-trained model, known as the Inception v3 convolutional neural network (CNN), was employed in the proposed work due to its impressive performance [20] in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). The architecture of the Inception v3 model is depicted in Figure 1. The dataset has been trained using transfer learning. This is because the performance of the deep learning models and training time are improved when the size of the dataset images is small. Thus, the proposed model was trained by applying the transfer learning technique, and all the learned attributes found from training the Inception v3 model on the ImageNet dataset were recycled. Moreover, an extensive configuration of nine inception modules was used in the investigation, together with an auxiliary classifier, two fully connected layers, and softmax functions [14]. A learning rate of 0.001 was used throughout the training phase, which involved randomly dividing the training dataset into 24 batches for each epoch and running the training method for a total of 500 epochs. The model’s weights were optimized, and its output capacity was increased by adjusting the hyperparameters, all of which helped with fine-tuning to improve the identification of mooring lines objects in the images.

3. Results and Discussion

This section determines the results and discussion of the Inception v3 model in the detection of mooring lines from marine images. However, to check the performance of the model for new images, a confusion matrix is usually employed. Thus, the performance of the model in the detection of mooring lines has been measured based on the various performance metrics. These include accuracy, sensitivity, specificity, precision, and F1 score.

The results of the proposed deep learning model have been computed based on the said metrics. In this view, it has been observed that the trained Inception v3 model obtained a very good accuracy of 87.33%, 93.27% sensitivity, 73.91% specificity, 88.99% precision, and a 91.09% F1 score in the accurate detection of mooring lines, as shown in

Table 1. Inception v3 model training accuracy in the detection of mooring lines from images.

Model	Detection	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)	F1 Score (%)
Inception v3	Mooring Lines	87.33	93.27	73.91	88.99	91.09

.

However, the same metrics have also been used to compute the performance of the Inception v3 model with new input images for the detection of mooring lines. We evaluated the performance of the trained model based on three different input cases to validate the testing accuracy on novel input images, as shown in Figure 2. The results of all three cases are depicted in

Table 2. Inception v3 model testing accuracy in the detection of mooring lines from new input images.

Model	Test Cases	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)	F1 Score (%)
Inception v3	(a)	82.67	90	68	84.91	87.34
	(b)	67.33	79.12	49.15	70.59	74.64
	(c)	72.67	78.26	63.79	77.42	77.84

. In the evaluation of the first case, it is reported that the Inception v3 model detected the mooring lines in the position of the boat stern, as well as at 60 degrees from the boat bow, but could not accurately detect all mooring lines at 30 degrees from the boat bow. This resulted in a slightly low accuracy of 82.67%, 90% sensitivity, 68% specificity, 84.91% precision, and a 87.34% F1 score, as shown in Figure 2a.

Furthermore, in the evaluation of the second case, the model failed to detect mooring lines and instead localized the boat object. Thus, for test case 2, the model accomplished a very low accuracy of 67.33%, and specificity of 49.15% in consideration of detection of mooring lines, as depicted in Figure 2b. In test case 3, the model performed satisfactorily. However, it could not accurately detect the mooring lines because of mistakenly localizing the mooring lines with the boat in focus, which is not the object of interest in the proposed study; this led to a low accuracy of 72.67%, and 63.79% specificity, as shown in Figure 2c. In addition to the above discussion, it has been observed that the present study has shown encouraging results in comparison with existing studies, as depicted in

Table 3. Comparison of the present study’s results with those of existing studies based on specific parameters.

Reference	Models	Specific Parameters	Accuracy (%)
[9]	DNN-GS	Mooring (case 3)	81.7
[11]	LSTM	Sea state 6	70.33
[14]	DNN	Sensor location (2,5,8)	84.1
[15]	RNN	Measurement level 1	76.58
Proposed Work	CNN (Inception v3)	Clear water	87.33

.

It is crucial to recognize a constraint in the created dataset and model training procedure in the context of mooring line detection. The impacts of water turbidity on marine life are not taken into account in the training images that are modelled. The visibility and behavior of aquatic species can be dramatically impacted by water turbidity, which is caused by the presence of suspended particles in the water. While our model has shown encouraging results in clear water, it is vital to remember that real-world marine settings frequently have varied degrees of turbidity. Future studies could look at expanding the model’s applicability to other aquatic environments by including turbidity effects.

4. Conclusions

This study deals with the crucial problem of mooring line early-stage failure detection in offshore mooring systems, which are essential parts for securing vessels during deep-water operations. The effects of mooring line failure are extensive, including significant financial losses, human injuries, interruptions in productivity, and environmental harm. Although previous research has looked into a variety of methods for tracking and forecasting mooring line behavior, there is still a great need for a dependable real-time mooring monitoring system that can pick up on anomalies and potential failures during offshore mooring operations, especially in difficult seaway conditions. In this study, a pretrained deep learning-based model for mooring (thin) line detection from marine images using bounding boxes has been presented. The accuracy rate of recognizing thin line objects in marine images using the CNN Inception v3 is 87.33%, which is an encouraging result. It is crucial to recognize the limitations of the model, which can result in inaccurate mooring line object recognition, misclassification, and lower accuracy in particular instances. In spite of these difficulties, this work marks a substantial advancement in the field of mooring line failure identification from maritime images. It lays the groundwork for possible answers, and emphasizes the demand for more research and progress in this area.