Design and Simulation-Based Validation of an AI Model for Predicting Grab-Type Ship Unloader Operation Data

Abstract

Along with seaports automation, there is growing interest in the automation of Grab-Type Ship Unloader (GTSU) that unloads coal and iron ore from bulk carriers. Autonomous unloading operations of GTSU offer the potential for significant productivity improvement and cost savings. In this paper, an AI model trained with manual operation data was designed for GTSU automation operation, and the AI model was verified through the equation-of-motion-based GTSU operation simulator. The operation data of hoist, grab, and trolley were predicted by training the designed AI model with the manual operation data of GTSU. Before applying the predicted data to the actual equipment, the predicted driving data was verified using the equation-of-motion-based GTSU operation simulator. The AI prediction model was designed using the Multi-Layer Perception network, a type of artificial neural network. The AI prediction model was evaluated with the Mean-Squared Error indicator, and the validation loss was found to be less than 0.02. In addition, verification of the prediction data was performed using the GTSU dynamics-based simulator. The Mean Relative Error was up to 0.50, and the R² score value exceeded 0.92, indicating that the model is effective in predicting operation data. The proposed AI prediction model will be effectively utilized to implement a fully automated unloading system.

1. Introduction

Industrial automation in factories increases production levels, reduces energy consumption, and produces consistent, reliable, and high-quality products. This leads to cost savings and improved productivity [1]. Automation is transforming businesses and is becoming a key factor in the global economic environment [2]. Currently, the industry is rapidly transitioning to an automated machine-based production system that replaces human capabilities with the advent of Artificial Intelligence (AI). The history of AI began in the 1950s, and with the advent of deep learning technology in the 2000s, the application of AI has already achieved significant results in various industries, with practical applications already demonstrated [3]. AI aims to program intelligence into machines by simulating human decision-making and reasoning processes, learning from experience, and adapting to environmental changes [4]. For this reason, AI technology is being used as a tool to develop automated systems that replace humans [5]. Many developers of AI systems now recognize that, for many applications, it can be far easier to train a system by showing it examples of the desired input-output behavior than to program it manually by anticipating the desired response for all possible inputs [6].

Ports are key players in receiving, unloading, and transshipping goods and minerals, as critical nodes in the maritime transportation network [7]. Generally, the process of transferring raw materials from ports to industrial companies is performed through unloaders such as Continuous Ship Unloader (CSU), Grab-Type Ship Unloader (GTSU), Stacker, and Reclaimer [8]. Among unloading machines, the GTSU plays an important role in quickly and safely unloading raw materials from bulk carriers. The GTSU is a grab type that can handle various bulk materials, has a low initial capital cost, and is easily adaptable to various port conditions and vessel sizes. It is mounted on rails or rubber tires and can move along the dock for efficient operation [9]. Currently, GTSUs are operated manually in Korea. Although manual operation can be effective, it frequently presents difficulties such as longer processing times, higher labor expenses, and occasional mistakes that could lead to material loss or, in more severe cases, accidents. To improve the many inefficiencies that occur in manual unloading operations, increase productivity, and ensure safety, companies are rapidly accelerating the application of automated unloading systems.

Seaports are also being transformed into ‘smart ports’ equipped with unmanned automation technology. The Port of Rotterdam, the largest port in Europe, introduced the world’s first unmanned automatic loading and unloading system in 2015. Unmanned ports operate 24 h a day, and AI technology greatly contributes to port automation [10]. When a ship enters a port, the ship-to-shore (STS) crane that unloads containers is equipped with cameras and sensors, and AI issues unloading commands to the crane. After unloading the containers, AI performs an efficient unloading process by giving a command to the Automated Guided Vehicle (AGV) to carry the container. The unmanned automated port reduced container unloading time by 40%, reduced labor and fuel costs by 37%, and increased productivity by 40%. To complete the smart port construction process, the GTSU automation for the transportation of raw materials such as coal and iron ore must be researched and developed. Port unloader automation is closely related to robotics [11]. The convergence of port automation and robotics provides significant benefits in many areas, including efficiency, cost savings, safety, precision, flexibility, environmental aspects, and enhanced global competitiveness [12]. The concept of the GTSU automation technology as part of port automation technology is closely related to robotic system control. The GTSU automation technology, which requires precise position control of the grab and flexible control of the wire rope, can be combined with cartesian and tendon-driven robots. The cartesian robot system can be applied in port automation to move cargo to a precise position through linear motion along the x, y, and z axes, so it can be used to control the grab to a precise position [13]. The tendon-driven robotic systems can be adapted to move flexibly in a variety of directions, so it can be used to rotate the grab and control its sway [14]. Additionally, this system must utilize cutting-edge sensor technology, AI, big data, and the Internet of Things. The GTSU automation aims to replace human tasks with sophisticated sensors and control systems [15]. As one of the GTSU automation functions in smart ports, manual operation methods must be replaced by automated algorithms. Regarding the recent major research trends related to GTSU automation, Hanbiao et al. presented a cooperative localization technology of a dynamic cleaning robot and a grabber [16]. This paper deals with technology to acquire sensor-based location information, control it with a cooperative location estimation algorithm, and adjust the location estimation algorithm by detecting real-time environmental changes. Therefore, not a technology that directly and automatically controls the GTSU but a technology that unloads it through a dynamic cleaning robot and grabber is being proposed. Ngo et al. presented a vision AI model for the GTSU automation system [17]. The main content of this study is the development and testing of cargo hold self-identification, determination of operating point coordinates, and provision of collision risk warning during operation using vision AI, but the content of operation automation of the GTSU is excluded. The iSAM AG in Germany has presented automated systems for unloading coal and iron ore and is also actively working on the raw material seaports [18]. This system operates according to predefined rules and is controlled through real-time feedback, which limits its ability to adapt to new work patterns or unexpected situations. On the other hand, an automatic operation using an AI can reflect the operator’s experience and judgment, allowing for flexible responses even in unexpected situations, so research on this is necessary.

This paper developed an AI-based GTSU operation data prediction model that operates similarly to the operator’s work patterns to build an automated system. Parameters related to automated operation were selected and trained in the manual operation data. An AI model was designed to predict operation data for Hoist, Grab, and Trolley through learning from GTSU’s manual driving data. Multi-Layer Perception (MLP) and the Long-Short Term Memory (LSTM) technique were designed and compared to develop an AI-based GTSU operation data prediction model. Before applying the predicted data to the actual operation, the prediction data was verified using the equation-of-motion-based GTSU operation simulator. As a result of comparing the MSE of the two networks, it was analyzed that there was little difference in prediction performance with a slight difference. Therefore, the MLP model, which has a simpler structure than the LSTM model, was adopted to predict the GTSU operation data. To verify the AI model, a dynamics-based simulation was performed. The Mean Relative Error (MRE) and R² score were used as indicators to compare the simulation results with the prediction results. When simulated with AI prediction data, it was found to be somewhat different from the actual movement location data. To compensate for this, post-processing was performed to compensate for the difference in movement amount. As a result of the simulation with post-processed data, the MRE was found to be up to 0.50, and the R² score was over 0.92, verifying that the post-processed AI prediction data was well in line with the operation order of the actual data. The proposed AI prediction model will be effectively utilized to implement a fully automated unloading system.

2. Overview of the GTSU

2.1. Operation Mechanism of the GTSU

GTSUs are generally used to handle materials such as coal and iron ore. These machines are designed with a seaborne arm that can be raised and lowered together with a cable-operated grab trolley. The drive winches that manage all the working movements (i.e., grab lifting, grab lowering, grab opening and closing, cross travel, and boom lifting and lowering) are all mounted on the main structure. Due to the relatively small mass of the grab, high acceleration and trolley speeds can be achieved [19]. The GTSU incorporates a hopper that feeds the bulk materials directly to a conveyor belt system [20]. The main unloading process of a GTSU can be described as a grab that picks up the unloader from the bulk ships and then drops it onto the hopper [21,22]. Figure 1 shows the structure of a general GTSU.

The main operation mechanism of the GTSU is composed of three parts: hoist, grab, and trolley. Each part is lifted, opened, and closed by a motor-driven drum. In particular, the grab control drum shares the load with the hoist control drum during hoisting and lowering. Therefore, it is necessary to process grab control data when analyzing manual operation data.

2.2. Operation Process of the GTSU

Analysis of the operation process was conducted to understand the operating characteristics of the GTSU. The unloading start point was defined as when the grab was open at the top of the hopper. The operation process of the GTSU consists of a series of actions, as shown in Figure 2. A detailed description of the GTSU’s operation process is given in

Table 1. Description of the GTSU’s operation process.

No.	Operation State	Details
1	Trolley travers to seaside	The grab is fully open at the initial start, and the trolley moves toward the seaside to position the grab from the top of the hopper to the bulk carrier.
2	Trolley travers to seaside and lowering	The lowering operation is carried out simultaneously with the trolley crossing the seaside from the area where the grab does not collide with the bulk carrier.
3	Lowering	Lowering is performed continuously to bring the grab into the cargo hold.
4	Grab bucket settling	Place the grab on the bulk in the cargo hold.
5	Closing on the stack	Closing and hoisting operations are performed simultaneously for picking up bulk using Grab.
6	Hoisting	Hoisting the grab under the full bucket state.
7	Hoisting and trolley traverse to landside	Hoisting operation is performed simultaneously with the trolley traversing to landside from the area where the grab does not collide with the bulk carrier.
8	Trolley traverses to landside	The trolley continuously travers to landside to discharge the bulk into the hopper.
9	Grab bucket opens to discharge	After the grab is located at the top of the hopper, it opens to discharge the bulk.

.

The GTSU unloads bulk by repeatedly performing a one-cycle operation process. When only about 15% of the material remains, the work cycle of the GTSU is complete, and the remaining work is done using a bulldozer.

3. Design of an AI Model for Predicting the GTSU Operation Data

3.1. Datasets

In this study, time-series data, including about 400 cycles of GTSU manual operation data, were used. The average one-cycle time of the training data was 84.7 s, and a total of 9.4 h of unloading data was collected as the training dataset. When training the neural network for the design of the operation data prediction model, the data in the normal operation of the GTSU were used. For the automation of the GTSU operation, the target should predict the operation data of the main driving parts, including the hoist, grab, and trolley. Therefore, encoder data of GTSU’s main driving parts were collected. In the case of the gantry in charge of the GTSU traveling, it was judged to be a separate automation function due to issues with receiving information related to the bulk cargo’s location, and it was excluded from the scope of automation in this study. The parameters of the training dataset are the position and speed data of the main parts and are listed in

Table 2. Parameters of the GTSU operation data.

Driving Part	Parameters	Variables	Unit
Hoist	Hold position of the hoist	HOLD POSITION	m
	Real speed of the hoist	HM_DRV_Actual_Speed	m/s
Grab	Close position of the grab	CLOSE POSITION	a.u.
	Real speed of the grab	CM_DRV_Actual_Speed	m/s
Trolley	Traversing position of the trolley	TROLLEY POSITION	m
	Real speed of the trolley	TR_DRV_Actual_Speed	m/s

. Here, the input data is the position data of the entire main driving parts, and the output data is the speed data of only one main driving part.

The data sample related to parameters selected from the operation data is shown in Figure 3. The position and speed data of GTSU’s main driving parts were collected in m and m/s, but GTSU’s operational data was displayed with the data scale normalized to 0 to 1 due to data security concerns. However, the position and speed data of each driving part were collected within the range according to the equipment specifications. The lift length is 45 m in the case of hoisting, and the out-reach and back-reach are 42 m and 40 m, respectively, in the case of traversing. The grab position data are arbitrary units because the data are collected as the difference between the hoist motor encoder value and the grab motor encoder value. In the grab position data, a value of 1 means open state, and a value of 0 means closed state.

The real speed data of the grab includes not only the grab open and close states but also hoisting and lowering states because the grab control motor operates by sharing the load during hoisting and lowering with the hoist control motor. Since the operation includes grab motor control signals during hoisting and lowering, extracting only grab open and close data was not considered.

3.2. Data Preprocessing

In the case of the GTSU fully operation data, data from non-operating times, such as bulk carrier arrival and departure times and operator shift times, are also included. Since the GTSU automated system should continue unloading except in situations where stopping is necessary, all sections with no change in operating data were excluded from the dataset. In addition, a dataset was constructed by selecting only data when driven according to the GTSU operation process mentioned in Section 2, and a sample of the selected dataset is shown in Figure 4.

In the case of GTSU manual operation, the operator visually checks the bulk in the cargo hold, determines where the grab bucket will settle down, and then controls the equipment to perform unloading. Therefore, it is necessary to collect not only driving unit control data but also unloading points within the cargo hold to train the manual operation pattern. Since the unloading point scanning data by a sensor was not considered in this study, the unloading point data was estimated by extracting the positions of the hoist and trolley at the point when the grab starts closing. Unloading point data is expressed as x, y, and z values. The x-axis value represents the trolley position, the y-axis value represents the gantry position, and the z-axis value represents the hoist position. In the case of the gantry, a separate automation function was implemented, so the x-axis and z-axis of the unloading point were implied as input datasets.

As a data scaling method, MinMaxScaler was used as Equation (1) [23].

$${x_i}_{new}={\displaystyle \frac{x_i-x_{min}}{x_{max}-x_{min} }}$$

(1)

The MinMaxScaler is obtained by changing the basic estimate of the corresponding element to 0 for each component, the most extreme value to 1, and scaling the data values to a value in the range between 0 and 1. In Equation (1), $x_{min}$ is the minimum value of data, and $x_{max}$ is the maximum value of data.

3.3. AI Algorithms for Predicting the GTSU Operation Data

Machine learning uses supervised learning when there are output variables for input variables and unsupervised learning when there are no output variables for input variables. In addition, supervised learning is divided into regression and classification based on whether the type of the output variable is a numeric variable or a categorical variable. The input and output variables used for the machine learning of the operation prediction model are numerical variables, so the machine learning used a regression model of supervised learning.

3.3.1. Multi-Layer Perception Network

When trying to predict GTSU driving data, which is nonlinear data, it is generally more appropriate to consider artificial neural networks rather than polynomial regression. A multi-layer perception (MLP) network is a typical representative of feedforward artificial neural networks [24]. An MLP is a neural network with two or more layers. An MLP consists of an input layer, a hidden layer, and an output layer. A multi-layer perceptron with only one hidden layer is called a shallow neural network, and a multi-layer perceptron with two or more hidden layers is called a deep neural network. Unlike single-layer perception, multi-layer perceptron can learn from nonlinearly distributed data. Because the multi-layer perception calculates linear equations for weights, it uses an activation function to convert linear data into nonlinear data when passing between layers. The MLP network can distinguish the nonlinear data relationship. In this paper, the MLP network is composed of six layers that contain input, four hidden layers, and an output layer. The input layer has five nodes that contain hoist position, grab status, trolley position, unloading point x-axis value, and unloading point z-axis value. The output layer has one node representing the speed of the main driving part. The structure of the MLP network for predicting the GTSU operation data is shown in Figure 5.

The specifications of the MLP network for predicting the GTSU operation data are shown in

Table 3. Specifications of the MLP network.

Specifications	Value
Number of inputs	5
Number of outputs	1
Number of hidden layers	4
Number of neurons in each hidden layer	36
Activation function in hidden layers	ReLU
Activation function of the output layer	Linear
Learning function	Adam
Epoch	50
Batch size	50

. Three MLP models were designed with the same specifications to predict the operation data of the GTSU by predicting the speed of each driving part. The outputs of these models are the hoist speed, grab speed, and trolley speed, respectively.

3.3.2. Long Short Term Memory

To solve gradient vanishing of the Recurrent Neural Network (RNN), exploding, and long-term dependencies problems, Long Short Term Memory (LSTM), which can have long-term memory, was proposed [25]. LSTM is useful for predicting time-series data because it retains important information for a long period of time and forgets unnecessary information through cell states and gate mechanisms. LSTM consists of cell states and four main gates. These gates improve the learning ability of the LSTM by selectively passing or blocking information. Cell state serves as the memory of the LSTM, flowing linearly and transferring information throughout the sequence without loss of information. The gate outputs values between 0 and 1 using sigmoid and hyperbolic tangent (tanh) activation functions, respectively. The configuration of the LSTM network is shown in Figure 6.

The four gates of the LSTM are composed of a forget gate $f_{l,t}$, an input gate $i_{l,t}$, a new memory unit $n_{l,t}$, and an output gate $o_{l,t}$, where $l$ is the layers, $t$ is the time step, $\sigma$ is the activation function, $U$ and $W$ are the weight matrices, and $h$ and $b$ are the bias vectors on the memory cell, respectively.

In an LSTM network, the forget gate is calculated as follows:

$$f_t=\sigma (W_f·\left[h_{t-1}, x_t\right]+b_f)$$

(2)

The input gate is calculated as follows:

$$i_t=\sigma (W_i·\left[h_{t-1}, x_t\right]+b_i)$$

(3)

The new memory unit is calculated as follows:

$$n_{l,t}=\mathit{tanh}(W_c·\left[h_{t-1}, x_t\right]+b_c)$$

(4)

The output gate is calculated as follows:

$$o_t=\sigma (W_o·\left[h_{t-1}, x_t\right]+b_o)$$

(5)

The hidden state and output cell state are calculated as follows:

$$h_{l,t}^k=\mathit{tanh}\left(c_{l,t}^k\right)\ast o_{l,t}$$

(6)

$$c_{l,t}^k={\mathrm{i}}_{\mathrm{l},\mathrm{t}}\ast n_{l,t}+f_{l.t}\ast c_{l,t-1}^k$$

(7)

The prediction vector ${\widehat{y}}^t$ is calculated using Equation (8).

$${\widehat{y}}^t=\sigma (W^{out}·{\widehat{y}}^{k,t}+b^{out})$$

(8)

The specifications of the LSTM network for predicting the GTSU operation data are shown in

Table 4. Specifications of the LSTM network.

Specifications	Value
Number of inputs	5
Number of outputs	1
Number of hidden layers	4
Number of neurons in each hidden layer	36
Activation function in hidden layers	ReLU
Activation function of the output layer	Linear
Learning function	Adam
Epoch	50
Batch size	50

. Similar to the MLP model, three LSTM models were designed with the same specifications to predict the operation data of the GTSU by predicting the speed of each driving part. The outputs of these models are the hoist speed, grab speed, and trolley speed, respectively.

3.4. Model Testing and Comparison

The MLP and the LSTM models for predicting the GTSU operation data were designed and trained. In general, AI models evaluate their overall performance through an evaluation indicator. In this study, mean squared error (MSE), which has the characteristic of reducing large errors and optimizing the predicted value to be closer to the average of the actual value, was selected as an evaluation index. The MSE was calculated using the predicted operation data and the real operation data of the GTSU by Equation (9).

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2$$

(9)

where n is the number of data, $y_i$ is the actual data, and $\widehat{y_i}$ is the predicted data.

Calculation results of the MSE are applied to evaluate the prediction of the main driving part speed by MLP and LSTM models. Data preprocessing, model evaluation, and hyperparameter optimization of the AI model were performed using the Scikit-learn library. Construction of the AI model was performed using Tensorflow and Keras libraries. In this study, the AI model was developed in a Python environment that provides various machine learning and deep learning libraries and frameworks.

4. Prediction Results and Simulation-Based Verification of an AI Model

4.1. Prediction Results of AI Models

Two AI models were developed to comprehensively predict GTSU operation data by predicting the speed of each driving part. AI models were trained using the same dataset, and the dataset was split into a training set, validation set, and test set in a 70:15:15 ratio.

Figure 7 shows the prediction results of each AI model. The prediction result graphs were created by extracting only five cycles from the test set of each AI model for intuitive viewing. In the case of the MLP model (Figure 7a), the trends between predicted data and actual data were found to be almost identical. In other words, the MLP model can predict the speed of each driving part of the GTSU with a very low residual. In the case of the LSTM model, the tendency between predicted data and actual data appears to be similar to MLP.

To compare the performance of MLP and LSTM models, the MSE values, which are evaluation indices for regression models, were compared and are presented in

Table 5. Evaluation results of the MLP model and LSTM model according to evaluation metrics.

Driving Part	MLP Model		LSTM Model
Hoist	MSETrain	MSEValidation	MSETrain	MSEValidation
	0.006	0.009	0.008	0.009
Grab	MSETrain	MSEValidation	MSETrain	MSEValidation
	0.010	0.015	0.011	0.014
Trolley	MSETrain	MSEValidation	MSETrain	MSEValidation
	0.003	0.005	0.004	0.006

. The MSE values mean that the smaller the value, the closer the model’s prediction is to the actual value.

When comparing the MSE of the two models, excellent fitting performance was confirmed as the MSE values of the training dataset were lower than that of the validation dataset. When comparing the MSE of the MLP and LSTM models, the difference is not significant, indicating that the two models have almost identical performance. In summary, since the performance of the two models is similar, the MLP model, which is relatively simple and follows faster changes in instantaneous speed, was judged to be suitable for predicting the operation of the GTSU.

4.2. Dynamic Simulator of the GTSU for Validation of the AI Model’s Prediction Results

It is very risky to apply AI prediction results directly to heavy equipment such as GTSU. Therefore, it is necessary to verify the prediction results, and an effective verification method is a model-based simulation. To verify the AI-based GTSU motion prediction results, GTSU simulation in a Dymola environment was performed. Dymola is a tool for modeling and simulation of integrated and complex systems for use within automotive, aerospace, robotics, process, and other applications [26]. The full GTSU simulator was modeled using Modelica language in the Dymola environment. The variables of the simulator are detailed in

Table 6. Variables of the GTSU simulator.

Symbol	Description	Unit
$\theta$	Angle of the wire rope	$\mathrm{d}\mathrm{e}\mathrm{g}\mathrm{r}\mathrm{e}\mathrm{e}$
$f_t$	Trolley motor force	$\mathrm{N}$
$f_d$	Load disturbances	$\mathrm{N}$
$x_t$	Position of the trolley	$\mathrm{m}$
$x_g$	Position of the grab	$\mathrm{m}$
$r$	Length of the wire rope	$\mathrm{m}$
$m_t$	Mass of the trolley	$\mathrm{k}\mathrm{g}$
$m_g$	Mass of the grab	$\mathrm{k}\mathrm{g}$
$d_t$	Friction coefficient of the trolley	$\mathrm{k}\mathrm{g}/\mathrm{s}$
$d_g$	Friction coefficient of the grab	$\mathrm{k}\mathrm{g}/\mathrm{s}$
$g$	Gravity	$\mathrm{m}/{\mathrm{s}}^2$

.

The parameters of the simulator model are listed in

Table 7. Parameters of the GTSU motion equation.

Symbol	Description	Value
$d_t$	Friction coefficient of the trolley	$0.5 \mathrm{k}\mathrm{g}/\mathrm{s}$
$d_t^{brake}$	Friction coefficient of the trolley with activated brake	${10}^5 \mathrm{k}\mathrm{g}/\mathrm{s}$
$d_g$	Friction coefficient of the grab	$0.01 \mathrm{k}\mathrm{g}/\mathrm{s}$
$g$	Gravity	$9.81 \mathrm{m}/{\mathrm{s}}^2$
$m_t$	Mass of the trolley	58 Ton
$m_g$	Mass of the grab	27 Ton
$r$	Length of the wire	45 m

. Each parameter was defined based on a simple crane system, and the values were selected considering the specifications of the GTSU.

The GTSU simulator was modeled based on dynamics, and the mathematical modeling of the trolley and grab referred to the cart-pendulum equation as the basic model. The equations of motion of the GTSU model are shown in Equations (10)–(12) [27].

$$\begin{array}{l}\ddot{x_t}\left(m_t+m_g{\mathit{sin}}^2\theta \right)\\ =-d_t\dot{x_t}+f_t+f_d{\mathit{sin}}^2\theta +m_g\mathit{sin}\theta \left(r·\dot{{\theta }^2}+\mathit{gcos}\theta \right)-d_g\dot{x_t}{\mathit{sin}}^2\theta \end{array}$$

(10)

$$\begin{array}{l}{r}^2·\ddot{\theta }\left(m_t+m_g{\mathit{sin}}^2\theta \right)\\ =\left(f_d·{\displaystyle \frac{m_t}{m_g}}-f_t+d_t{\dot{x}}_t\right)rcos\theta \\ -\left(g\left(m_t+m_g\right)+m_{g}r{\theta }^{2}cos\theta \right)rsin\theta \\ -d_g\left({\displaystyle \frac{m_t}{m_g}}\right(\dot{x_t}rcos\theta +r^2\dot{\theta })+r^2\dot{\theta }{\mathit{sin}}^2\theta )\end{array}$$

(11)

$$x_g=x_t+rsin\theta$$

(12)

The simulator implemented based on the dynamic motion equation is shown in Figure 8. Dynamics-based equations within the simulator were used to implement the movements of the main driving parts. The main driving parts include hoist, grab, and trolley. The position data is input to each driving part, and the GTSU 3D model is driven according to the dynamics formula. The simulator input values are the hoisting position for the hoist, open/close status data for the grab, and traversing position for the trolley.

4.3. Validation Results of the AI Prediction Model Using the GTSU Dynamic Simulator

To verify whether the MLP-based AI model prediction results can be applied to actual equipment, a comparison was performed between the GTSU dynamic-based simulation results and the AI model prediction results. The simulation input is the position of each driving part, and the output is the speed of each driving part. The comparison results of the position data and the speed data are shown in Figure 9. The output of the AI prediction model is the speed data. Therefore, the position data of each driving part were calculated according to the correlation between position and speed and used as input data to verify the AI prediction results through the dynamic-based simulator. The simulation was performed for 80 s.

As a result of operating each driving part with the predicted speed as input in the simulation, it was confirmed that the position of the driving part over time was somewhat different from the actual position. This was determined to be a problem caused by the AI-predicted speed following the trend of the actual speed but not accurately tracking the change in speed. However, if the AI model is modified so that the predicted speed follows the real speed with high accuracy, the overfitting problem occurs [28]. In other words, the accuracy may decrease further when new data is input. To reduce the error between real and predicted data, the task of calculating the difference in movement amount between real and predicted data and compensating was added. This is an additional step to apply reliable prediction results to actual equipment and corresponds to post-processing [29,30,31]. In the case of the grab, the open/close status data should be checked instead of the position data since the operation is open/close. This can be confirmed by subtracting the grab position encoder value from the hoist position encoder value. A value of approximately 140 or more indicates an open state, and 0 indicates a closed state. Instead of compensating for speed, the grab data is post-processed based on the value of 0.8, so it opens when it exceeds 0.8 and closes when it falls below 0.8.

The dynamics-based simulation results of the post-processed AI prediction speed are shown in Figure 10. The start and end points, as well as the position when grabbing the bulk, appeared to closely match the actual values.

To determine that the operation simulation results of the prediction model are valid, it is necessary to verify the simulation results numerically. The mean relative error (MRE) and the R² score were used as indicators for evaluation. The MRE represents the average of the relative error across all data points, so it is useful for determining overall accuracy. The R² score indicates how well the predicted value explains the variability of the actual value, so it is useful when determining whether the prediction model correctly represents the tendency of the actual value.

Table 8. Comparison results of the before and after post-processing.

Driving Part	Position Error before Post-Processing		Position Error after Post-Processing
Hoist	MRE	R2 score	MRE	R2 score
	1.47	0.97	0.44	0.99
Grab	MRE	R2 score	MRE	R2 score
	2.56	0.98	0.50	0.92
Trolley	MRE	R2 score	MRE	R2 score
	0.07	0.98	0.03	0.98

shows the calculation values of an evaluation index for the simulation results before and after post-processing. As a result of post-processing, it was confirmed that the MRE was below 0.5 for all driving parts. In the case of R² scores, the hoist and the trolley showed significantly high values, but the grab showed lower post-processing values. This is because the grab predicted data without post-processing and followed the trends of actual data better. On the other hand, the MRE value was lower in the results after post-processing. As a result, it was confirmed that when simulated after post-processing, it matched the operation sequence of the actual data well.

It is judged that the GTSU automation control can be performed if the results of this study and safety-related functions, such as limiting the range of unloading, are implemented together afterward.

5. Conclusions

This paper developed and validated an AI model to predict GTSU operation data. When designing an AI prediction model for the automatic driving GTSU, the operator driving patterns were trained using the GTSU manual operation data. Two networks, including the MLP and the LSTM were designed, and the suitable network was finally selected by comparing the prediction results. The prediction results of the AI model were additionally verified using a dynamics-based simulator. The AI model was designed to predict the speed of each driving part by inputting the moving position and unloading position. The main driving parts of the GTSU contain a hoist, a grab, and a trolley. The GTSU operation data was presented in the form of repeating one-cycle operations for each cargo hold of a bulk carrier. Since this study is a one-cycle automatic operation for one cargo hold, it is necessary to select normal one-cycle data for AI learning. Additionally, data scaling was performed using MinMaxScaler for AI learning. Considering that the GTSU’s operation data is time series data, MLP and LSTM networks were designed as AI learning techniques, and their prediction performances were compared. To verify the prediction performance of the final selected AI model, the GTSU dynamic simulator was used to evaluate whether the AI prediction results could be applied to the automated operation of actual equipment. The simulation results showed that AI prediction data can be applied to actual equipment. The MSE was used as an evaluation index to compare the prediction performance of MLP and LSTM networks. Analysis of MSEs in both networks showed that the training MSE had a lower value than the validation MSE, indicating that the AI model was not overfitting. As a result of comparing the MSE of the two networks, it was analyzed that there was little difference in prediction performance with a slight difference. Therefore, the MLP model, which has a simpler structure than the LSTM model, was adopted to predict the GTSU operation data. The equation-of-motion-based GTSU operation simulation was performed to verify the AI prediction model. The MRE and R² score were used as indicators to compare the simulation results and the prediction results. In the simulation, when driving each driving part with the predicted speed as input, it was confirmed that the position of the driving part over time was somewhat different from the actual position. To reduce the error between the real and the predicted data, the post-processing of calculating the difference in movement amount and compensating was performed. After post-processing, the MRE was up to 0.50, and the value of the R² score exceeded 0.92, indicating that the model is effective in predicting operation data. The analysis demonstrated the effectiveness of the AI prediction model.

Since the AI model predicts output by training actual data, there is no significant improvement when comparing the one-cycle time of actual operation data and predicted operation data. However, the daily unloading stop time for manual operation is about 3 h, including operator shift time and rest time. Comparison can be made assuming that there is no stop time for automatic operation; the unloading rate is expected to improve from 39.4% for manual operation to 45.0% for automatic operation of the GTSU, which is 2000 T/H class with an unloading capacity of 900 tons per hour. As the operator’s operation control position is moved from the GTSU cabin to the ground remote operation room, musculoskeletal disorders are prevented, and the risk of accidents related to equipment access is reduced. Additionally, it can provide automation functions that serve as the basis for an automation system that allows a single operator to control multiple unloaders. This study predicted the GTSU operation data using an AI and verified the predicted data using a dynamic simulator, but it is difficult to automate GTSU fully using only an operation data prediction model. To perform fully automated GTSU control, target unloading points must be extracted and connected to the operation data prediction model. Target unloading points can be extracted by acquiring and analyzing bulk scanning data through LiDAR and Radar sensors. Therefore, we plan to analyze sensor data to extract target unloading points and implement a fully automated GTSU system by connecting it with an operation data prediction model in further study.