Classification of Ship Type from Combination of HMM–DNN–CNN Models Based on Ship Trajectory Features

Abstract

This study proposes an enhanced ship-type classification model that employs a sequential processing methodology integrating hidden Markov model (HMM), deep neural network (DNN), and convolutional neural network (CNN) techniques. Four different ship types—fishing boat, passenger, container, and other ship—were classified using multiple ship trajectory features extracted from the automatic identification system (AIS) and small fishing vessel tracking system. For model optimization, both ship datasets were transformed into various formats corresponding to multiple models, incorporating data enhancement and augmentation approaches. Speed over ground, course over ground, rate of turn, rate of turn in speed, berth distance, latitude/longitude, and heading were used as input parameters. The HMM–DNN–CNN combination was obtained as the optimal model (average F-1 score: 97.54%), achieving individual classification performances of 99.03%, 97.46%, and 95.83% for fishing boats, passenger ships, and container ships, respectively. The proposed approach outperformed previous approaches in prediction accuracy, with further improvements anticipated when implemented on a large-scale real-time data collection system.

1. Introduction

Transportation of freight through marine passage constitutes approximately 90% of international freight transport [1], and every day, more than 50,000 ships navigate all over the ocean [2]. Furthermore, the volume of ship traffic is continuously increasing, making maritime traffic safety an important issue worldwide. Especially in coastal areas, strong safety management is required during navigation, as they possess narrow channels and complicated ship routes for berthing and unberthing ships. Additionally, to meet the requirement of the increasing shipment volume, navigation ship support facilities such as light buoys in sea areas have been developed, which increase the high-risk potential of marine accidents. At the Busan port, in South Korea, the world’s top-ten port in trade shipment volume, 14,042 maritime accidents have happened, and the port is showing an increasing rate of 0.8 accidents per year [3]. Furthermore, the damages from marine accidents, such as oil pollution, create a critical condition in the marine environment and ecosystem that takes several decades to recover. To prevent marine accidents, the International Maritime Organization (IMO) declared that navigating vessels over 300 tonnes must be equipped with the automatic identification system (AIS), one of the vessel monitoring systems (VMS), for enhanced marine surveillance [3]. Moreover, a small fishing vessel tracking system (V-Pass) has been developed by the coast guard of Korea for monitoring small vessels under 300 tonnes, which are excluded from the target of the AIS [3,4].

In the case of surveillance, the AIS is considered an efficient tool that provides the current location of ships as well as real-time dynamic information. Besides, static information on ships, such as length, width, and ship type (fishing, passenger, container, etc.), can be obtained from the AIS, which becomes helpful in predicting the behavior of ships and avoiding accidents. Importantly, the ship behavior pattern in the coastal sea region depends on the type of ship, which can be differentiated by the shipment information. However, 25–60% of static data is frequently missed from the AIS [5] due to the failure of equipment [6,7] and the unfamiliarity of seafarers with handling the AIS equipment [8,9].

To overcome this chronic problem of missing data, several studies related to the interpretation of AIS data have been conducted in the past. Ref. [5] utilized the static data from AIS, which includes nationality and ship information, to classify the ship type by employing a simple method in a specific sea area. Afterward, mathematical and probabilistic statistical methods have been widely used to extract the ship type feature from various parameters of AIS data to elevate the prediction accuracy [1]. In particular, a fishing activity prediction algorithm based on the hidden Markov model (HMM) has been used to classify ships, and it showed an effective result in classifying fishing boats and other types of ships [10]. Trajectories of large ships berthing and departing from ports have been classified, and the prediction of similar ship types has been conducted by employing hierarchical clustering techniques, including DBSCAN, that reflect the characteristics of large ships [11,12,13]. Furthermore, Ref. [3] used AIS big data and proposed a ship-type classification method based on the random forest machine learning technique. Moreover, several studies have been conducted to fill the gap in missing AIS data by combining different types of data to enhance the interpretation of ship movements [14]. Ref. [15] proposed the improvement method for obtaining ship position from satellite data using image processing. Through fusing CNN and the k-nearest neighbor (KNN) techniques, a ship-type classification study with different types of satellite-based ship detection images was proposed [16], and Ref. [17] attempted to predict the ship trajectories using oceanic data. Even with improved methodology and data fusion, effectively extracting the distinctive nature of trajectory features to notably increase classification accuracy remains limited.

Currently, deep learning models have been applied to AIS data with the advancement of AI-based machine learning and deep learning technologies, which can replace traditional statistical methods. Among them, recurrent neural networks (RNNs) are effective models for predicting time series data, such as AIS data, with a deep learning model used to predict the destination of ships [18]. Besides, ship trajectory prediction has been performed by convolutional neural networks [18,19,20,21] and autoencoders [22,23], where the image-based deep learning models that were used could be spatially transformed with AIS data and displayed improved performance in comparison with the RNN models. Moreover, different deep-learning models were employed to predict the ship trajectory and classify the ship type on the basis of pre-processed AIS data [24,25,26,27]. In spite of the improvement of AI techniques, the classification accuracy of multiple ship types has surprisingly not increased, primarily due to the diverse nature of ship trajectory features influenced not only by the ship type but also by various parameters like ship speed and course within each ship type. Therefore, the development of an application model tailored to selected parameters is essential.

Thus, in this study, we propose a method to classify the ship type based on different ship trajectory features and utilized multiple deep-learning and statistical-based techniques to achieve higher accuracy. In the case of AIS/V-Pass, a dataset suitable for each model was created using data generation, enhancement, and augmentation techniques, and this dataset was divided into training and testing sets. A model was created with the training dataset, and the results were verified with the test dataset. Each ship type classification model was applied sequentially, and a total of four types of ships were classified, which include fishing boat, passenger, container, and other ship.

Our contributions encompassed the following:

• Ship-type classification methodology was proposed by combining a DL model and a thresholding method, which incorporates dataset enhancement with filtering based on the trajectory features of each ship type, resulting in significantly higher classification accuracy compared to commonly used simple DL techniques.
• The optimal ship classification model, combining multiple DL models (HMM, DNN, and CNN), was utilized.

The remaining chapters of this paper are organized as follows: Section 2 introduces the spatial and temporal scope of this study, outlines the specificities of the training and test ship dataset, and details the workflow of the HMM, DNN, and CNN-based ship classification techniques, utilizing trajectory features for various ship types. Section 3 describes the analysis of trajectory features for each ship type, while Section 4 depicts the classification results using the test dataset. Finally, Section 5 discusses the findings, and Section 6 concludes the paper.

2. Materials and Methods

2.1. Study Area

The sea region in the vicinity of Busan was selected as the study area, and to classify the ship type, AIS and V-Pass data were used. Data acquired by AIS indicates an AIS-ship and data from V-Pass represents a small fishing vessel. Both datasets covering the sea area (128.5–130.0°E, 34.5–35.5°N) from 6 to 10 February 2021 were collected, and 4207 ships were observed in that period (Figure 1). Korea Institute of Ocean Science and Technology (KIOST) has operated an AIS and V-Pass collection system in Busan (Figure 1), which received real-time encrypted packet information and automatically stored it on a database server computer [28]. Both AIS and V-Pass data are classified into dynamic and static data, where the dynamic data has a sequence of time-series data comprising longitude, latitude, speed over ground (SOG), and course over ground (COG) in which the interval period of receiving information ranges from 1 s to 2 min. Static data, which includes ship type, ship length, width, etc., is only provided by the AIS, and based on this information, multiple types of ships can be classified. Trajectories of ship types are depicted in Figure 1, where AIS-ships covered a wider area in the sea while fishing boats commonly stayed in the sea near the coast.

2.2. Explanation of the Ship Trajectory Data

To perform classification using a model, ship trajectory data need to be constructed by considering the ship information data, which are made up of total dynamic data, such as ship position and speed, and ship-type data, as a label. Though AIS static data provided information on the ship type, the quantity of information was limited. Thus, in this study, the ship type as a label was not only made up of AIS static data but also referenced by Marine Traffic information due to the lack of categorization of AIS-ship types [4].

The ship trajectory data were prepared as training and test data for the classification of four ship types: fishing boat, passenger, container, and other ship. The training dataset was created using information collected from 6 February to 9 February 2021, where a total of 144 fishing boats, 3 passenger ships, 86 container ships, and 203 other ships were observed. The test dataset consisted of 94, 1, 23, and 82 ships for each respective ship type on 10 February 2021. The dataset was split into 80% for training for 4 days (6–9 February 2021) and 20% for testing for 1 day (10 February 2021) for fishing boats and container ships. However, passenger ship data were not enough to divide into training and test data due to the characteristics of the Busan port, and therefore, the data-obtaining period was extended until 28 February 2021. Therefore, the passenger ship dataset was split into 18 days (6–23 February 2021) and 5 days (24–28 February 2021) as training (4 ships) and test datasets (4 ships), respectively.

Additionally, to prevent a severe bias in the deep learning model due to the insufficient data on some types of ships, we augmented the fishing boat, passenger, and container ship datasets to 207, 200, and 204 ships, respectively. Similarly, test data were fitted to 20% of the training dataset by augmentation of passenger and container ships, while the number of fishing boats and other types of ships was reduced by the random selective method (

Table 1. Description of the training and test datasets for the ship type class.

Ship Type	No. of Training Ship Dataset	No. of Test Ship Dataset
Fishing boat	207	52
Passenger	200	50
Container	204	51
Other ship type	203	50
Total dataset	815	203

).

Figure 2 depicts the trajectory of each ship type. The passenger ships showed a common trajectory pattern due to the fixed navigation route from the Busan port to Japan, while the trajectory of the container ships was distributed over the sea, and they seemed to be navigating from various departures to the one-point area of the Busan port or berthing at the Busan port. In the case of fishing boats, major activities were seen near the coastal sea area.

2.3. Methodology

Figure 3 illustrates the overall flowchart for classifying four types of ships using HMM–DNN–CNN combination models with optimized parameters. Initially, the dataset was converted into the most suitable form for each model through two steps: dataset generation and enhancement. After categorizing the dataset as either training or validation, augmentation was performed to compensate for the insufficient data. In the next step, parameters for each model were selected. In the case of fishing boats, SOG and rate of turn (ROT) were set as the input parameters of the HMM model (blue; Figure 2a) to better highlight the movement characteristics of the boat. For passenger ships, berth distance, rate of turn in speed (ROTS), and heading were used as the input parameters of the DNN model (green; Figure 2b), whereas for container ships, longitude/latitude, SOG, and COG were utilized as the input parameters of the CNN model (red; Figure 2c). The models were applied sequentially: first, the HMM model classified fishing boats and other ships, and next, the DNN model differentiated passenger ships and other ships. Finally, the CNN model reclassified the other ships into container ships and a final other ship type.

2.3.1. Dataset Generation

First, data splitting was conducted to remove the noise from the AIS and V-Pass data. Noise can arise from factors such as intentional or unintentional power-off periods of the AIS/V-Pass devices, motionless tracks from ships at berth or anchor, and testing of AIS devices on land. It can be assumed that vessel characteristics could not be reliably extracted when data were missing for over 30 min or when there was no spatial movement. Consequently, this led to significant errors in ship type classification, and therefore, data splitting was required. Thus, the algorithm was designed to create a dataset from data collected before the specified conditions were met. Once these conditions occurred, a new dataset was initiated. This approach resulted in the generation of distinct datasets, which were then merged into a final dataset. Secondly, interpolation of data was conducted, as incomplete information remained within the data because of an irregular transmission cycle during bad weather conditions. Therefore, the dynamic information, which includes longitude, latitude, SOG, COG, and heading, was interpolated to a 1 min interval. In spite of the data interpolation, some noise data still existed, which is known as the common issue of raw data. To overcome this, we defined the outlier values and eliminated them by using Gaussian filtering, and at least 60 or more data points were configured as a dataset. The main feature of the dataset created by this method was that, despite the period of the input data, it provided multiple datasets with unique characteristics of tracks after eliminating the noise, which enhanced the performance of the model.

2.3.2. Dataset Enhancement

The previously described data generation method was effective for a single model, but significant performance deviations were observed when applied to multiple models. Therefore, to improve ship type classification performance, adjustments were made to the temporal and spatial coverage of the dataset for each model. For the HMM model, the dataset length was adjusted so that 5–95%, 10–90%, and 15–85% of the data were used, resulting in a 10%, 20%, and 30% reduction in noise ratio, respectively. Among these three different lengths of the dataset, the one with the highest probability was selected to evaluate whether the vessel was a fishing boat. Secondly, a region of interest (ROI) filtering technique was employed to specify an area within the sea, where the DNN model was applied for classifying the passenger ship. Thirdly, a polygon was created around piers typically used by container ships and other ships. If a ship was located within the polygon boundary, it was classified as either a container ship or another type of ship accordingly. After applying the model, the dataset with the highest probability among those generated from each trajectory was selected and used for ship-type classification. While previous studies have developed classification models using approaches such as multi-feature ensemble learning [29] and multi-view feature fusion networks [30], our research emphasizes generating distinct models based on multiple datasets. Moreover, KIOST has an operational AIS/V-Pass collection system that acquires data from antennas and converts it into a real-time database on a server, enabling the model to be applied to ship data at any given time.

2.3.3. Dataset Augmentation

An adequate quantity of data is necessary for deep learning models. However, the data acquired for each ship type from AIS/V-Pass was not dispersed equally, and depending on the characteristics of the sea area, some ship types had relatively inadequate data quantities. In order to overcome this problem, data augmentation for specific ship types with limited data was performed. Data augmentation entails generating similar information from the model’s five input data types: longitude, latitude, COG, SOG, and heading. To minimize the introduction of significant bias in the learning process, a minimal constant value of 0.001 multiplied by the number of iterations (n) was added to points based on the existing raw trajectory data. This process was repeated until the desired number of datasets was obtained.

2.3.4. Ship-Type Classification Techniques

HMM is a probability statistics-based technique that uses time-series data to estimate state emission and transition probabilities prior to computing hidden state probabilities. In this study, fishing boat was the first ship type considered for classification utilizing the hierarchically generated first and second HMMs, as illustrated in Figure 4. In the first step, time-series data of SOG and ROT acquired from AIS/V-Pass were used from the initial time (t) to the final time (t_f) as input for the HMM.

Both parameters were effectively processed by the respective HMMs, and the probability of being in either a fishing or non-fishing state was calculated for each position. Figure 4a illustrates the detailed construction of the first HMM for deriving P_fishing (SOG). Firstly, emission probabilities from the training dataset that had already been trained and processed SOG data were used to calculate the P_fishing (SOG) probability (hidden state). The probability was also influenced by the transition probability, which took into consideration the relationship between t and t + 1, and the fishing or non-fishing state was ultimately determined by the statistical combination method with both results of hidden state probabilities. Particularly, the SOG parameter is recognized as the HMM model’s preferred option for predicting the fishing state [11,31].

In this study, the ROT parameter was additionally included in the input data for the HMM in order to configure the new HMM structure and reinforce the prior technique. By reclassifying a fishing state dataset that was incorrectly classified using the first HMM’s acquired results, this second HMM could increase the classification probability.

To operate the HMM model, labeled data reflecting fishing activity characteristics were essential for calculating the pre-learned coefficient. Therefore, points related to fishing boat activities were labeled based on the ship’s speed and course from trajectories recorded over a four-day period, from 6 to 9 February 2021. These labeled data were used to derive emission probabilities based on SOG, as well as transition probabilities that represent the likelihood of shifting between fishing and non-fishing states.

In particular, emission probabilities were derived using ranked values rather than raw parameter data. These ranks represent characteristic ranges, with fishing activities occurring primarily at speeds of 3 knots or lower, while non-fishing activities occurred mainly at speeds above 7 knots. The HMM model, reflecting the SOG tendencies of fishing boats, was based on a validated five-level grading system from prior research [31]. In the first HMM, based on SOG, a vessel was classified as a fishing boat when the proportion of tracks involving fishing operations exceeded 71% of the total tracks.

Moreover, analysis of the ROT revealed high frequencies below 18° for both fishing and non-fishing activities. However, ROT values exceeding 48° were absent during non-fishing activities. Accordingly, ROT was classified into multiple ranges, which include both narrow (0° ≤ 18° and 18° < 48°) and broad (48° < 133°) ranges. Thus, the vessel was reclassified as a fishing boat in the second HMM based on ROT when the proportion of tracks associated with fishing activities reached 59%. These threshold values were determined through trial-and-error analysis.

In this study, the navigation of passenger ships along the fixed Busan–Japan route was analyzed in comparison to other ship types. The heading was chosen as the most suitable parameter for this constrained trajectory. Additionally, the distance from the ship to the berth location was estimated and used as a parameter, since passenger ships could commonly only berth at a single pier in the Busan port. Finally, the ship’s ROTS was also incorporated as a parameter to account for the passenger ship’s superior maneuvering capabilities relative to other ship types. To further improve model performance, spatial filtering (ROI filtering) was applied to each parameter. In particular, ROI1 was designated around the entrance to the Busan port for the ROTS parameter, while ROI2 was designated in the open sea for the heading parameter. As demonstrated in Figure 5, specific conditions were applied to each parameter instead of using all trajectory values as learning inputs to better emphasize their characteristics. The berth distance threshold was established at 550 m, corresponding to the measured length of the pier. In ROI1, passenger ships displayed a significant reduction in SOG near the breakwater compared to other ship types. Consequently, a ROTS threshold of 0.1 to 0.2 knots per minute was determined as optimal for identifying passenger ships. Furthermore, distinct heading angles of 120° ≤ 150° or 300° ≤ 330°, which are typical for the Busan port–Japan route, were used to refine the classification.

The DNN model, serving as the second model in the multi-type classification technique, was built using the three parameters selected for passenger ship classification. This model, characterized by forward connections between nodes, follows the fundamental structure of deep learning techniques. A neural network with multiple hidden layers is referred to as a DNN [28]. Due to its simple architecture and ease of training, the DNN is frequently used by researchers and practitioners across various engineering fields. In this study, a DNN model with two hidden layers was used. Each layer’s unit usage was set to 128 and 64, respectively. Rectified linear unit (ReLU) was used as the activation function. This network structure adheres to a simple layout without adding unnecessary complexity. It was determined that the model would operate satisfactorily with this fundamental network configuration despite the small quantity of passenger ship data available (a total of 815 ship data), which cannot be categorized as big data.

Figure 6 depicts the process for classifying container ships using pier masking and the CNN model. Using trajectory point data from Lon/Lat, it was possible to anticipate whether a container ship would enter the container pier (CP) or not. Notably, ships arriving into and departing from the Busan port include several ship types, along with container ships. However, due to the constrained routes, these ships possessed similar trajectory patterns, especially those navigating through the Busan port. This is why categorizing container ships was challenging using the CNN model’s image-based classification technique. Subsequently, within the port area, distinct spatial zones for CP and NCP (non-container pier), which include ship types other than container ships, were designated. The threshold value for pier masking involved calculating the difference between two factors: the number of overlap points associated with the CP polygon and trajectory points (P_pm(CP)) and the number of overlap points involving the NCP polygon and trajectory points (P_pm(NCP)). If the result was positive, the ship was classified as a container ship; if not, it was classified as other ship type. The process continued up to the CNN model phase, when there were no overlapped points from either the CP or NCP (resulting in a value of 0).

The CNN model includes individual layers for image input, output, flattening, fully connected, and output scoring, as well as iterative layers for convolution, activation, and max-pooling processes [16]. It receives input data in the form of a two-dimensional array, generally referred to as an image, typically consisting of three RGB channels. The input image is represented as col × row × channel (density map). However, the format of trajectory data was point-based, so a conversion process was necessary to render them into 3D data with multiple channels. Consequently, the longitude range of 128.5 to 130.0 degrees and the latitude range of 34.5 to 35.5 degrees were designated to define the outline of the resulting 3D image’s col × row dimensions. To create data for each channel, we assigned SOG to the R band, COG to the G band, and the number of trajectories to the B band. Additionally, we applied particular criteria to capture each parameter’s characteristics in a way that’s identical to the passenger ship classification approach. For Lon/Lat’s trajectory count, we filtered grid cells with over 50% average counts of trajectory to reduce noise. Besides, a threshold for SOG values (8 ≤ 16 kt) was established, considering the relatively faster speed of container ships. Additionally, the COG range (120° ≤ 140° or 320° ≤ 330°) was constrained to highlight the characteristics of container ships primarily navigating to and from the Busan port.

In the CNN networks, a convolution layer was established with dimensions of 20 × 30 × 32, utilizing a set of 32 weight filters in a 3 × 3 layout. The ReLU activation function was used to address the problem of disappearing gradients in deeper layers, where gradients approach near-zero values. This process leads to the generation of a conclusive feature map with three dimensions. Subsequently, after the initial layer, a max-pooling operation was implemented to reduce the size of the activation map and produce an image with condensed features. This was accomplished using a kernel size of 1 × 2 × 2 × 1, resulting in a new layer sized 10 × 15 × 32. These steps were repeated for each layer’s depth. Eventually, a flattening layer was created, which was followed by a fully linked layer that used the softmax function to calculate the output score. The probability that the input would be classified as a container ship was indicated by this output score.

In this study, parameters and ship trajectory patterns associated with each model were selected based on prior research (

Table 2. Parameters and setting of each model used in this research.

Model	Input	Parameters and Setting	Output
HMM	SOG, ROT	Structure: two-level hierarchical model Emission probability: five grading of SOG, ROT Transition probability: two grading of fishing state	Fishing boat
DNN	Berth distance, ROTS, heading	Network - 1st hidden layer: 128 - 2nd hidden layer: 64 Activation model: ReLU, Softmax Learning rate: 0.001 Optimizer: Adam Batch size: 10 Training epoch: 20 Cost function: cross entropy error	Passenger ship
CNN	Lon, Lat, SOG, COG	Network - 1st conv. layer: 32 filters with 3 × 3 size - 2nd conv. layer: 64 filters with 3 × 3 size - 1st hidden layer: 128 Same as DNN parameters and setting	Container ship

). SOG and ROT are two of the key features that characterize the wake behavior of fishing vessels and have been used in conjunction with the HMM model to predict fishing activities [32,33]. Additionally, extensive trial-and-error experiments were conducted with various input parameters and vessel types to optimize DNN and CNN models for the study area [34].

2.3.5. Model Evaluation

The developed model was used to classify fishing boats, passenger ships, and container ships independently and sequentially. Accuracy and F-1 scores, which are frequently used to evaluate classification results in deep learning and machine learning models [11,16,31], were employed in this study. The F-1 score incorporates four factors to evaluate classification performance: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). By incorporating ground truth with projected values, it provides a numerical representation of performance and offers an objective assessment.

3. Parameters-Based Trajectory Analysis

Figure 7a depicts the trajectory of a fishing boat on 10 February 2021 in the test dataset. The trajectory described conventional fishing behavior: when the vessel moved quickly (blue circle), it was regarded as a vessel engaged with non-fishing activities; when it drifted slowly with minimal movement (red circle), it was considered a vessel engaged with fishing activities. Besides, the vessel’s direction altered more at lower speeds, suggesting significant shifts in COG.

The ratios of fishing and non-fishing activities were compared after analyzing the entire training dataset using the SOG and ROT parameters. It was observed that fishing activities were carried out at speeds below 5 knots, whereas non-fishing activities were prevalent at speeds exceeding 5 knots (Figure 7(b-1)). In terms of ROT, a more widespread distribution with a pronounced kurtosis around 0 was exhibited by non-fishing activities, while fishing activities displayed a broader dispersion (Figure 7(b-2)).

Data from observed passenger ships between 6 and 9 February 2021 were utilized to comprehend the passenger ship trajectory features. Moreover, they were employed to depict movement near berthing locations and show trends in SOG and COG for different ROI areas. Figure 8 depicts the paths of a typical passenger ship, covering the ROI areas. When they reach the Busan port within ROI1, there are likely to be noticeable SOG changes due to their readiness to pass the narrow breakwater. Furthermore, they maintain a steady heading while entering Busan from Japan, encompassing ROI2. In the entire training dataset, when comparing passenger ships to other ships regarding berth distances, passenger ships exhibited several center points over 0 in broadly expanded distributions caused by specific small kinds of datasets, while other ships tend to display probabilities converging towards 0 (Figure 8b(b-1)). Particularly, the ROTS probability trend shows an obvious distinction between passenger ships and other ships, which parameters could be efficient to model performance. When referring to the third parameter, heading probability, there is a variance between the two types of vessels: other vessels exhibit higher densities when the probability is under 0.05, whereas passenger ships display higher probabilities when the probability is over 0.05 (Figure 8b(b-3)).

After classifying fishing boats and passenger ships, the pier masking method and CNN model were used to classify container ships and other ships using the remaining ship datasets as input. As the initial step of pier masking, distinct piers for container ships and other vessels in the Busan port were identified. Cargo-specific pier data from the Busan Port Authority (BPA) was referred to, and the shape of shipments at different pier locations was visually verified using Google Earth imagery. Then, for the locations of the indicated piers, polygons for CP (red area) and NCP (blue region) were created, as shown in Figure 9. The green symbol represents the trajectory of a ship entering the Busan port, indicating its entry into the CP area. Consequently, the CP probability for the sample ship was calculated with a value greater than zero (the green circle in the right map), while the NCP probability was assigned a value of 0. Therefore, since the CP probability minus the NCP probability yielded a positive value, the sample ship was classified as a container ship.

The remaining dataset was analyzed with the CNN model as the second step after container ships were identified using the pier masking method. Figure 10 illustrates trajectory counts’ density using the Lon/Lat parameter, depicting the grid distribution based on container ship locations. It was found that the container ships were closely clustered within a defined boundary near the Busan port and appeared to predominantly move in specific directions (Figure 10a, shown by arrows). The trajectories of container ships, particularly their major movement direction, become apparent in the results after using threshold filtering for the number of trajectory and SOG parameters, which was noticeable in Figure 10(b-1,b-3). However, despite the filtering, other vessels do not show intensified trajectories for specific movement directions except for the Busan–Japan route for passenger ships (Figure 10(b-2,b-4)). The COG parameter map revealed more frequent noise trajectories in other vessels compared to passenger ships, as shown in Figure 10(b-5,b-6).

4. Results

In this study, each model was evaluated individually, with variations in the datasets used in the sequential approach. These evaluations included visualizing ground truth versus classification outcomes (Figure 11) and calculating a confusion matrix for each model (Figure 12).

Figure 11 illustrates the independent application of the HMM, DNN, and CNN models, depicting the ground truth and classification results. The HMM-based fishing boat classification is represented by blue circles, the DNN-based passenger ship classification by green circles, and the CNN-based container ship classification by red circles. The test dataset included trajectory information for 203 ships. After applying the HMM, DNN, and CNN models, 54 ships were classified as fishing boats, 55 as passenger ships, and 51 as container ships.

The confusion matrix was calculated for each model, consisting of four parameters: TP, TN, FP, and FN. In Figure 12, (1,1) represents TP, (1,2) corresponds to TN, (2,1) indicates FP, and (2,2) denotes FN. Figure 12a, and 12b show two and five instances of TP, respectively, where other vessels were incorrectly classified as fishing boats or passenger ships. These errors were deliberately minimized during model development by addressing non-detection in the first and second stages. In Figure 12c, four instances of TN occurred, where container ships were inaccurately categorized as other types of vessels. The F-1 score for each ship-type class was calculated as 99.33%, 97.46%, and 95.83% for fishing boat, passenger ship, and container ship, respectively.

Furthermore, since the performance of the HMM, DNN, and CNN models can vary depending on the order in which they are applied, we computed and compared evaluation metrics across six possible combinations.

Table 3. Accuracy performance results based on the combination of HMM, DNN, and CNN models.

Combination	Model	Accuracy (%)	F-1 Score (%)	Average (F-1)
HDC	HMM	99.01	99.33	97.54
	DNN	96.69	97.46
	CNN	96.04	95.83
CDH	CNN	93.1	95.45	97.31
	DNN	98.03	98.51
	HMM	98.04	97.96
DHC	DNN	95.57	96.97	97.27
	HMM	98.69	99
	CNN	96.04	95.83
CHD	CNN	93.1	95.45	97.12
	HMM	98.68	98.99
	DNN	98.68	98.99
DCH	DNN	95.57	96.97	97.03
	CNN	94.77	96.15
	HMM	98.04	97.96
HCD	HMM	99.01	99.33	96.75
	CNN	92.05	94
	DNN	97	96.91

presents the accuracy results for each model using the test dataset. All models consistently displayed strong classification performance, exceeding 90% accuracy, regardless of their position. The HMM achieved a remarkable accuracy of over 99% in the first position and consistently outperformed the DNN and CNN models in the second and third positions as well. The DNN also exhibited strong classification accuracy, ranging from 95% to 98%, depending on the model arrangement. In contrast, CNN’s performance was slightly lower, maintaining approximately 95% accuracy across all positions. Comparing accuracy across the six combinations, the HMM–DNN–CNN combination (Figure 3) yielded the highest overall performance.

5. Discussion

This study proposed a method for classifying fishing boats, passenger ships, container ships, and other vessel types based on trajectory data using the combination of three DL models to achieve optimal classification performance. Data splitting and interpolation were performed to AIS/V-Pass data to remove noise and address irregular transmissions, with outliers filtered to enhance model performance. Initially effective for single models, performance deviations arose with multiple models, necessitating adjustments in temporal and spatial coverage. The data collection interval was initially set to 24 h, and datasets were tested based on this interval. However, if trajectory features for specific ship types could not be extracted, the data collection period was extended by an additional 24 h to improve feature extraction. As the input parameter, the HMM model used SOG and ROT, the DNN model used berth distance, ROTS, and heading, and the CNN model utilized longitude/latitude, SOG, and COG. Datasets with various lengths were used in the HMM model for fishing boat classification, the ROI filtering approach was performed in the DNN model for passenger ships, and polygon-based classification was used in the CNN model for container ships. The dataset with the highest probability was selected for classification, supported by the KIOST’s real-time data system.

The classification performance of the HMM model was evaluated for fishing boats and other ships using the entire dataset. The DNN model assessed the classification of passenger ships and other vessels, excluding those classified as fishing boats by the HMM model. The CNN model then evaluated the classification of container ships and other vessels, excluding passenger ships and fishing boats previously categorized by the HMM and DNN models. All models achieved high classification accuracy, exceeding 90% across different sequences. The HMM model achieved over 99% accuracy in its first position and consistently outperformed the DNN and CNN models in subsequent positions. The DNN model reached an accuracy range of 95% to 98%, while the CNN model maintained approximately 95% accuracy throughout.

Although the classification performance was satisfactory, there were some exceptions. The HMM model incorrectly classified vessels moving south of the port as fishing boats, and the DNN model misclassified ships traveling from the Busan port to the Busan south port as passenger ships. In container ship classification, some vessels entering the Busan port were misclassified as other ship types due to insufficient trajectory data within the port, which complicated evaluation with the pier masking approach. Additionally, criteria such as berth distance and ROI filtering may need adjustment based on future port and facility developments. The classification model, trained and tested on a five-day dataset, demonstrated high performance in environments similar to the training data but showed limited generalizability for long-term application. Future work will focus on supplementing the dataset and improving model performance by analyzing the characteristics of misclassified vessels.

6. Conclusions

This study focused on generating and testing (pilot-applying) a ship-type classification model utilizing AIS and V-Pass data. The model was designed by analyzing the trajectories of fishing boats, passenger ships, and container ships, each with distinct movement patterns. The objective was to design models tailored to these patterns to improve classification accuracy. A systematic approach was employed to generate datasets specific to each model. The test dataset included three sequential models: the HMM for fishing boats, the DNN for passenger ships, and the CNN for container ships, achieving a classification accuracy of 97.54%. Future work will involve continuous testing using real-time data to address unforeseen issues and further improve accuracy through refined methodology.