Semantic Modelling of Ship Behavior in Harbor Based on Ontology and Dynamic Bayesian Network

Abstract

Recognizing ship behavior is important for maritime situation awareness and intelligent transportation management. Some scholars extracted ship behaviors from massive trajectory data by statistical analysis. However, the meaning of the behaviors, i.e., semantic meanings of behaviors and their relationships, are not explicit. Ship behaviors are affected by navigational area and traffic rules, so their meanings can be obtained only in specific maritime situations. The work establishes the semantic model of ship behavior (SMSB) to represent and reason the meaning of the behaviors. Firstly, a semantic network is built based on maritime traffic rules and good seamanship. The corresponding detection methods are then proposed to identify basic ship behaviors in various maritime scenes, including dock, anchorage, traffic lane, and general scenes. After that, dynamic Bayesian network (DBN) is used to reason potential ship behaviors. Finally, trajectory annotation and semantic query of the model are validated in the different scenes of harbor. The basic behaviors and potential behaviors in all typical scenes of any harbor can be obtained accurately and expressed conveniently using the proposed model. The model facilitates the ships behavior research, contributing to the semantic trajectory analysis.

1. Introduction

The maritime data from multi-sources has rich meaning in the big data era, especially the meaning of ship behaviors [1]. As the carrier of maritime transportation, ships are the decisive factors of maritime safety [2]. Its dynamic behaviors are difficult to be recognized in a complex situation, even with the improvement of storage, indexing, and querying of trajectory data [3]. Most of the existing studies focus on the data analysis in the ships behavior research [4,5], but there are some problems—the semantic meanings of behaviors and their relationships are not explicit; the data from different sources or dimensions cannot be connected; and the traffic rules are difficult to consider [6]. This has led to the development of semantic ship behaviors based on the semantic trajectory [7].

The original ship trajectories are difficult to interpret, query, or visually identify [8]. The main reason is that the raw data cannot express the semantic meanings of behaviors and their relationships. The semantic meanings are the refined, standardized concepts and relationships extracted from original trajectory data and context information [9]. There are different semantic dimensions in maritime situations, and semantic meanings in different dimensions can interact with each other to form ship behaviors [10]. For example, “speed equals to zero” in speed dimension (extracted from Automatic Identification System data) and “in a dock” in geographic dimension (extracted from chart information) can complement each other to obtain “berth” behavior. Furthermore, the maritime traffic rules can be expressed at the semantic level [11], so the semantic model can easily extract semantic behavior from the rules. According to the rules, the dynamic Bayesian network (DBN) can be used to reason high-level potential behaviors in all maritime scenes using a small amount of data.

The semantic concepts can be expressed by ontology, which has the following advantages. Firstly, the ontology can be reused [12] to eliminate the repetitive calculations of raw data. Secondly, the ontology and their relationships can be defined at different levels in the semantic network, which is good for the semantic richness of data and semantic reasoning of behavior. Finally, the ontology has the characteristics of easy sharing and expression, which makes it machine-processable and human-comprehensible [13].

As the semantic model is an effective approach to obtain the ship behaviors from trajectory data, the work establishes the semantic model of ship behavior (SMSB). The related work is presented in Section 2, and the semantic network is constructed in Section 3. Then, the ship states (the basic ship behavior) of all typical scenes contained in semantic network are recognized in Section 4. DBN is used for reasoning in Section 5 to obtain the potential behavior based on the states. Section 6 shows the application examples, which verifies the proposed model. Finally, we discuss future work in Section 7.

2. Literature Review

Research on ship behavior mainly uses data analysis rather than semantic analysis. Where some studies obtain the regional distribution of ship behavior based on statistical models [4], some focus on identifying abnormal behavior [14,15,16], and some obtain simple behaviors based on one type of data [5]. However, these methods face the problems as mentioned above.

In the transportation, some research assigns semantics to the traffic data, and proposes some models of the semantic trajectory [17,18]. Bogorny et al. [19] presented a model named CONSTAnT, which defines the concepts of semantic trajectory, including semantic sub trajectory, semantic point, geographical places, events, goals, environment, and behavior. They believed that the CONSTAnT can give users a comprehensive semantic view of raw trajectory. A semi-supervised algorithm, named RGRASP-SemTS, is proposed by Junior et al. [20] to segment trajectories based on semantics. The main advantage of this algorithm is that it can achieve high accuracy even when few labelled examples are available. Ilarri et al. [21] argued that exploiting semantic techniques in mobility data management can benefit to many domains, such as traffic management, urban dynamics analysis, and ambient assisted living. Ruback et al. [22] proposed a conceptual framework for the semantic enrichment of movement data using Linked Open Data as the unifying formalism and the source of contextual data. The framework converts the movement data to the semantic trajectory repository in Linked Open Data.

After that, some methods are proposed for analyzing semantic behavior based on semantic trajectory. Yuan et al. [23] provided an overall picture of semantic trajectory research, believing that behavior detection is one of the nine important tasks and cutting edge studies. deGraaff et al. [24] proposed a method named PIE to extract the points-of-interest and annotated them to the trajectory automatically. A framework that contains three methods for automatic annotation of semantic trajectories is proposed in the thesis of Nogueira [25]. It can handle the context information and find relevant information to describe the situation where the moving object is. Baglioni et al. [26] presented an approach to provide the interpretation of movement behavior. This approach provides a model for the conceptual representation and deductive reasoning of trajectory patterns obtained from mining raw trajectories.

These methods cannot be applied to maritime transportation because the behaviors of ships are different from the behaviors of cars or pedestrians. So the semantic models in maritime domain are proposed. The simple event model (SEM), proposed by Van Hage [7], is used in ship trajectories to bridge the gap between the behaviors and semantics. It may be the first systematic study of the ship’s semantic trajectory. The ship behaviors are obtained from the trajectory by a piecewise linear segmentation. Different facets are used in the SEM to represent the ship behaviors. A system (RMSAS), proposed by Brüggemann S [27], combines static data from different sources using semantic techniques. Its applications verify that the system can increase the value of data and improve the processing workflow in the maritime domain. Considering the semantic trajectory as the “first-class citizen”, the datAcron project proposed the datAcron ontology to advance the integrated exploitation and management of massive and heterogeneous data in the maritime domain [1]. The critical points of the trajectory are kept after using the data-summarization techniques. Then, the trajectories are revisited with the datAcron ontology, represented at the semantic level.

Some literature focus on the maritime big data integration and fusion tasks using semantic technologies, and involve the ship’s semantic behavior. Dividino et al. [28] presented a data architecture for real-time data representation, integration, and querying over a multitude of data streams from AIS station, climate station, and ice station. The marine behaviors, such as approaching heavy weather condition areas and approaching areas of heavy ice, can be queried based on these data. In 2015, Santipantakis et al. [29] presented two ontology-based data integration systems for the recognition of maritime behaviors. The concepts of low and high level behaviors are defined, with some behavior examples. In 2018, Santipantakis et al. [10] proposed the novel framework based on their previous work, providing a unified representation of mobility data and other data sources. Some basic behaviors, such as stops and changes in speed and heading are recognized in the proposed framework. Claramunt et al. [30] summarized recent literature of maritime data integration and analysis, believing that the early recognition of behaviors is crucial to safety and operations at sea.

However, some aspects are not considered in the mentioned literature. Most studies refer to the ship behaviors based on the ship trajectory without context information (such as geographic information and traffic rules). Since the behaviors in typical scenes of harbor are not proposed in these literatures, they cannot be conveniently used in the harbor. Meanwhile, due to few studies on inherent relationships of ship states and behaviors, the high-level ship behaviors are mined by complex algorithms that are not universally applied to various scenes. Last but not least, the advantages of the semantic model have not been fully exploited, for example, the natural language can be expressed to users based on semantic query.

3. Semantic Network of Ship Behavior

As defined by Sowa [31], a semantic network is a graphic notation for representing knowledge in patterns of interconnected nodes and arcs. In the work, the semantic network is the network of classes/individuals (nodes) and the properties (arcs). The OWL API [32] is used to construct the semantic network, and Java is the programming language. To meet the current demand for the ship behavior research, the semantic network should

• express the concepts and the implicit correlations of ship behaviors in typical scenes clearly and comply with the rules;
• store the historical behaviors for reasoning, trajectory annotation, and semantic query;
• contain the reasoning method to obtain the potential behavior from the basic behavior.

3.1. Framework of the Semantic Network

The semantic network is in the form of triple:

$$\mathit{S}\mathit{N}=\{\mathit{C},\mathit{P},\mathit{I}\}$$

(1)

where $\mathit{S}\mathit{N}$ is the semantic network; $\mathit{C}=\{C_1,C_2,\dots ,C_n\}$ is the Class (all semantic terms in the semantic network are italic in the work), which contains the ship behaviors and the other concepts; $\mathit{P}=\{P_1,P_2,\dots ,P_n\}$ is the Property, which is the collection of relations, and the interactions between Classes; $I_i$ is the Individual of the Class, which means the specific object, e.g., a container named KUOTAI is an Individual of Ship.

The semantic model (See Figure 1) includes eight core Classes, which means $\mathit{C}=\{S,St,B,P,T,Ty,TraS,TraP\}$ (

Table A1. Used abbreviations in this manuscript.

Abbr.	Term	Abbr.	Term	Abbr.	Term
An	Anchor	I	Inter-Slice Influence	St	State
Anc	Anchorage	i	individual	STP	Should Turn to Port
Ap	Approach	inGD	in General Direction	STS	Should Turn to Starboard
Ar	Arrival	isS	is Safe	STto	Should Turn to
B	Behavior	isUns	is Unsafe	SU	Speed Up
Be	Berth	J	Join	sub	has subclass
BT	Begin Time	KC	Keep Clear	s = 0	Speed = 0
C	Characteristic	L	Leave	T	Time
Cr	Cross	P	Place	t	turning
D	Deviate	Pro	Probability	TL	Traffic Lane
d	deviate	RA	Right Angle	TP	Turn to Port
De	Departure	R/S	Run/Stop	TraP	Trajectory Point
Do	Dock	S	Ship	TraS	Trajectory Segment
DtoP	Deviate to Port	s	speed change	TS	Turn to Starboard

shows the used abbreviations in the work).

• Ship is represented by the ship’s unique identifier—MMSI, or ship’s name (such as “KUOTAI”).
• Place can be represented by name, latitude and longitude, or relative position of other geographical locations. It can be related to other ontologies such as GeoNames [33].
• Time should be consistent with the W3C standard, such as 2018-06-25 T11:55:56+08:00. If the behavior is not finished or the start time is unknown, Time will be ambiguous. It has subclasses Begin Time and End Time, which connect with Type and Trajectory Segment.
• Type indicates the type of ship, including Container Ship, Ferry, and High Speed Ship. The same ship may have different types at different time, e.g., a ship is a tug over a period and a towed ship over another period. Therefore, Type has the properties of has Begin Time and has End Time.
• State (basic behavior) is the information obtained from the trajectory data and the context data directly. It is usually at a certain moment, such as the turning direction and the location. States in all typical scenes of harbor are recognized in Section 4.
• Behavior (potential behavior) usually occurs over a period, such as Turn to Starboard and Speed Down. Behaviors are reasoned from States by DBN, as shown in Section 5.
• Trajectory Segment is part of the trajectory. There is has Filiation property that represents a filiation relationship between two Trajectory Segments, which can guarantee the continuity of trajectory segments. The Trajectory Segment, connecting the Begin Time and the End Time, occurs during the period between them.
• Trajectory Point is the collection of all trajectory points, connected with Time by at Time property.

The relationships in the semantic model include the Object Property and the Data Type Property. In the work, the Object Property $R_O$ is as follows:

$$R_O=\left\{isTy,atP,atT,hasBT,hasET,hasTraS,hasTraP,hasB,hasSt,hasF,hasI,hasC\right\}$$

Trajectory Segment connects Behavior by has Behavior because the behavior usually lasts for a period and covers multiple consecutive trajectory points (except for the Enter/Leave behavior in Figure 2, it connects a special Trajectory Segment with only one trajectory point). In contrast, as each trajectory point has its own state, the Trajectory Point always connects State by has State. In this way, the ship’s behaviors and states are stored in the semantic model, which can facilitate reasoning and querying.

There are the Property has Characteristic (hasC) between the behaviors at the same time, and has Inter-Slice Influence (hasI) between the behaviors in adjacent time. hasC and hasI are the conditional probability and the transfer matrix in the DBN, respectively.

The Data Type Property is the data description or data restriction of the Class or Individual, e.g., the probability of Speed Up is 0.73. The Behavior is reasoned by DBN to get the probability, so it has a Data Type Property named has Probability.

3.2. State and Behavior in the Semantic Network

Figure 2 shows State and Behavior in the semantic network. According to the International Regulations for Preventing Collisions at Sea (COLREGS), the local Vessel Traffic Services (VTS) rules and the good seamanship, the inherent relationships of States and Behaviors can be obtained. The States and Behaviors in all typical scenes of harbor are introduced as follows. When the behavior is influenced by historical behaviors, there will be the hasI property on itself.

General Scene:

Behaviors:

• Speed Change (SC): The significant speed change over a period, with three Individuals including Speed Up (SU), Speed Down (SD), and Run/Stop (R/S).
• Turning (TU): The significant direction change over a period, with three Individuals including Turn Starboard (TS), Turn Port (TP), and Go Straight/Stop (GS/S).
• Enter/Leave (E/L): The ship enters or leaves an area, and it has three Individuals, including Enter (E), Leave (L), and not Enter and Leave (EandL).

States:

• speed change (s): The velocity change at certain time, and it has similar Individuals with Speed Change.
• turning (t): The direction change at certain time, with similar Individuals with Turning.
• 2TimeSlice in/out (i/o): Two adjacent trajectory points in/out an area, with four Individuals including inΛin, inΛout, outΛout, and outΛin.

Property:

There are hasC properties between Speed Change and five historical states (turning t-4–t) because the result will be inaccurate if only one Trajectory Point is used. The work chooses five as the threshold through a large amount of data validation. The Turning behavior is as same as Speed Change behavior. The property between Enter/Leave behavior and i/o state is hasC because the they are in the same time slice.

Dock:

Behavior:

• Arrival/Departure (Ar/De): The ship arrives or leaves a dock, with three Individuals including Arrival (Ar), Departure (De), not Arrival, and Departure (ArandDe).
• Berth (B): The ship moors at a dock.

State:

• Dock (Do): The ship is in a dock.
• speed = 0 (s = 0): The velocity equals to 0.
• Type: The type of the ship, such as container. It used to indicate whether the dock is suitable for the type of ship.

Property:

If a ship berths, it has the Speed Down behavior apparently; in contrast, when the ship leaves the dock, it has the Speed Up behavior. Thus, there are hasC between Arrival/Departure and Speed Change, and hasI between Berth/Anchor and Speed Change. The Berth behavior is reasoned by the ship in a Dock and speed = 0, so there is the hasC property between Berth and Dock/speed = 0. The ship must be moored at a dock suitable for its type, so there is hasC between Type and Arrival/Berth.

Anchorage:

Behavior:

• Anchor (An): The ship anchors at an anchorage.
• Approach (Ap): The ship is close to the traffic lane after anchoring.
• Join (J): The ship joins the main traffic flow in the traffic lane after Approach behavior (COLREGS rule 10).
• Cross (C): The ship crosses the traffic lane after Approach behavior (COLREGS rule 10).

State:

• Anchorage (Anc): The ship is in an anchorage.
• speed < 1 (s < 1): The velocity is less than 1 kn.
• Right Angle (RA): The ship approaches the traffic lane at a right angle.
• Small Angle (SA): The ship approaches the traffic lane at a small angle.

Property:

As the speed may be greater than 0 (but usually less than 1) when a ship is anchoring, the Anchor behavior has the hasC with s < 1. If a ship enters a harbor area, and anchors in the Anchorage, it will Approach the traffic lane, and finally choose to Join or Cross the traffic lane. Therefore, there is Property hasI among the Behaviors around Anchorage. The ship should Cross with a Right Angle or Join with a Small Angle under COLREGS rule 10, so there is hasC between the Behavior and State around anchorage.

Traffic Lane:

Behavior:

• Deviate (D): The ship deviates to the boundary of the traffic lane in a period, and has three Individuals, including Deviate to Starboard (DtoS), Deviate to Port (DtoP), and not Deviate (D). Deviate behavior can give the ship an early warning and guarantee the navigation safety.
• Should Turn to (STto): The right direction that the ship should turn to, with three Individuals including Should Turn to Starboard (STS), Should Turn to Port (STP), and Should Go Straight (SGS).
• is Safe (isS): The safety index in the traffic lane.

State:

• deviate (d): The ship deviates to the boundary of the traffic lane at certain time.
• in General Direction (inGD): The ship proceeds in the general direction of the traffic flow in the traffic lane (COLREGS rules 9 and 10), and it has four Individuals, which are in General Direction I–IV. It is used to check whether the ship is navigating along the traffic lane.
• Keep Clear (KC): The ship keeps a traffic separation line/zone clear in the traffic lane (COLREGS rules 9 and 10), and it has three Individuals, which are Keep Clear I–III. It is used to check whether there is enough space with the boundary of the traffic lane.

Property:

There is hasC property between the Deviate behavior and historical deviate states, as same as the Turning and Speed Change behavior. The Deviate, in General Direction, and Keep Clear have hasC property with the Should Turn to. They also have the hasC property with is Safe, which represents the safety index.

4. Recognition of State

The State should be recognized from raw data accurately. Based on it, the high-level potential behavior can be reasoned by DBN.

4.1. Recognition of States in General Scene, Dock and Anchorage

Speed change and turning: Figure 3a shows the state turning is recognized by the vector product $\overrightarrow{\mathit{c}}=(x\overrightarrow{\mathit{i}},y\overrightarrow{\mathit{j}},z\overrightarrow{\mathit{k}})=\overrightarrow{\mathit{a}}\times \overrightarrow{\mathit{b}}$. $\overrightarrow{\mathit{a}}$ and $\overrightarrow{\mathit{b}}$ are lines connected by two adjacent trajectory points. When $z$ is positive, $\overrightarrow{\mathit{a}}$ to $\overrightarrow{\mathit{b}}$ is counter clockwise, then the ship at point A is turning to port; otherwise, the ship is turning to starboard. When $z$ equals to 0, $\overrightarrow{\mathit{a}}$ and $\overrightarrow{\mathit{b}}$ are collinear are collinear, and the ship goes straight. The recognition of speed change is based on the acceleration of the ship. If the acceleration of a trajectory point is positive, the ship at this point is in speed up state; if negative, it is in speed down state.

In/out: Figure 3b shows if the states of adjacent points $e$ and $f$ is the out Λ in, then the ship has Enter behavior, and the other combinations can be found in Section 6. This method can be used in area as long as it is a closed area. In addition to dock, traffic lane and anchorage, it can be used in the bridge area, foul area, fish trap area, fish haven, precautionary area, and prohibited area. If a ship enters a “no entry” area, such as the environment protection area and military area, the early warning can be given to the ship.

Other States can be obtained from raw data easily. The method to determine a ship in a Place (containing Anchorage, Dock, and Traffic Lane) is to judge a point in the polygon. Judging whether the ship Approaches the fairway at Small Angle or Right Angle is through calculating the angle between the ship’s heading and the traffic lane’s boundary.

4.2. Recognition of States in Traffic Lanes

Deviate: The deviate state is recognized by deviation length (DL). The DL is the trajectory length between the ship’s current position and the position when the ship crosses the boundary. If DL exceeds the threshold (given by the experienced ship officers or pilots familiar with the ship condition and sailing area), the ship has the deviate state. The deviation length considers the ship’s real-time position, movement status, and boundary shape, so DL can be used as a quantitative indicator of deviate.

The bow (position A in Figure 4) crosses the boundary when the ship deviates, and the Automatic Identification System (AIS) or radar data only has antenna installation position (position K in Figure 4). Therefore, the bow position should be calculated based on the position of AIS or radar data. d is the distance between A and K, whose specific value is determined by the ship type and antenna position; β is the heading of the ship.

A’s position is calculated by Equation (2) when $0\le \beta <90$/$90\le \beta <180$/$180\le \beta <270$/$270\le \beta <360$.

$$\begin{gathered} \{\begin{array}{l}{x}_a=x_k+d\sin \beta \\ y_a=y_k+d\cos \beta \end{array}\{\begin{array}{l}{x}_a=x_k+d\cos \beta \\ y_a=y_k+d\cos \beta \end{array} \\ \{\begin{array}{l}{x}_a=x_k-d\sin \beta \\ y_a=y_k+d\cos \beta \end{array}\{\begin{array}{l}{x}_a=x_k+d\sin \beta \\ y_a=y_k+d\cos \beta \end{array} \end{gathered}$$

(2)

According to the reasoning result of Turning behavior, when the ship has Go Straight behavior, the ship motion status is considered as a uniform linear motion. When the ship has Turn to Starboard/Turn to Port behavior, the ship motion is considered as a uniform circular motion (see Figure 5). The instantaneous trajectory radius of points A and K are denoted by $R_{av}$ and $R_{kv}$, respectively. The bearing of A and K’s instantaneous linear velocity direction are ${\phi }_a$ and ${\phi }_k$ respectively, and $\omega =|\beta -{\phi }_k|$.

The boundary of the traffic lane is generally a straight line in open waters and the smooth curve in some coastal waters and inland curved channel. The curved boundary is considered as connections of several curve segments that are arcs in the work. Moreover, the curved boundary may be the convex boundary or concave boundary, so there are six combinations of the boundary and Turning behavior (See Figure 6).

The work gives the calculation methods of Figure 6a,d, and the methods of other combinations are similar. Figure 6a shows the ship Runs near the convex boundary. Wherein, C is the point at which the ship crosses the boundary; O the circle center of the boundary; W the point of intersection of the line AO and the boundary; and $\theta$ the acute angle between the trajectory direction of A and the tangent of the boundary. The following relationship exists in Figure 6a.

$$l_{AC}=l_{AE}-l_{CE},l_{AE}=l_{AO}sin\theta ,l_{CE}=\sqrt{l_{CO}^2-l_{EO}^2},l_{EO}=l_{AO}cos\theta ,l_{AO}=y_r+R_r$$

(3)

DL is the length of line segment AC as follows.

$$DL=\left(R_r+y_r\right)sin\theta +\sqrt{R_r^2-{\left[\left(R_r+y_r\right)cos\theta \right]}^2}$$

(4)

Figure 6d shows the ship has Turn to Starboard/Turn to Port behavior near the convex boundary, and O_v represents the circle center of the trajectory of the bow.

$$l_{AO}=R_r+y_r,l_{AOv}=R_{av},\angle OA{O}_v=\theta$$

(5)

In the △AOO_v and △COO_v, according to the cosine theorem,

$$\{\begin{array}{l}cos\angle OA{O}_v=\frac{R_{av}^2+{(y_r+R_r)}^2-l_{O_{v}O}^2}{2R_{av}\times (y_r+R_r)}\\ \angle A{O}_{v}O=\frac{180}{\pi }\times arccos\frac{R_{av}^2+l_{O{O}_v}^2-{(R_r+y_r)}^2}{2R_{av}{l}_{O{O}_v}}\end{array},\angle C{O}_{v}O=\frac{180}{\pi }\times arccos\frac{R_{av}^2+l_{O_{v}O}^2-R_r^2}{2R_{av}{l}_{O_{v}O}}$$

(6)

Then DL can be calculated as

$$DL=\frac{\pi }{180}\times R_{av}\times \left(\angle A{O}_{v}O-\angle C{O}_{v}O\right)$$

(7)

In General Direction and Keep Clear: The in General Direction is recognized by calculating the angle between Course over Ground (COG) and the direction of the traffic lane. The Keep Clear is recognized by calculating the distance between the ship position and traffic separation line/zone. The degree of the two behaviors is classified in Figure 7.

4.3. Mapping Recognised States to Semantic Network

After the recognition of State, every Trajectory Point will have at least two States, i.e., speed change and turning. The running example (See Figure 8) shows the recognized Individuals of the State (speed change) and other Individuals when the ship (name: KUOTAI, MMSI: 371625000, type: container ship) arrives at a dock.

5. Semantic Reasoning of Ship Behavior Using DBN

There are some traditional reasoners based on logical reasoning in the semantic web, such as Racer, Fact++, Pellet and Hermit, which can be used to check the inconsistency of the ontology [34]. However, the reasoners are difficult to deal with the uncertainty and dynamic characteristics of ship behaviors. Therefore, a reasoning method is needed to adapt to the ship behavior characteristics.

Bayesian network is a graphical model of probabilistic inference, widely used in domains that need to handle the uncertain knowledge [35]. If the Bayesian network is used to reason the probability of ship behavior, the result will be more specific and accurate than the logical reasoning. When the source data is inaccurate or incomplete, the Bayesian network can give credible inference results based on the information of other nodes and its historical state, without missing results like logical reasoning. At the same time, the water traffic situation and the ship’s navigation state are changing with time, so DBN is required to infer the probability of current ship behavior under the time series dynamically.

The network structure of the semantic network and the DBN has high similarity, so the mutual conversion can be realized [36]. The semantic network and the DBN can be combined to make up their defects and give full play to their advantages.

5.1. Definition of DBN

DBN can be defined as an initial network and a transfer network (See Figure 9). Specifically, Figure 10 shows the DBN when the ship is in the Dock.

In the t-th time slice (t = 0), the semantic network is transformed into the initial network, and the probability distribution $P(X_0)$ of the initial time is defined. The subclasses belonging to State and Behavior in the semantic network are converted into the nodes of the DBN. The node corresponds to random variable $X_i$ with probability value $P(X_i)$. Individual corresponds to the value of random variable $X_i$, and all the values are discrete. Properties between subclasses correspond to directed arcs between nodes, indicating the direct influence between nodes, with corresponding conditional probabilities. The joint probability of all nodes within the initial network is

$$P(X_1,X_2,\dots ,X_{\mathrm{n}})={\displaystyle \prod _{\mathrm{i}=1}^{\mathrm{n}}P}(X_i|Pa(X_{\mathrm{i}}))$$

(8)

where $Pa(X_i)$ are all the parent nodes of any node $X_i$. If there is no parent node, then $X_i$ is the root node, and $P(X_i|Pa(X_i))=P(X_i)$ (indicating its prior probability).

On the one hand, the nodes in the t-th time slice ($t>0$) may be affected by the nodes in the previous time slice. On the other hand, the probability of the next time slice node may be predicted by the probability of the previous time slice node, so the transfer network needs to be defined. Assuming that the DBN conforms to the first-order Markov process, the transfer network is a Bayesian network that contains two adjacent time slices. Between the time slices, there are the influences between the Behavior nodes. The conditional distribution of the t-th time slice under all previous time slices is

$$P({\mathit{B}}_{\mathbf{t}}|{\mathit{B}}_{\mathit{0}:\mathbf{t}-\mathbf{1}})=P({\mathit{B}}_{\mathbf{t}}|{\mathit{B}}_{\mathbf{t}-\mathbf{1}})={\displaystyle \prod _{m=1}^{\mathrm{n}}P}(B_t^m|Pa(B_t^m))$$

(9)

where $B_t^m$ is the m-th Behavior node ($m=1,2,\dots ,\mathrm{n}$) in the t-th time slice; $Pa(B_t^m)$ the parent node of $B_t^m$, which can be in the same time slice or the previous time slice. The conditional probability of the State node is

$$P({\mathit{S}}_{\mathbf{t}}|{\mathit{B}}_{\mathbf{0}:\mathbf{t}-\mathbf{1}},{\mathit{S}}_{\mathbf{0}:\mathbf{t}-\mathbf{1}})=P({\mathit{S}}_{\mathbf{t}}|{\mathit{B}}_{\mathbf{t}})$$

(10)

DBN can be expanded to the T-th time slice by the initial network and the transfer network. The joint probability distribution from the 0-th time slice to the T-th time slice is

$$P({\mathit{X}}_{1:\mathrm{T}})=P({\mathit{B}}_0)·P({\mathit{S}}_0)·{\displaystyle \prod _{t=1}^{\mathrm{T}}P}({\mathit{B}}_{\mathit{t}}|{\mathit{B}}_{\mathit{t}-\mathbf{1}})P({\mathit{S}}_{\mathit{t}}|{\mathit{B}}_{\mathit{t}})$$

(11)

5.2. Parameter Learning

The conditional probability in the initial network and the transfer matrix in the transfer network are parameters $\mathit{\theta }$ in the DBN, and need to be given before reasoning. The work uses the maximum likelihood estimation method for parameter learning. Nodes in the DBN are all discrete random variables, and their distribution law is as follows.

$$\mathit{P}\{X=x\}=p(x;\mathit{\theta }),x=x^{(1)},x^{(2)},\dots$$

(12)

where $\mathit{\theta }={({\theta }_1,{\theta }_2,\dots ,{\theta }_{\mathrm{m}})}^{\mathrm{T}}$ is the unknown parameter, and ${(X_1,X_2,\dots ,X_{\mathrm{n}})}^{\mathrm{T}}$ the sample from $\mathit{X}$. The joint distribution law of the sample ${(X_1,X_2,\dots ,X_{\mathrm{n}})}^{\mathrm{T}}$ is called likelihood function, denoted denoted as $L(\mathit{\theta })$.

$$L(\mathit{\theta })={\displaystyle \prod _{\mathrm{i}=1}^{\mathrm{n}}P}\{X=x_{\mathrm{i}}\}={\displaystyle \prod _{\mathrm{i}=1}^{\mathrm{n}}p\{x_{\mathrm{i}};\theta \}}$$

(13)

Then, the parameter value that maximizes the likelihood function is chosen as the estimated value of the unknown parameter $\mathit{\theta }$, and the likelihood equation is

$${\frac{\partial \ln L}{\partial {\theta }_i}|}_{\theta =\widehat{\theta }}=0,\mathrm{i}=1,2,\dots ,\mathrm{m}$$

(14)

Thus, the maximum likelihood estimate ${\widehat{\theta }}_{\mathrm{i}}$ is obtained.

5.3. Dynamic Reasoning

The reasoning of a network with T time slices is to calculate the conditional probability of potential Behaviors under the observed States:

$$P(B_1^{1:\mathrm{n}},B_2^{1:\mathrm{n}},\dots ,B_T^{1:\mathrm{n}}|S_1^{1:\mathrm{m}},S_2^{1:\mathrm{m}},\dots ,S_T^{1:\mathrm{m}})$$

(15)

Through Bayesian formula,

$$P(B|S)=\frac{P(B,S)}{P(S)}=\frac{P(B,S)}{{\displaystyle {\sum }_{Z}P(B,S)}}$$

(16)

Through the independence hypothesis of Bayesian Network, it can be calculated as

$$\begin{array}{l}P(B_1^{1:n},B_2^{1:n},\dots ,B_T^{1:n}|S_1^{1:m},S_2^{1:m},\dots ,S_T^{1:m})=\frac{P(B_1,B_2,\dots ,B_n,S_1,S_2,\dots ,S_m)}{{\displaystyle {\sum }_{B_1,B_2,\dots ,B_n}P(B_1,B_2,\dots ,B_n,S_1,S_2,\dots ,S_m)}}\\ =\frac{{\displaystyle {\prod }_{u,v}P(S_u^v|Pa(S_u^v))}{\displaystyle {\prod }_{p,q}P(S_p^q|Pa(S_p^q))}}{{\displaystyle {\sum }_{B_1^{1:n},B_2^{1:n},\dots ,B_T^{1:n}}{\displaystyle {\prod }_{u,v}P(S_u^v|Pa(S_u^v))}{\displaystyle {\prod }_{p,q}P(S_p^q|Pa(S_p^q))}}}\end{array}$$

(17)

where $\mathrm{u}=1,2,\dots ,\mathrm{T}$; $\mathrm{v}=1,2,\dots ,\mathrm{m}$; $\mathrm{p}=1,2,\dots ,\mathrm{T}$; $\mathrm{q}=1,2,\dots ,\mathrm{n}$; $B_p^q$ is an actual value ${\mathit{B}}_{\mathrm{p}}^{\mathrm{q}}$; $Pa(S_u^v)$ the set of parent nodes of the evidence variable $S_u^v$. Based on it, the inference is transformed into the calculation of the known conditional probabilities, and the probability of the behavior can be inferred.

5.4. Mapping Reasoned Behaviors to Semantic Network

Figure 11 shows the reasoned Behaviors when KUOTAI arrives at Xiamen Dock. The Speed Down Behavior is reasoned by five speed down States, and the Arrival Behavior is reasoned by Speed Down (Behavior) and Container (Type) and Xiamen Dock (Place). The Xiamen Dock is a container dock, so the ship has the Arrival behavior only when the ship’s type is Container. There is the hasI Property between Berth and Arrival and Speed Down, which means that the ship will have the Berth Behavior at the dock if it has Arrival and Speed Down Behaviors at the dock.

6. Application Examples

The dataset, consisting of AIS data from 514 ships and geospatial data, has been used for examples in the Xiamen harbor of China on 13 April 2016. These ships all have the same type (container), similar weight (10,000–20,000 t), and length (100–200 m). The information of the traffic lanes, anchorages, and dock is obtained from Route Regulations of Xiamen VTS Area. The data of the ship named KUOTAI is mainly used to verify the accuracy and practicality. Table 1 shows the types of AIS data, and the geospatial data types are longitude and latitude. Figure 12 shows KUOTAI’s trajectory in Xiamen harbor.

6.1. Reasoning of Behavior Using DBN

After being marked by simple judgment and manual labelling, the AIS dataset is used as the parameter learning sample of DBN. The probability is shown in

Table 2. Marginal probability of DBN.

P(B)		P(Tu)		P(E/L)		P(Ar/De)		P(STto)
B	0.53	TS	0.33	E	0.01	A	0.03	STS	0.33
		TP	0.32	L	0.01	D	0.03	ST	0.33
B	0.47
		GS/S	0.34	EandL	0.98	AandD	0.94	SR	0.34

, Table 3 and Table 4, omitting the probability of some highly correlated behaviors (for example, if the t-1-th time slice has a Berth behavior, then the t-th time slice has a high probability of Berth behavior).

Table 5. Number and proportion of ship behaviors. The most of behaviors are Speed Change and Turning. Some ships berth/anchor at dock/anchorage all day, so the number of Berth/Anchor behavior is not equal to that of Arrival/Approach behavior.

Area	Behavior	Number	Proportion
Anchorage	Anchor	517	3.07%
	Approach	504	3.00%
	Join	347	2.06%
	Cross	157	0.93%
Dock	Berth	526	3.13%
	Arrival	521	3.10%
	Departure	519	3.09%
Traffic Lane	Deviate	897	5.34%
	is Unsafe	925	5.50%
	Should Turn to	925	5.50%
General Scene	Turning	3987	23.72%
	Speed Change	4609	27.42%
	Enter/Leave	2376	14.13%

shows the number and proportion of ship behaviors on that day. Based on the reasoned ship behaviors, the behavioral patterns can be mined.

The line charts in Figure 13 illustrate how the probability of KUOTAI’s behaviors varies in typical scenes of the harbor. The following describes the probability changes of Figure 13a.

Initially, the Speed Down behavior increases sharply until it reaches 1, followed by the Arrival behavior due to the hasI property. Then the probability of Run/Stop behavior increases when the probability of Speed Down and Arrival behavior decreases, because the ship will Berth at the Dock. Over the following 130 time slices, in spite of some minor ups and downs, the probability almost remains unchanged in all behaviors except probability of Departure increases slowly for the hasI property between Departure and Berth. After that, the probability of Departure still maintains an upward trend, and the Speed Up behavior shows great similarity with a more rapid rise. After a period of stability, the ship leaves the Dock, and the probability of Speed Up and Departure gradually drops to zero. In short, all behaviors are accurately inferred, and have specific probability values at all time slices.

Every trajectory point has a time slice, and when the probability of Behavior exceeds 0.7, it is annotated to the trajectory (See Figure 14). Figure 14c shows when KUOTAI has abnormal behavior, the Deviate, is Unsafe, and Should Turn to behaviors are inferred. These behaviors can give the ship clear instruction as Figure 14c.

6.2. Semantic Query Using SPARQL

The users can query anything from the semantic network using the Simple Protocol and RDF Query Language (SPARQL), which is a query language and data transmission protocol in semantic engineering [37]. The query mainly contains two clauses—SELECT and WHERE. The variable behind the SELECT clause represents the returned variable after the query. The specific content to be queried is behind the WHERE clause. In addition to some solution sequence modifiers (such as ORDER BY, DISTINCT, and LIMIT), other commonly used queries are as follows:

• FILTER query

In FILTER query, the corresponding result can only be returned when the return value is true. The ships that have Speed Up behavior can be obtained by the following query. The LIMIT modifier is used to limit the number of returned results.

Prefix Ship Behavior: <http://www.semanticweb.org/ontology/ShipBehavior.owl/> SELECT ?x WHERE { ?x Ship Behavior: has Speed Change ?y FILTER REGEX(?y, Speed Up) }LIMIT 5

• OPTIONAL query

The OPTIONAL specifies an optional part that will be returned with the result, but it allows the returned results without the optional part. For example,

Prefix Ship Behavior: <http://www.semanticweb.org/ontology/ShipBehavior.owl/> SELECT ?x ?y WHERE { ?x Ship Behavior: in Place Ship Behavior: Traffic Lane OPTIONAL (?x Ship Behavior: has Type ?y) }

• Integrated query

Multiple limits can be used to obtain accurate results, for example,

Prefix ShipBehavior: <http://www.semanticweb.org/ontology/ShipBehavior.owl/> SELECT ?Trajectory Segment ?Begin Time ?Dock WHERE { KUOTAI Ship Behavior: has Trajectory Segment ?Trajectory Segment. ?Dock rdf: type Ship Behavior: Dock. ?Trajectory Segment ShipBehavior: at Place ?Dock. ?Trajectory Segment Ship Behavior: at Begin Time ?Begin Time }

Based on the semantic query, semantic information can be expressed as the natural language to users with limited semantic background. In the emergency situation, the natural language can be sent to the user as a warning. Examples in different situations are given as follows:

• KUOTAI (Container) ends Anchor in No.3 anchorage at 2016-04-13T02:42:14+08:00 and begin Speed Up at 2016-04-13T02:44:54+08:00;
• KUOTAI (Container) is Approaching the Main Traffic Lane at 2016-04-13T02:49:23+08:00 and will Join or Cross the Main Traffic Lane.
• WARNING! KUOTAI (Container) is Unsafe in the Main Traffic Lane and Should Turn to Port because it is Deviate to Starboard at 2016-04-13T21:24:40 +08:00;

7. Discussion

In the work, a model named semantic model of ship behavior (SMSB) was proposed to extract ship behaviors from trajectory data in the semantic layer rather than in the data layer. As

Table 6. Comparison of existing models with our model. Yes means the model includes the corresponding function; limited means the model includes partial corresponding function; and no means the model does not include the corresponding function.

Models	Behaviors				Reasoning	Query
	General Scene	Dock	Anchorage	Traffic Lane
SEM [7]	No	Limited	No	Limited	No	Yes
RMSAS [21]	No	No	No	Limited	No	Yes
datAcron [1]	No	No	No	Yes	No	No
SMSB	Yes	Yes	Yes	Yes	Yes	Yes

shows, in the existing models, there are few systematic studies on the semantic modeling of ship behaviors. Compared to other models, the SMSB proposes recognition methods of states in all typical scenes as well as the reasoning method of the potential behavior.

The proposed model benefits users (such as ship officers, pilots, VTS operators and decision makers) with maritime traffic management and collision avoidance, as well as the smart ship to make quick decisions within a limited time. It can be used for trajectory annotation and semantic query in all typical scenes of harbor. The semantic network contains all the necessary semantic information from trajectory data, thus making the smart ship fully perceive the behaviors of the surrounding ships to analyze the traffic situation. At the same time, since the ontology is in a machine-readable form and easy to share and query, the semantic model enables smart ships to obtain information services efficiently. In addition, the ontology can be reused, which greatly reduces the repetitive calculations of raw trajectory data.

In the future, the semantic database of trajectory data will be constructed. Therein, the potential semantic information will be mined, such as accident-prone areas, and economic analysis between ports. Meanwhile, we will study behaviors in natural environment, including the wind, wave, and current. Then, more meaningful behaviors will be extracted from the trajectory data, such as behaviors in different weather.