Searching for Scalable Networks in Unmanned Aerial Vehicle Infrastructure Using Spatio-Attack Course-of-Action

Abstract

Unmanned aerial vehicles are increasingly being applied to various applications for a variety purposes, such as delivery, communication relay, mapping, and surveillance services. Through these, it is possible to provide flexible and stable network services. Unmanned aerial vehicles perform a wide range of tasks using Internet-of-Things technology, which needs Internet access. These internet connections, however, make it more possible for attackers to execute various security attacks on unmanned aerial vehicles. Therefore, it is crucial to identify the attack behavior of the adversary, which is called “course-of-action”, to preserve security in the unmanned aerial vehicle infrastructure. Based on learned data, the existing course-of-action method has the drawback of not functioning on various networks. As a result, in this paper, we propose a novel heuristic search-based algorithm to apply to various unmanned aerial vehicle infrastructures. The algorithm can build the optimal heuristic functions in various unmanned aerial vehicle network environments to explore the attack course-of-action and design the optimal attack paths to maximize total reward. Applying the proposed algorithm in two unmanned aerial vehicle network scenarios allowed us to confirm that the best attack path is well established.

1. Introduction

Nowadays, it is feasible to operate and configure various networks by constructing mobile infrastructure utilizing unmanned aerial vehicles (UAVs) [1]. UAVs or drones, as their name implies, are aircraft that fly autonomously under the control of computer software or via remote control [2]. UAVs are being employed for several purposes, including delivery, communication relay, and mapping, and this an increasing need for them [3,4]. In particular, using UAVs can provide reliable surveillance services and provide flexible and reliable network services, and various studies using them are actively being conducted [5,6,7,8,9]. Since all UAV components are connected, IoT technology is required to accomplish numerous functionalities using these UAVs [10]. However, due to this internet connection, various security threats to UAVs can occur if such UAV communication is hacked and used by the attacker. In particular, as shown in Figure 1, security threats, such as deauthentication attack, denial-of-service attack (DoS attack), and snooping attack, can occur in UAVs. Therefore, security is becoming increasingly crucial in UAV situations as the network grows more complicated and extended [11,12].

A cyber security technique examines the network environment to evaluate system security or identify vulnerabilities, which is often called “penetration testing” or “course-of-action (COA)”. To comprehend the behavior of potential attackers in these UAV infrastructures, it is necessary to approach the attackers’ activity. As a result, in this paper, we used attack COA to verify the security of the UAV infrastructure and to prepare for attack behavior. To design the attack COA search algorithm, various state-of-the-art artificial intelligence (AI) methods can be used, such as deep and distributed learning [13,14,15,16], reinforcement learning (RL) [7,17,18], etc. Most recent research has employed deep learning-based approaches.

However, they have problems in that they do not account for the conditions of various networks since they operate on only learned data. Therefore, these existing algorithms cannot be scalable. In this paper, to overcome these limitations, we propose a new algorithm, the “spatio-attack COA algorithm”, based on the heuristic search, among various heuristic methods [19]. Our suggested algorithm can search for an attack COA and plan paths in various types of networks by constructing optimum heuristic functions. Our algorithm’s structure is rooted in the A-star search algorithm, which focuses on finding the shortest path between the source and target nodes in scalable networks. However, unlike the traditional A-star search algorithm, our algorithm is designed for maximizing total reward as a Markov decision process (MDP). The major contribution of this paper are as follows:

• The proposed algorithm operates automatically with no data issues in scalable networks.
• The proposed algorithm can maximize total rewards while finding the optimal attack path.
• The proposed algorithm was applied to two UAV network scenarios to verify that the optimal attack path is well established.

The rest of this paper is constructed as follows. Section 2 describes the security threats in UAV infrastructures, and Section 3 identifies our proposed algorithm, the “spatio-attack COA search”. In addition, Section 4 displays the performance evaluation. Finally, Section 5 presents this paper’s conclusion and outlines future work.

2. Preliminaries

The associated models, methods, and prior research results related to the proposed algorithm are described in this section. In Section 2.1, we discuss the security threat models in UAV infrastructure, and in Section 2.2, we present the existing COA attack search methods. Finally, the MDP-based algorithm is described in Section 2.3.

2.1. Security Threat Models in UAV Infrastructure

As communication using UAVs has become more complex and diverse, the possibility of security threats has increased. The attacker may exploit the code of the software to hack sensitive data on the user’s device, which can cause serious adverse effects on communication in applications using UAVs [2]. Examples of representative security threats that can be posed to UAVs are as follows: deauthentication attack, DoS attack, and snooping attack. The details of each attack are described below.

Most UAV communications are based on the 802.11 protocol, Wi-Fi, which allows communication between the user and the ground control station (GCS). However, since the Wi-Fi is not encrypted with the management authentication system, the attacker may be able to access and take advantage of it without much difficulty [20]. In this case, the attacker may transmit the manipulated authentication frame to the GCS to obstruct the operating UAV connection [21]. The attacker may now control the UAV through this.

In addition, UAVs are especially vulnerable to DoS attacks since UAVs use limited computing resources and power. A DoS attack is the traditional flooding attack [22], in which the user’s network system is used to consume and deplete resources for the attacker’s malicious goals, prohibiting the user from using those resources for intended and regular applications [20]. Existing studies have been done to carry out DoS attacks on UAVs [20,22], and if many attack packets are sent to UAVs, the user’s system becomes paralyzed, and resources are insufficient. UAVs may not receive commands from the GCS, and an accident may occur [20].

The snooping attack occurs when a malicious attacker pretends to be a valid user and secretly bypasses the victim’s system to spy on sensitive information unlawfully. Furthermore, snooping attacks may also be executed on UAVs. Since a UAV uses the installed software of a given company, it could be abused for this attack [2]. For example, SensePost announced that the attacker could hack UAVs for malicious purposes and snoop data from the victim’s smartphone [2]. It has been mentioned that this attack can be particularly effective in crowded places with a density of smartphone users [23,24].

2.2. Existing COA Attack Search Methods

COA attacks are used in cyber attack simulation technologies to discover prospective attack risk factors by evaluating the organization’s policies and status and implementing security improvement plans. As a result, COA attack has become a crucial tool in cyber security for performing cyber analytics and exploring and analyzing a collection of security issues [25].

On the other hand, existing approaches of manually configuring cyber security threat COA attack strategies have several limitations, as shown in

Table 1. Technical comparison between previous COA methods and our spatio-attack COA search.

Method	Automatic	No Data Issues	Maximize Reward	Scalable
Traditional COA attack	X	X	X	X
Attack Tree	O	X	X	X
Attack Graph	O	X	X	X
Reinforcement Learning	O	O	O	X
A-star	O	O	X	O
Spatio-Attack COA	O	O	O	O

. Traditional COA attack approaches need the cooperation of security specialists with cyber security knowledge, and the effectiveness of their implementation is dependent on their skill. Furthermore, it is time-consuming and costly because cyber security must be examined directly. As a result, technology that can automatically conduct COA attacks is required to improve the efficiency of cyber security research and generate effective defense plans. As a result, various COA attack search methodologies have emerged, including sophisticated COA search and modeling techniques such as attack tree [26], attack graph [27], and game theory [28]. Furthermore, multiple cases of COA search using various AI-based algorithms have recently emerged.

Attack graphs and trees are explainable models for evaluating system security and identifying probable attack methods. However, because both methods need comprehensive knowledge of the target network’s structure and the configuration of all hosts, they are impossible to use from the standpoint of an actual attacker. Furthermore, due to the increase in state space and complex modeling processes, they are difficult to use in large-scale network scenarios [29]. The scalability issue arises in the attack graph and attack tree due to the exponential growth in the number of nodes and edges, which becomes a significant problem as the network size increases [30]. This issue is primarily caused by the necessity to consider all the complete attack paths in both attack graphs and attack trees. Additionally, difficulties occur when penetration testing is conducted using game theory. When employing game theory, the risk of not being able to see the setup of all networks and the possibility of deploying unreliable attack tools should be considered. When using game theory for penetration testing, a wholly controlled stochastic game is required, which makes satisfying a real-world network challenging. environment [31]. Furthermore, there is a limit to applying existing AI algorithms in the cyber security field. Variability issues might develop due to a lack of data, and there is a possibility of misleading conclusions being drawn due to data overfitting issues. To overcome these limitations, RL algorithms [32] have appeared in the security field.

2.3. Markov Decision Process (MDP)-Based Algorithm

RL is a machine learning area corresponding to the MDP. As shown in Figure 2, MDP consists of an agent, state, action, policy, environment, and reward. RL is a process where a decision maker observes state information from the environment in which the decision maker exists and probabilistically determines behavior through policies; when such behavior is applied to the environment, the decision maker is rewarded and receives the following state information. In other words, decision-making is achieved by continuously repeating state observation, behavioral decision, state transition, the next state, and the immediate reward. RL makes decisions to maximize the total amount of cumulative rewards available until the system ends. In our environment, we use MDP to make decisions to maximize the cumulative attack value when the agent attacks. In network topology, each node is in a state visited by an agent, and selecting the next node is an action. The agent makes a continuous decision with the reward obtained by evaluating the current state and behavior. To maximize the rewards that an agent receives, finding the optimal policy in each state is necessary. However, as shown in Table 1, since RL algorithms are based on training, there is a problem in that they cannot have scalability.

3. Spatio-Attack COA Search

To overcome the limits of the previous COA methods, as shown in Table 1, we designed a new method. We have coined the term and algorithm “spatio-attack COA algorithm” to test the UAV infrastructure for various security exploits. The term “spatio” refers to spatial intelligent attack COA heuristic for scalable networks. Our suggested spatio-attack COA algorithm may be used for various types of networks by developing appropriate heuristic functions for attack COA search and attack path planning.

As our aim is to handle scalable networks, our proposed algorithm is especially designed and implemented by utilizing intelligent and adaptive search-based techniques with an appropriate heuristic function definition. In addition, the algorithm’s structure is based on the A-star search algorithm [33], which is good at scalable networks. The A-star search algorithm is a popular graphic search algorithm, which is regarded as one of the most significant intelligent search algorithms since it combines breadth-first and depth-first search advantages. It is also a highly efficient heuristic method to locate a variable or low-cost path. It is worth noting that while our algorithm’s base structure is the A-star search algorithm, our algorithm is designed to maximize the total reward, while the A-star algorithm only considers the shortest path from the source node to the target node. Therefore, in the spatio-attack COA algorithm, the optimal attack path search process in a network environment can be modeled as the MDP.

As a result, as shown in Table 1, our spatio-attack COA algorithm operates automatically with no data issues in scalable networks and maximizes reward. In other words, our approach incorporates both reward maximization, a feature of reinforcement learning algorithms, and scalability, a feature of A-star algorithms. The detailed descriptions of each part are described below.

We designed the spatio-attack COA algorithm for scalable networks to establish an optimal attack path from the source node, where the attacker is located, to the target node. We first proposed that there is an intermediate node between the source node and the target node. The attack path from the source node to the target node is therefore made up of the attack paths from the source node to the intermediate node and the intermediate node to the target node combined. The value of the attack is calculated as follows for this purpose:

$$f\left(n\right)=g\left(n\right)+h\left(n\right)$$

(1)

$$g\left(n\right)=\left\{\begin{array}{cc}0\hfill & \hfill \mathrm{source} \mathrm{node}\\ {\sum }_{t^{\prime }=1}^{t}r({s_t}^{\prime },{a_t}^{\prime })\hfill & \hfill \mathrm{other} \mathrm{nodes}\end{array}\right.$$

(2)

$$h\left(n\right)=\sum _{t^{\prime }=1}^{T}r({s_t}^{\prime },{a_t}^{\prime })$$

(3)

As shown in the Formula (1), we defined the heuristic function $f\left(n\right)$ as the attack value at the current middle network node n. In addition, $g\left(n\right)$ denotes the addition of attack values from the attack source node to the existing middle network node n. Meanwhile, $h\left(n\right)$ is defined as the approximation of the attack values from the current middle network node n to our target node. The components of the formula in detail are described below.

As shown in the Formula (2), $g\left(n\right)$ is a value obtained by calculating the sum of edges passed to reach the intermediate node, which is the current node. The attack value in the case of the source node is 0, and the other nodes are ${\sum }_{t^{\prime }=1}^{t}r({s_t}^{\prime },{a_t}^{\prime })$. t denotes a time step as long as it advances to the next node. s and a indicate a state and an action, respectively. $r({s_t}^{\prime },{a_t}^{\prime })$ means the weight value of the edges passing by when moving from the current node s to the next node $s^{\prime }$.

$h\left(n\right)$, as shown in the Formula (3), is defined as an approximate value for the attack path from the current node to the target node, which may be considered a heuristic function. In this case, it is essential to accurately design $h\left(n\right)$ to identify the optimal attack path. We use vulnerability information and a standard vulnerability scoring system (CVSS) as $h\left(n\right)$ for all nodes [34]. CVSS prioritizes responses by scoring vulnerabilities in a standardized way that is mainly used in the security field to determine the severity of vulnerabilities. As shown in Formula (4), CVSS consists of two components: base score and exploitability score. The base Score shows the vulnerability’s risk, while the exploitability score indicates the probability of exploiting a specific vulnerability, with the value range reaching up to 10. In this paper, we weigh the base score, considering how feasible it is to use a specific vulnerability with the explorability score [35]. The detail of the h(n) function’s design is as follows:

$$h\left(n\right)=Scor{e}_{vul}=Base Score\times \frac{Exploitability Score}{10}$$

(4)

We used the corresponding Formula (4) in $h\left(n\right)$ as follows. Base score denotes a score between 0 and 10 based on the framework’s evaluation of the vulnerability’s attributes and severity. The exploitability score assesses the current vulnerability attack technology or code availability state. The public availability of simple exploit codes increases the severity of the vulnerability by allowing inexperienced attackers to exploit it, thereby increasing the number of possible attackers. Assuming that a significant vulnerability would likely reach the target node, we assigned a higher $h\left(n\right)$ value to nodes with higher vulnerabilities. In other words, 0.01 and 100 were given for the source and target nodes, respectively. Furthermore, $h\left(n\right)$ was assigned using CVSS information to nodes that attack vulnerabilities, 1.5 to nodes that access or execute files using execCode or accessFile, respectively, and 0 to all other nodes.

In the case of $g\left(n\right)$, the sum of the weight values of the edges between the nodes was passed by, and the weight value was also constructed using CVSS information of two adjacent nodes, while if it was not connected, −1 was given. In summary, f(n) is modeled using vulnerability information and learns to select nodes where attackers receive higher rewards. Therefore, a spatially optimal attack path search is preferentially performed using the corresponding algorithm. As a result, the values of each node in the f(n) function are shown in

Table 2. Information of vulnerability in scenario 1.

Node	Description	Value
0	execCode(UAV, root)	100
1	RULE 4 (Trojan horse installation)	0.1
2	accessFile(UAV, ‘/usr/local/share’)	0.1
3	RULE 16 (NFS semantics)	0.1
4	accessFile(fileServer, ‘/export’)	0.1
5	RULE 10 (execCode implies file access)	0.3
6	canAccessFile(fileServer, root, ‘/export’)	0.1
7	execCode(fileServer, root)	0.3
8	RULE 2 (remote exploit of a server program)	1.794
9	netAccess(fileServer, tcp, 80)	0.1
10	RULE 5 (multi-hop access)	0.1
11	hacl(webServer, fileServer, tcp, 80)	0.1
12	execCode(webServer, ‘Apache httpd’)	0.3
13	RULE 2 (remote exploit of a server program)	3.698
14	netAccess(webServer, tcp, 443)	0.1
15	RULE 6 (direct network access)	0.1
16	hacl(internet, webServer, tcp, 443)	0.1
17	attackerLocated(internet)	0.01
18	networkServiceInfo(webServer, https, tcp, 443, ‘Apache httpd’)	0.1
19	vulExists(webServer, ‘CVE-2015-3185’, https, remoteExploit, privEscalation)	3.698
20	networkServiceInfo(fileServer, http, tcp, 80, root)	0.1
21	vulExists(fileServer, ‘CVE-2006-3011’, http, remoteExploit, privEscalation)	1.794
22	RULE 17 (NFS shell)	0.1
23	hacl(webServer, fileServer, nfsProtocol, nfsPort)	0.1
24	nfsExportInfo(fileServer, ‘/export’, webServer)	0.1
25	nfsMounted(UAV, ‘/usr/local/share’, fileServer, ‘/export’, read)	0.1

and

Table 3. Information of vulnerability in scenario 2.

Node	Description	Value	Node	Description	Value
0	execCode(UAV2, root)	100	24	vulExists(fireWall, ‘CVE-2012-0883’, https, remoteExploit, privEscalation)	2.346
1	RULE 4 (Trojan horse installation)	0.1	25	networkServiceInfo(webServer, https, https, 80, ‘Apache httpd’)	0.1
2	accessFile(UAV2, ‘/usr/local/share’)	0.1	26	vulExists(webServer, ‘CVE-2012-0053’, https, remoteExploit, privEscalation)	3.698
3	RULE 16 (NFS semantics)	0.1	27	RULE 5 (multi-hop access)	0.1
4	accessFile(fileServer, ‘/export’)	0.1	28	execCode(webServer, root)	0.3
5	RULE 10 (execCode implies file access)	0.3	29	RULE 4 (Trojan horse installation)	0.1
6	canAccessFile(fileServer, root, ‘/export’)	0.1	30	accessFile(webServer, ‘/export’)	0.1
7	execCode(fileServer, root)	0.3	31	RULE 17 (NFS shell)	0.1
8	RULE 2 (remote exploit of a server program)	1.794	32	hacl(fireWall, webServer, nfsProtocol, nfsPort)	0.1
9	netAccess(fileServer, tcp, 80)	0.1	33	nfsExportInfo(webServer, ‘/export’, fireWall)	0.1
10	RULE 5 (multi-hop access)	0.1	34	networkServiceInfo(fileServer, http, tcp, 80, root)	0.1
11	hacl(webServer, fileServer, tcp, 80)	0.1	35	vulExists(fileServer, ‘CVE-2006-3011’, http, remoteExploit, privEscalation)	1.794
12	execCode(webServer, ‘Apache httpd’)	0.3	36	RULE 17 (NFS shell)	0.1
13	RULE 2 (remote exploit of a server program)	3.698	37	hacl(webServer, fileServer, nfsProtocol, nfsPort)	0.1
14	netAccess(webServer, https, 80)	0.1	38	nfsExportInfo(fileServer, ‘/export’, webServer)	0.1
15	RULE 5 (multi-hop access)	0.1	39	RULE 17 (NFS shell)	0.1
16	hacl(fireWall, webServer, https, 80)	0.1	40	nfsMounted(UAV2, ‘/usr/local/share’, fileServer, ‘/export’, read)	0.1
17	execCode(fireWall, ‘Apache’)	0.3	41	RULE 16 (NFS semantics)	0.1
18	RULE 2 (remote exploit of a server program)	2.346	42	accessFile(mailServer, ‘/export’)	0.1
19	netAccess(fireWall, tcp, 443)	0.1	43	RULE 17 (NFS shell)	0.1
20	RULE 6 (direct network access)	0.1	44	hacl(webServer, mailServer, nfsProtocol, nfsPort)	0.1
21	hacl(internet, fireWall, tcp, 443)	0.1	45	nfsExportInfo(mailServer, ‘/export’, webServer)	0.1
22	attackerLocated(internet)	0.01	46	RULE 17 (NFS shell)	0.1
23	networkServiceInfo(fireWall, https, tcp, 443, ‘Apache’)	0.1	47	nfsMounted(UAV2, ‘/usr/local/share’, mailServer, ‘/export’, read)	0.1

, and the values of the edges are equal to the sum of the values of the two connected nodes.

4. Performance Evaluation

The results are described in this section. In Section 4.1, we discuss the evaluation setting in the UAV infrastructure. In Section 4.2, we present the example-based evaluation results using the attack graph. Finally, in Section 4.3, the comparison result is described.

4.1. Evaluation Setting

In this study, we employed two scenarios, as shown in Figure 3a,b. Scenario 1 is a network topology with six nodes, and the attacker was configured to access the network topology over the Internet, as illustrated in Figure 3a. Furthermore, the GCS provided access to each end node, webserver, fileserver, and UAV. In addition, the UAV was designated as a target node. Scenario 2 consisted of nine nodes for network topology, and this scenario had a firewall that allowed only trusted users to communicate, as illustrated in Figure 3b. In addition, the webserver, fileserver, UAV1, and UAV2 were the end node of the network topology in scenario 2. Furthermore, we set UAV2 as a target node.

4.2. Example-Based Evaluation Results Using Attack Graph

Additionally, we used a multihost multistage vulnerability analysis tool (MULVAL) to perform COA attacks on UAVs. MULVAL, a logic-based security analyzer, is one of the most extensively used extendable attack graph analysis tools. Depending on the various network topologies, such as in Figure 3a,b, this tool generates a distinct attack graph [36]. The attack graph is a systematic depiction of the vulnerabilities in the system and their relationships generated by the approach of predicting the path the attacker would take to get into the system. In addition, the attack graph represents all network connections. It describes all potential attack paths based on vulnerability information, including CVSS score, the Shodan data used in Mulval, and the network environmental configuration such as firewalls and IDS (intrusion detection system) rules.

Using MULVAL, we generated attack graphs, as illustrated in Figure 4a and Figure 5 from the UAV-based network environment of scenario 1 and 2, respectively. It can be seen that the complexity of the attack graph increases with the complexity of the scenario. In addition, Table 2 and Table 3 show the information of vulnerability including that regarding the network connections or rules of IDs, in scenario 1 and 2, respectively. Furthermore, these tables determine the optimal attack path of the UAV infrastructure. As shown in Table 2 and Table 3, the start nodes are the numbers 17 and 22, respectively, indicating the attacker is located on the Internet, while the target node number is 0, indicating the attack is being executed on the UAV. The attack graph has several attack paths from the source node to the target node, and the attacker has no idea which one is the best. As a result, it is critical to identify the optimal attack path in the attack graph.

Using our proposed spatio-attack COA algorithm, we verified the optimal attack path that maximizes the reward in the attack graph of the scalable UAV-based environment. The optimal attack path from the source node to the target node in scenario 1 is in the order of 17 -> 15 -> 14 -> 13 -> 12 -> 10 -> 9 -> 8 -> 7 -> 5 -> 4 -> 3 -> 2 -> 1 -> 0, as shown in Figure 4b. In addition, in scenario 2, the optimal attack path is in the order of 22 -> 20 -> 19 -> 18 -> 17 -> 15 -> 14 -> 13 -> 12 -> 10 -> 9 -> 8 -> 7 -> 5 -> 4 -> 3 -> 2 -> 1 -> 0, as shown in Figure 5b. These attack paths can maximize the reward score between the source and target nodes in each scenario. Therefore, we verified that our proposed algorithms could determine the optimal attack path in various scenarios.

4.3. Comparison Results

In this section, we describe the comparison results between the performance of the spatio-attack COA algorithm and of the A-star search algorithm, which is the base structure of our algorithm. We conducted these comparisons in the two mentioned scenarios and compared the performance using reward and step metrics. The comparison results for the correlation between step and reward in each scenario are shown in Figure 6a,b, respectively for scenario 1 and scenario 2. Note that a short step indicates that the target was detected more quickly, and the attack value increases with reward quantity, representing better performance. In addition, the details on the performance are described in

Table 4. The comparison of step and reward between the spatio-attack COA algorithm and the A-star algorithm in scenario 1.

Algorithm	Reward	Step
Spatio-Attack COA	14.293	15
A-star	13.493	16

,

Table 5. The details of the correlation between step and reward between the algorithms in scenario 1.

Step	Algorithm
	Spatio-Attack COA		A-Star
	Node	Reward	Node	Reward
1	17	0.01	17	0.01
2	15	0.11	15	0.11
3	14	0.31	14	0.31
4	13	7.706	13	7.706
5	12	8.306	12	8.306
6	10	8.506	22	8.106
7	9	8.706	4	8.306
8	8	12.294	3	8.506
9	7	12.894	3	8.506
10	5	13.494	2	8.706
11	4	13.694	2	8.706
12	3	13.894	1	8.906
13	2	14.094	1	12.294
14	1	14.294	1	12.894
15	0	Target found	1	13.494
16	-	-	0	Target found

,

Table 6. The comparison of step and reward between the spatio-attack COA algorithm and the A-star algorithm in scenario 2.

Algorithm	Reward	Step
Spatio-Attack COA	19.986	19
A-star	14.198	30

and

Table 7. The details of the correlation between step and reward between the algorithms in scenario 2.

Step	Algorithm
	Spatio-Attack COA		A-Star
	Node	Reward	Node	Reward
1	22	0.01	22	0.01
2	20	0.11	20	0.11
3	19	0.31	19	0.31
4	18	5.002	18	5.002
5	17	5.602	17	5.602
6	15	5.802	31	5.402
7	14	6.002	30	5.602
8	13	13.398	30	5.802
9	12	13.998	30	5.802
10	10	14.198	30	6.002
11	9	14.398	28	6.402
12	8	17.986	28	6.202
13	7	18.586	39	6.202
14	5	19.186	4	6.402
15	4	19.386	4	6.402
16	3	19.586	3	6.602
17	2	19.786	3	6.602
18	1	19.986	3	6.602
19	0	Target found	2	6.802
20	-	-	2	6.802
21	-	-	1	7.002
22	-	-	1	10.39
23	-	-	1	10.99
24	-	-	1	11.59
25	-	-	1	13.398
26	-	-	1	13.998
27	-	-	1	13.798
28	-	-	1	13.798
29	-	-	1	14.198
30	-	-	0	Target found

.

4.3.1. Scenario 1

The comparison results of scenario 1 are shown in Figure 6a and Table 4. In the reward, we discovered that the spatio-attack COA algorithm outperforms the A-star algorithm: spatio-Attack COA algorithm (14.293) and A-star algorithm (13.493). In addition, in the step, the spatio-attack COA algorithm requires fewer steps to find the target node than does the A-star algorithm: spatio-attack COA algorithm (14 steps) and A-star algorithm (15 steps). Overall, the spatio-attack COA algorithm devised a more optimal attack path than did the A-star algorithm.

In Table 5, further details of each algorithm’s performance are described. As shown in Table 5, the path is identical until the 12 node, with the step being at 5. However, in step 6, each algorithm decides other paths to the target node. Additionally, as shown in the A-star algorithm part of the table, it can be seen that there are duplicate node numbers: steps 8 to 9 (=node 3), steps 10 to 11 (=node 2), and steps 12 to 15 (=node 1). This means that the A-star algorithm searches for the number of cases in path selection, while the spatio-attack COA algorithm does not. Therefore, it can be confirmed that the spatio-attack COA is more efficient than the A-star algorithm in terms of time and cost.

4.3.2. Scenario 2

The comparison results for scenario 2 are presented in Figure 6b and Table 6. In the reward, we discovered that the spatio-attack COA algorithm outperforms the A-star algorithm: spatio-attack COA algorithm (19.986) and A-star algorithm (14.198). In addition, in the step, the spatio-attack COA algorithm requires fewer steps to find the target node than does the A-star algorithm: spatio-attack COA algorithm (18 steps) and the A-star algorithm (29 steps). Overall, the spatio-attack COA algorithm showed devised a more optimal attack path than did the A-star algorithm.

In Table 7, further details of each algorithm’s performance are provided. As shown in Table 7, the path was identical until the node 17, where the step number is 5. However, at step 6, each algorithm decides other paths to the target node. Additionally, as shown in the A-star algorithm part of the table, it can be seen that there are duplicate node numbers: steps 7 to 10 (=node 30), steps 11 to 12 (=node 28), steps 14 to 15 steps (=node 4), steps 16 to 18 (=node 3), steps 19 to 20 (=node 2), and steps 21 to 29 (=node 1). This means that the A-star algorithm searches for the number of cases in path selection, while the spatio-attack COA algorithm does not. Therefore, it can be confirmed that the Spatio-Attack COA is more efficient than the A-star algorithm in terms of time and cost.

4.4. Discussion

In this section, we discuss further details of the experimental results. By comparing the experimental results, we found the optimal attack path for these UAV infrastructure scenarios using the spatio-attack COA algorithm and its benefits.

As shown in the Table 5 and Table 7, scenario 1, it can be seen that the spatio-attack COA method significantly outperforms the A-star algorithm in terms of overall reward. In Table 5, scenario 2, the difference in each algorithm’s reward is at 0.8, which appears negligible. However, in Table 7, the difference in each algorithm’s reward is 5.788, which is significant. Therefore, we could confirm that the reward difference of each algorithm is more significant according to the scales of the scenarios.

In addition, as shown in the Table 5 and Table 7, it can be seen that the spatio-attack COA method significantly outperforms the A-star algorithm in terms of overall steps. In Table 5, scenario 1, the difference in each algorithm’s is at step is at step 1, which could seem negligible. However, in Table 7, scenario 2, the difference in each algorithm’s step is at 11, which is significant. Therefore, we could confirm that the step difference of each algorithm is more significant according to scales of the scenarios. Additionally, if the duplicate nodes and step calculating are calculated, the A-star method’s path is shorter overall. However, it can be seen that the spatio-attack COA algorithm made a faster decision by calculating the path while moving forward at each node without duplicating.

Furthermore, since the spatio-attack COA algorithm’s structure is built on the A-star algorithm, it has certain advantages. The A-star algorithm’s ability to backtrack may help it escape infinite loops. The A-star method stops calculating the heuristic function and backtracks to all nodes constituting the path if a node forming an endless loop is discovered, resulting in the shortest path from the source to the target node [37]. The spatio-attack algorithm, therefore, has the advantage of not having an endless loop.

Based on these benefits, our spatio-attack COA algorithm approach can also be employed for IoT-enabled UAVs. There is also a risk of attacks on UAVs, which supports a variety of IoTs, such as smart farms and virtual environments [38,39,40,41]. Therefore, our method can be used to assess security systems and discover their weaknesses.

5. Conclusions and Future Work

Recently, UAVs have been used for various purposes using computer software. UAVs will become more popular, as they provide flexible and robust network services. However, several security vulnerabilities, such as DoS attacks and snooping attacks by malicious hackers, may occur since UAVs interact over the Internet. As a result, identifying an attacker’s potential attack behavior in a UAV is essential to maintaining the security of the UAV infrastructure. Therefore, applying the COA technique to UAV infrastructure among cyber security techniques that evaluate system security and identifying vulnerabilities by investigating the network environment are necessary. However, as the existing COA approaches are based on learned data, they cannot be used with scalable networks.

Therefore, in this paper, we proposed and evaluated a novel spatio-attack COA algorithm in the UAV-based scalability network. To handle scalable networks, we developed and implemented the suggested algorithm based on the A-star algorithm, employing optimal heuristic function definitions and intelligent and adaptive search-based approaches. In addition, this algorithm can determine the best attack path to maximize attack value based on the MDP method. Furthermore, we demonstrated that our proposed method performs effectively in two UAV-based network environments.

In future work, we will need to consider time-varying networks because our algorithm now considers only spatially scalable networks. In UAV infrastructure especially, it is essential to consider time-varying networks due to climate environment and connection with other networks. Therefore, we will consider the time-varying networks in UAV infrastructure including Monte Carlo (MC) tree search methods. In addition, our proposed algorithm should be evaluated compared to popular attack COA search methods. The suggested approach will determine if peripheral solutions are more beneficial for the attack COA in scalable and time-varying networks using the MC-based tree search. Moreover, for more practical applications, partially visible networks must be considered. Therefore, as opposed to those that know the whole network topology, we will consider algorithms that emulate actual hackers to determine optimal attack paths based purely on the knowledge of K-hop.