Dealing with Multiple Optimization Objectives for UAV Path Planning in Hostile Environments: A Literature Review

Abstract

As Unmanned Aerial Vehicles (UAVs) are becoming crucial in modern warfare, research on autonomous path planning is becoming increasingly important. The conflicting nature of the optimization objectives characterizes path planning as a multi-objective optimization problem. Current research has predominantly focused on developing new optimization algorithms. Although being able to find the mathematical optimum is important, one also needs to ensure this optimum aligns with the decision-maker’s (DM’s) most preferred solution (MPS). In particular, to align these, one needs to handle the DM’s preferences on the relative importance of each optimization objective. This paper provides a comprehensive overview of all preference handling techniques employed in the military UAV path planning literature over the last two decades. It shows that most of the literature handles preferences by the overly simplistic method of scalarization via weighted sum. Additionally, the current literature neglects to evaluate the performance (e.g., cognitive validity and modeling accuracy) of the chosen preference handling technique. To aid future researchers handle preferences, we discuss each employed preference handling technique, their implications, advantages, and disadvantages in detail. Finally, we identify several directions for future research, mainly related to aligning the mathematical optimum to the MPS.

1. Introduction

The first documented military use of Unmanned Aerial Vehicles (UAVs) occurred in 1849 when the Austrian forces deployed around 200 hot-air balloons to bomb Venice [1]. Half a century later, in the Great War, the first pilotless aircraft was developed [2]. Although the balloons had little effect, and the aircraft were never used operationally, both showed a glimpse of the future of warfare. It was in the Vietnam war when reconnaissance UAVs were deployed on a large scale for the first time [2], flying over 3000 missions [3]. Later, during the battle of the Beka’a valley in 1982, decoy UAVs helped the Israeli air force win a decisive victory, destroying 17 out of 19 Syrian SA-6 air defence systems in ten minutes [3,4]. As a response, the Syrians sent out a large number of MiG jet fighters. However, with the help of a reconnaissance drone providing live video imagery of the take off of the jet fighters, Israel was perfectly prepared for the attack. At the end of the war, Syria lost 85 MiGs, whilst failing to destroy a single Israeli aircraft. As a result of the massive success of the operation the Israeli Air Force increased their UAV force substantially [3]. Nowadays, UAVs have become such a crucial asset on the battlefield, that the Russia-Ukraine war has been named the first drone war. Small inexpensive radio-controlled drones are being used on a wide scale to destroy millions of dollars of tanks and artillery systems [5]. One major threat against these radio-controlled drones is the use of Electronic Warfare (EW) systems to jam the drones’ radio frequencies or to trace the signal back to the drone operator. In fact, Russia’s EW systems are causing Ukraine to lose approximately 10,000 drones per month [6]. In response, Ukraine and Russia are investing heavily in Artificial Intelligence guided drones [5], for they have no radio frequency to jam nor an operator to trace the signal back to. Although the technology required still needs to be developed further, many believe that autonomous drones will be at the heart of future drone warfare [5,7].

One key feature for UAVs to operate without direct human control is autonomous navigation and flight. It involves the formulation of a global pre-mission flight path as well as local optimization based on real-time data during flight. In military applications, missions are meticulously planned, making pre-mission flight path planning especially crucial. Therefore the scope of this paper is on pre-mission path planning and for the remainder of the paper, the term “path planning” specifically refers to pre-mission path planning. Path planning can be defined as finding the optimal feasible route from point A to point B (possibly involving more points). A feasible route is one that meets the constraints posed by the UAV’s flight dynamics. Besides feasibility, it is important for a route to achieve certain goals. For example, it needs to be short, have minimal fuel consumption, and in a military context, be safe from hostile actors. Mathematically, these goals are quantified as optimization objectives and they can be conflicting. Conflicting objectives cannot be optimized simultaneously and thus trade-offs need to be made. For instance, a possible detour to evade a radar system will decrease the probability of detection, but increase route length. Similarly, flying fast will reduce the time in air but increase fuel consumption. Although the scope of the body of literature examined is on military UAV path planning, civilian UAV path planning also has conflicting objectives (e.g., speed and fuel-efficiency). The same methods of handling multiple objectives (i.e., preference handling) used in military applications are used in non-military applications. Therefore, the discussion below, especially the detailed discussion on different preference handling methods, is also relevant for non-military UAV path planning applications. The presence of these conflicting optimization objectives characterizes the UAV route planning problem as a multi-objective optimization (MOO) problem described by Equation (1). For illustrative purposes, we have avoided unnecessary mathematical rigor and described the problem loosely.

$$\begin{array}{cc}\hfill \underset{x}{\text{“}\mathrm{minimize}\text{”}}& (f_1\left(x\right),\dots ,f_k\left(x\right))\hfill \\ \hfill \mathrm{subject}\mathrm{to}& x\in \mathcal{X}\hfill \end{array}$$

(1)

The MOO problem loosely described by Equation (1) consists of five components. For each decision variable (1) x, a set of k objectives (2) ($(f_1\left(x\right),\dots ,f_k\left(x\right))$) is computed. In the context of route planning, the decision variable x represents a route, which is a time-dependent function $d\left(t\right)$ over 3-dimensional spatial coordinates. From this function, parameters such as velocity, acceleration, and attitude can be derived. Even though $d\left(t\right)$ is continuous, many path planning methods work on a discretized version of the problem. The k objective functions measure the performance of each route based on a certain metric. For instance, in the simplified example visualized in Figure 1, the objectives ($k=2$) of minimizing route length and danger are included. Moreover, each solution (i.e., route) must satisfy the flight constraints (3) imposed by the UAV’s flight dynamics to be included in the feasible set $\mathcal{X}$. Based on these ingredients, an optimization algorithm (4) searches for the optimal route. However, in multi-objective optimization, optimality is less straightforward than in single-objective optimization (SOO), which is why the quotation marks in “minimize” are included. Rather than dealing with scalar objective values, multi-dimensional objective vectors are considered, meaning the minimum is not simply the lowest objective value. Consequently, a different notion of optimality is needed. Pareto optimality defines a solution x as optimal when none of its objectives $f_i\left(x\right)$ can be improved without deteriorating the performance of another objective $f_j\left(x\right)$ [8]. If such an improvement $x^{\prime }$ was possible, route $x^{\prime }$ would be said to dominate route x. Of course, no rational decision-maker (DM) would ever prefer a dominated solution over the solution that dominates it. This is exactly what Pareto optimality captures; all non-dominated solutions are Pareto optimal. This (possibly infinite) set of optimal solutions is called the Pareto Front. Mathematically speaking, all routes on the Pareto Front are equally good. However, a DM will prefer some routes over others. For example, in the simplified Figure 1a, two optimal 2-dimensional routes are visualized: the short but dangerous route $x_1$, and the long but safe route $x_2$. As they are both on the Pareto Front (see Figure 1b), neither route $x_1$ nor route $x_2$ is mathematically better than the other. Yet, depending on the mission scenario a DM might prefer $x_1$ over $x_2$ or vice versa. In particular, the relative importance of each objective can vary per mission. For instance, a high-risk, time-critical, rescue mission will accept higher danger levels than a simple transport mission. Moreover, losing a small quadrotor UAV is less important than losing a large expensive fixed-wing UAV. Essentially, the DM’s preferences determine which solution on the Pareto front is the DM’s most preferred solution (MPS). Preference handling (5) integrates the DM’s subjective preferences into the optimization model to align the mathematical optimum to the DM’s subjective optimum (i.e., the MPS). This is crucial, as the goal of the path planning problem is not as much to arrive at a mathematical optimum as to help the DMs arrive at their MPS. Commonly, a scalar composite objective function $F(f_1\left(x\right),\dots ,f_k\left(x\right))=h(f_1\left(x\right),\dots ,f_k\left(x\right))$ is defined that aggregates the individual objectives. The specific aggregation function h differs per method (e.g., product in Section 4.1 and weighted-sum in Section 4.2.1). Many other methods exist, each having their own advantages and disadvantages. So far, we have only discussed the path planning of a single UAV. For cooperative applications (i.e., multiple UAVs; for a review see Fevgas et al. [9]), the multi-objective nature of the problem is exacerbated. Not only does each UAV have its conflicting optimization objectives, but the optimal paths of all UAVs together are likely to be conflicting as well. Therefore, MOO is even more important in the case of cooperative path planning. For the remainder of this paper, we have not differentiated between the two. However, this would be an interesting direction for future research.

Over the past decades, UAV path planning has been an active research field. Many papers have proposed novel approaches to model one (or multiple) of the five aforementioned components. So far, the literature has predominantly focused on developing new optimization algorithms. This trend is also evident in existing review papers. Indeed, as shown in

Table 1. Focus of existing surveys.

References	Year	OA	RR	OM	C	PH
Zhao et al. [16]	2018	✔	✗	✗	✗	✗
Song et al. [14]	2019	✔	∼	∼	∼	∼
Allaire et al. [12]	2019	✗	✔	✔	✔	✗
Aggarwal & Kumar [10]	2020	✔	✔	✗	✗	✗
Saadi et al. [15]	2022	✔	✗	∼	∼	✗
Jones et al. [11]	2023	✔	✔	✗	✗	✗

OA: Optimization Algorithm, RR: Route Representation, OM: Objective Modeling, C: Constraints, PH: Preference Handling; ✔: Concept covered, ∼: Concept introduced, ✗: Concept not covered.

, Nearly every survey has extensively covered the optimization algorithms (component (4)) proposed by the literature. Component (1), how to model routes, has been covered by Aggarwal & Kumar [10], Jones et al. [11] , and Allaire et al. [12]. They proposed a classification based on Cell Decomposition approaches, Roadmap approaches, and Artificial Potential Field approaches. As most of the approaches make a discrete approximation of the continuous 3D space by a grid or a graph, the planned route has a lot of sharp turns. Therefore, to ensure the route satisfies the flight constraints imposed by the UAV, the route needs to be smoothed. An overview of trajectory smoothing techniques is provided by Ravankar et al. [13]. Allaire et al. [12] cover the modeling of the optimization objectives (2) and constraints (3). They identified that most of the current literature oversimplifies the dynamical constraints. Additionally, they gave an extensive overview of the optimization objectives used in 60 papers [14,15]. Preference handling (component (5)) , on the other hand, has largely been neglected. It is only briefly touched upon by Song et al. [14], as they only briefly mention the weighted sum method as a potential modeling choice without going into detail. To address this gap, this literature review discusses the preference handling techniques employed in the literature. As mentioned before, preference handling aims to align the DM’s subjective optimum with the mathematical optimum of the optimization model. Clearly, even if the optimization algorithm is perfect, it will never reach the MPS if preference handling is done incorrectly. Therefore, focusing on preference handling is of paramount importance. This review provides a comprehensive overview of preference handling in the military UAV path planning literature over the last twenty years. Both the methods and the implications arising from choosing these methods are discussed. By doing so, we aim to answer the following two research questions (RQs):

• RQ1: What preference handling techniques have been used in the military UAV path planning literature?
• RQ2: For each of the identified methods: What are the advantages and disadvantages of using these methods?

By answering RQ1, we aim to provide an overview of the preference handling methods used in the military UAV route planning literature. This overview provides insight into the state of the art, thereby identifying gaps in the research literature. In particular, we aim to identify the extent to which DMs have been included in the literature. By answering RQ2, we aim to provide future researchers with an overview of the advantages and disadvantages of the employed preference handling methods. This critical discussion intends to provide future researchers the necessary tools to determine which preference handling method to use. In particular, this aims to help researchers ensure that their problem modeling aligns the mathematical optimum to the DM’s MPS. As the same preference handling techniques are used in non-military applications, this discussion is relevant to research on non-military UAV path planning as well.

The remainder of this paper is organized as follows. Section 2 describes the research methodology. In particular, the process of selecting the papers included in this review are described. In Section 3, to answer RQ1, we provide a comprehensive overview of the different preference handling techniques employed in the literature. Furthermore, we investigate the extent to which DMs have been included in the performance evaluation of the proposed methods. Next, in Section 4, to answer RQ2, a critical discussion of the advantages and disadvantages of each preference handling method employed in the literature is provided. Finally, Section 6 concludes the paper and identifies directions for future research.

2. Methodology

This scoping review is largely based on the guidelines for PRISMA Scoping Reviews by Tricco et al. [17]. The review’s scope focuses on military UAV path planning. In particular, a military environment is characterized by the presence of hostile actors such as enemy radars and anti-air defences. These actors form danger zones in the environment. As these radars and anti-air defences are set up in such a way to best detect or destroy the UAV, completely avoiding these danger zones is usually not possible. Even if it were possible, some missions have crucial time requirements and therefore, a decrease in the flight time at the expense of an increase in danger can be warranted. To identify potentially relevant papers, the databases of the Netherlands Defence Academy (NLDA), the American Naval Postgraduate School (NPS), and the American Institute of Aeronautics and Astronautics (AIAA) were searched from 2003–2024. The NLDA and NPS libraries consist of a collection of multiple databases. For example, the NLDA library yields access to IEEE, ProQuest, Springer, and many more. The NPS library yields access to over 300 databases. Finally, AIAA is the world’s largest aerospace technical society with over 200,000 technical articles, including both military and civilian applications. The focus on hostile environments corresponds well to the military setting covered by the NPS and NLDA. Their extensive collection should cover most of the literature relevant to the review. The remaining gaps will be covered by the literature from AIAA, which includes a large set of technical aviation papers.

To search these databases the keywords need to cover the route planning aspect, as well as the hostile environment aspect. To include the route planning aspect, the following keywords are used: “Route planning” or “path planning”. For the hostile environment, we identified that the literature predominantly includes radars or anti-air defence systems as hostile actors. Therefore, we include the combination of threat or hostile and specify it to a military setting by adding the keywords missile or radar. Note that we have chosen not to restrict our keyword search to UAV path planning per se. As path planning for manned systems shares a lot of similarities to path planning for unmanned systems, we have chosen to include both. However, even though we did not restrict ourselves to UAV path planning, no papers focusing on pre-mission path planning for manned systems were retrieved. The resulting keyword search which was last performed on 13 August 2024 is:

• (“Route planning” OR “Path planning”) AND (hostile OR threat) AND (radar OR missile)

The keyword search resulted in a set of 848 papers, out of which 187 were duplicates. To filter this large set of papers to a representative and relevant set that can answer the proposed research questions, the procedure visualized in Figure 2 was performed. First, duplicates were removed. Secondly, for the remaining 661 papers, relevancy was judged based on the title and abstract. In this first check, 408 articles were removed. The reasons for removal were mainly because the papers were either not about flight path planning, or had no military application. Subsequently, from the 253 reports identified, 251 papers were retrieved. For these papers, the relevance was judged based on a full text read, and the eligibility criteria in

Table 2. Inclusion and exclusion criteria of the paper selection.

Inclusion Criteria	Exclusion Criteria
Significant contribution to research ( Average yearly citations ≥ 1)	Low academic impact (Average yearly citations < 1)
Threat as optimization objective	Completely avoid or ignore threats
Handle multiple objectives	Sole Focus on Threat or Distance minimization
Route planning in the flight domain	No route planning or different domain
Sufficient explanation of methodology	No details
Propose novel work	Minor change from existing work

were used. The criterion of academic relevance was included to restrict the set of papers to a set representative of the general academic consensus. The general idea is that papers that are rarely cited, have not made an impact on the route planning research community. Note that by requiring papers to be cited at least once a year, new papers with few citations are still included in the paper selection. Next, papers that completely avoided threats or ignored them altogether were removed because in military missions it is rarely possible to completely avoid threats. Completely ignoring threats, on the other hand, simplifies the problem in such a way that it is irrelevant to actual practical applications. Similarly, by solely minimizing the distance or threat the multi-objective nature of the problem is neglected and is therefore irrelevant to this literature review. Papers that do not focus on flight route planning clearly fall out of the scope of the paper. Finally, papers where no details on the methods were reported or which only had minor changes to existing research were removed. The final set of papers included in this review consists of 76 studies that form a representative subset of military UAV route planning papers.

For each of these papers, we categorized the preference handling technique into four different classes based on the point at which the DM’s preferences were included. Furthermore, we extracted more detailed information on the preference handling technique. More particularly, the specific method used was documented for each paper. Based on this information, a general overview of the state of the art of preference handling in the military UAV path planning community was provided to answer RQ1. Additionally, to identify whether papers evaluated the practical usability of their methods, we extracted how the method was evaluated (e.g., was a subject matter expert used as DM). To answer RQ2, we explain each method in further detail. In particular, we describe the advantages and disadvantages of the modeling choices identified in the preference handling stage.

3. Preference Handling in the UAV Route Planning Literature

Based on the point in the optimization process at which the preferences of the DM are included, the multi-objective optimization (MOO) literature [19] uses four different categories of handling conflicting objectives (see Figure 3). The preferences of the DM decide which routes from the Pareto front are preferred over others. The preferences can be integrated either not at all (no preference method), before (a-priori), after (a-posterior), or progressively during the optimization process (interactive). For the military UAV route planning problem, the DM can be the drone operator, an intel officer, or anyone else who plans the route.

• No-preference Preference Handling
  Firstly, in no-preference methods, the DM is not involved in the process. Instead, a solution which is a neutral compromise between all objectives is identified. This method is commonly used when no DM is present and it usually constructs a scalar composite objective function $F(f_1\left(x\right),\dots ,f_k\left(x\right))$.
• a-priori Preference Handling
  Secondly, in a-priori methods, a DM specifies their preference information before the optimization process is started. Most commonly, the preference information is used to construct a scalar composite objective function $F(f_1\left(x\right),\dots ,f_k\left(x\right))$. This has the advantage that the problem is simplified into an SOO problem. As a result existing SOO algorithms can be used. The main limitation is that the DM has limited knowledge of the problem beforehand, which can result in inaccurate and misleading preferences [20,21]. Intuitively, asking the DM to specify their preferences before the optimization corresponds to asking them to select a point on the Pareto Front (e.g., by specifying objective weights) before knowing what the actual Pareto Front and its corresponding tradeoffs look like. Consequently, a set of preferences specified a-priori might lead to solution x, whereas the DM might have preferred y when presented with both options.
• a-posterior Preference Handling
  Alternatively, a-posterior methods can be used. These methods include the DM’s preferences after the optimization process. First, they approximate the Pareto Front and then allow the DM to choose a solution from said front. The main advantage is that the DM can elicit their preferences on the basis of the information provided by the Pareto Front. The DM has a better idea of the tradeoffs present in the problem and can thus make a better-informed choice. Furthermore, it can provide the DM with several alternatives rather than a single solution. The disadvantage of this class of methods is the large computational complexity of generating an entire set of Pareto optimal solutions [22]. Additionally, when the Pareto Front is high-dimensional, the large amount of information can cognitively overload and complicate the decision of the DM [20].
• Interactive Preference Handling
  Finally, interactive preference elicitation attempts to overcome the limitations of the a-priori and a-posterior classes by actively involving the DM in the optimization process. As opposed to the sequential views of a-priori and a-posterior methods, the optimization process is an iterative cycle. Optimization is alternated with preference elicitation to iteratively update a preference model of the DM and to quickly identify regions of the Pareto Front that are preferred. Undesired parts of the Pareto Front need not be generated, which saves computational resources. Instead, these resources can be spent on better approximating the preferred regions, improving their approximation quality [23]. Lastly, being embedded in the optimization process helps the DM build trust in the model and a better understanding of the problem, the tradeoffs involved, and the feasibility of their preferences [24,25,26]. One challenge of interactive methods is that the interaction needs to be explicit enough to build an expressive and informative model of the DM’s preferences whilst also being simple enough for the DM to not cognitively overload them.

Within the UAV route planning literature, there is a large focus on a-priori preference handling approaches. Indeed, Figure 4 (for the raw data see

Table A1. Overview of Preference Handling in the Military UAV Path Planning Literature. N = Normalized, L = Linear, WS = Weighted Sum.

Reference	Class	Method	Evaluation
Flint et al. [28]	No Preference	Nash Product	Not Applicable
Kim & Hespanha [101]	a-priori	N-L-WS	Preference Assumed
Misovec et al. [102]	a-priori	L-WS	Preference Assumed
Dogan [62]	a-priori	Goals and Priorities	Multiple preferences visualized
Chaudhry et al. [63]	a-priori	Goals and Priorities	Multiple preferences visualized
Qu et al. [103]	a-priori	L-WS	Multiple preferences visualized
McLain & Beard [104]	a-priori	L-WS	Preference Assumed
Ogren & Winstrand [64]	a-priori	Goals and Priorities	Preference Assumed
Changwen et al. [105]	a-priori	L-WS	Preference Assumed
Weiss et al. [106]	a-priori	N-L-WS	Preference Assumed
Foo et al. [107]	a-priori	L-WS	Multiple preferences visualized
Ogren et al. [59]	a-priori	Goals and Priorities	Preference Assumed
Kabamba et al. [60]	a-priori	Goals and Priorities	Not Applicable
Zabarankin et al. [65]	a-priori	Goals and Priorities	Multiple preferences visualized
Zhang et al. [108]	a-priori	L-WS	Preference Assumed
Lamont et al. [72]	a-posterior	L-WS, Pareto Front	Provide Pareto Front
Zhenhua et al. [73]	a-posterior	Pareto Front	Provide Pareto Front
de la Cruz et al. [57]	a-priori	Goals and Priorities	Preference Assumed
Xia et al. [109]	a-priori	L-WS	Multiple preferences visualized
Tulum et al. [110]	a-priori	L-WS	Heuristic approximation
Qianzhi & Xiujuan [111]	a-priori	L-WS	Preference Assumed
Wang et al. [112]	a-priori	L-WS	Preference Assumed
Zhang et al. [113]	a-priori	L-WS	Preference Assumed
Besada-Portas et al. [58]	a-priori	Goals and Priorities	Preference Assumed
Xin Yang et al. [114]	a-priori	N-L-WS	Preference Assumed
Yong Bao et al. [115]	a-priori	L-WS	Preference Assumed
Kan et al. [97]	a-priori	L-WS	Multiple preferences visualized
Lei & Shiru [98]	a-priori	L-WS	Preference Assumed
Zhou et al. [99]	a-priori	L-WS	Preference Assumed
Holub et al. [116]	a-priori	L-WS	Multiple preferences visualized
Chen et al. [117]	a-priori	L-WS	Preference Assumed
Li et al. [118]	a-priori	L-WS	Preference Assumed
Wallar et al. [119]	a-priori	N-L-WS	Preference Assumed
Yang et al. [56]	a-priori	Goals and Priorities	Preference Assumed
Qu et al. [120]	a-priori	L-WS	Multiple preferences visualized
Duan et al. [121]	a-priori	L-WS	Preference Assumed
Chen & Chen [117]	a-priori	L-WS	Preference Assumed
Wang et al. [122]	a-priori	N-L-WS	Preference Assumed
Xiaowei & Xiaoguang [123]	a-priori	L-WS	Preference Assumed
Wen et al. [124]	a-priori	L-WS	Expert DM
Zhang et al. [125]	a-priori	L-WS	Preference Assumed
Erlandsson [46]	a-priori	reward shaping	Multiple preferences visualized
Humphreys et al. [126]	a-priori	L-WS	Preference Assumed
Tianzhu et al. [127]	a-priori	L-WS	Preference Assumed
Wang & Zhang [128]	a-priori	L-WS	Preference Assumed
Jing-Lin et al. [129]	a-priori	L-WS	Preference Assumed
Savkin & Huang [61]	a-priori	Goals and Priorities	Not Applicable
Zhang et al. [130]	a-priori	L-WS	Multiple preferences visualized
Maoquan et al. [131]	a-priori	N-L-WS	Preference Assumed
Danacier et al. [132]	a-priori	L-WS	Multiple preferences visualized
Chen & Wang [133]	a-priori	L-WS	Preference Assumed
Zhang & Zhang [96]	a-priori	L-WS	Preference Assumed
Patley et al. [134]	a-priori	L-WS	Preference Assumed
Ma et al. [135]	a-priori	L-WS	Preference Assumed
Dasdemir et al. [85]	interactive	Reference Point-based	Author as DM
Zhang et al. [136]	a-priori	L-WS	Preference Assumed
Wu et al. [137]	a-priori	L-WS	Preference Assumed
Xiong et al. [138]	a-priori	N-L-WS	Preference Assumed
Zhang et al. [139]	a-priori	L-WS	Preference Assumed
Yan et al. [47]	a-priori	reward shaping	Preference Assumed
Abhishek et al. [29]	no preference	Nash product	Not Applicable
Zhou et al. [140]	a-priori	N-L-WS	Preference Assumed
Zhang et al. [141]	a-priori	L-WS	Multiple preferences visualized
Xu et al. [142]	a-priori	L-WS	Preference Assumed
Huan et al. [143]	a-priori	L-WS	Preference Assumed
Leng & Sun [100]	a-priori	N-L-WS	Multiple preferences visualized
Yuksek et al. [48]	a-priori	reward shaping	Preference Assumed
Woo et al. [144]	a-priori	L-WS	Preference Assumed
Lu et al. [145]	a-priori	L-WS	Preference Assumed
Zhang et al. [146]	a-priori	L-WS	Preference Assumed
Alpdemir [49]	a-priori	reward shaping	Preference Assumed
Dasdemir et al. [76]	a-posterior	Pareto Front	Provide Pareto Front
Zhao et al. [50]	a-priori	reward shaping	Preference Assumed
Tezcaner et al. [77]	a-posterior	Pareto Front	Provide Pareto Front
Zhang et al. [147]	a-priori	L-WS	Preference Assumed
Luo et al. [148]	a-priori	N-L-WS	Multiple preferences visualized

), shows that, despite its weaknesses, over 90% of the approaches use a-priori preference handling techniques. In fact, only two papers used a no-preference handling method, four used preference handling after optimization, and only one paper used interactive preference handling. The remaining 69 papers all used a-priori preference handling. One likely cause for this predominant focus on a-priori preference handling is that these methods simplify the problem to an SOO problem, thereby allowing the authors to use SOO techniques. Another cause might be that authors are not familiar with multi-objective optimization. This is substantiated by the fact that 70% of the papers use the simple intuitive weighted sum method (see Section 4.2.1), often a method that comes to mind first when dealing with multiple optimization objectives. Besides a methodological bias on a-priori preference handling techniques, many papers neglect the DM altogether. In particular, as shown in Figure 5, 50 papers (66%) simply assumed the preference information to be known. They did not experiment with different preferences, nor how a DM should arrive at them. On the other hand, 15 papers (20%) did provide some examples of different preference information, such as a different weight vector. However, again, they simply selected some arbitrary preference information, without specifying how a DM should go about selecting the preferences. Similarly, all four papers that provided the DM with a Pareto front did not address how a DM should select the most preferred solution from the Pareto front. Moreover, one of the papers simply replaced the DM with a heuristic, assuming to be an accurate approximation of the preferences. For the interactive preference handling method, the author assumed the role of the DM and did not provide an evaluation of the practical usability for real-world domain experts. None of the aforementioned papers included a domain expert as DM in the evaluation of the method. Therefore, it is unclear whether the chosen preference handling methods can accurately model the preferences of the DM, let alone ensure that mathematical optimum is aligned to the MPS. Alternatively, one paper did include domain experts as DMs, extracting preference information via the Analytical Hierarchical Process [27]. However, they did not evaluate whether the optimal solution found was indeed the most preferred solution.

In conclusion, preference handling and its role in aligning the mathematical optimum with the DM’s MPS is neglected. Often preferences are simply assumed to be known, and real DMs are rarely included in the evaluation of the proposed methods.

4. Preference Handling Methods and Their Implications

To aid future researchers handle the preferences of the DM, we provide an extensive discussion of the methods used in the literature. More particularly, for each method, we introduce the method and its mathematical interpretation. Moreover, we highlight the different modeling choices available, state the implications, advantages, and disadvantages.

4.1. No Preference—Preference Handling

When a solution needs to be chosen in the absence of a decision-maker, a compromise solution needs to be found. Within the military route planning literature, two papers, by Flint et al. [28], and Abhishek et al. [29], have used no-preference methods. Both of these methods use a simplification of the Nash Product. In the military UAV path planning problem, the relative importance of individual objectives differs per scenario. For example, in high-risk, time-crucial missions, minimizing flight time might be more important than in low-risk transport missions. The lack of adaptability to different scenarios is therefore a crucial disadvantage of no-preference methods.

Nash Product

Flint et al. [28], and Abhishek et al. [29] maximized the product of the objective functions, as defined in Equation (2). Although the mathematical equation seems very simple, there is a lot more to it than first meets the eye.

$$\underset{x}{max}F(f_1\left(x\right),\dots ,f_k\left(x\right))=\prod _{i=1}^k{f}_i\left(x\right)$$

(2)

In fact, this method is (a simplification of) the Nash Product [30] or Nash Social Welfare function [31]. Essentially it finds a natural compromise between efficiency and fairness [32,33]. It aims to come close to each objective’s optimum whilst ensuring no objective is degraded by an unreasonable amount. The interpretation of the Nash Product stems from the 1950s when John Nash introduced his famous Nash Bargaining Solution [30] for bargaining games. By viewing the optimization objectives as player’s utility functions, multi-objective optimization problems can actually be seen as bargaining problems. To solve bargaining problems one can use the Nash Bargaining Solution, which requires a disagreement point $d_i$ to be provided. The disagreement point basically corresponds to the utilities under no collaboration. By collaborating, players can improve their utility. This added surplus, defined in Equation (3), is exactly what the Nash Bargaining Solution maximizes.

$$\underset{x}{max}\prod _{i=1}^n(f_i\left(x\right)-d_i)$$

(3)

Geometrically, as can be seen in Figure 6, this method finds the point on the Pareto front that maximizes the rectangle spanned by the vertical and horizontal axes between the point and the disagreement point [34]. Since the disagreement point defines the start of the vertical and horizontal axes, it has a direct influence on the optimization problem. For example, points with an objective value $f_i$ less than disagreement point $d_i$ cannot be optimal [35] (compare the orange rectangle to the red rectangle in Figure 6). Additionally, as can be seen in Figure 7, when the disagreement point is set too low, the Nash bargaining solution can provide an extreme solution instead of a compromise solution. Therefore, it is important to set the disagreement point properly. One common approach is to set the disagreement point’s values to the worst objective values of points on the Pareto front (i.e., nadir point) [34]. By doing so, one finds a balanced solution over the entire front. Note that for bi-objective optimization problems, the nadir point is easy to estimate. For more objectives, however, this becomes complicated. More details can be found in Section 4.3.1.

Besides the effect of the disagreement point, the Nash bargaining solution has some additional implications. Most of these properties are implied by the axioms upon which the Nash product is based. It is important to note that the four axioms below are proven for convex bargaining problems (f is convex). However, multiple extensions to non-convex bargaining problems with slightly different axioms exist [36].

• Pareto Optimality—Efficiency
  The identified Nash Bargaining Solution is a Pareto-optimal solution [34], which, of course, is a desirable property.
• Symmetry—Fairness
  When the game is symmetric, which is the case for the Nash Bargaining Solution defined in Equation (3), the solution does not favor any objective but returns a fair solution [37]. The concept of Nash Bargaining can be extended to include asymmetric games by introducing weights to the problem, as in Equation (4). These weights can be determined by the DM, where they correspond to the preferences with respect to certain objectives [38]. This extends the Nash Bargaining Solution from a no-preference method to an a-priori method.

  $$\underset{x}{max}\prod _{i=1}^n{(f_i\left(x\right)-d_i)}^{w_i}$$

  (4)
• Scale Invariance—No Normalization
  Mathematically, this property means that any positive affine transformation of the objectives does not influence the decision-theoretic properties of the optimization problem. Intuitively, this means that the Nash Bargaining Solution is independent of the unit of the objectives. It does not matter whether one is measuring route length in kilometers or miles. Instead of comparing units of improvement in one objective to units of degradation in another objective, ratios are compared. For example, a route twice as dangerous but half as long as another route is equally good in this view. One important implication of this is that there is no need to normalize the objectives before optimizing.
• Independence of Irrelevant Attributes
  This axiom states that removing any feasible alternative other than the optimal Nash bargaining solution does not influence the optimal solution. This axiom has often been critiqued [39], but in the rational optimization context, it makes sense. Removing a nonoptimal solution, should not change what the optimal solution is.

In conclusion, the Nash Product, if properly used, finds a Pareto-efficient compromise solution that balances efficiency within objectives and fairness across objectives. In order to use the Nash Product correctly, it is important to set the disagreement point properly. A common approach is to set the disagreement point as the nadir point, for which many different approximation methods exist. By doing so, one finds a compromise solution over the entire Pareto Front. Additionally, the Nash Product is scale-invariant, meaning there is no need to normalize objectives.

4.2. A-Priori Preference Handling

The most common method to deal with preferences in the UAV route planning literature is to deal with them a-priori. There are many different ways to incorporate preferences before the optimization process starts. Within the route planning literature, as visualized in Figure 8, three different general cases have been used. First, by far the most common method (54 papers), is to aggregate the objectives via a weighted sum (see Section 4.2.1). Subsequently, an SOO method can be used. Secondly, in five papers, the authors have handled the preferences of the DM by shaping a reward function to be optimized. As shown in Section 4.2.2, this is actually equivalent to the weighted sum method. Finally, in 10 papers, a DM can set goals for certain objective values and priorities in achieving them (see Section 4.2.3). Subsequently, lexicographic optimization or goal programming techniques can be used to solve the defined optimization problems.

4.2.1. Weighted Sum Method

The most used way to handle conflicting objectives is to scalarize the multi-dimensional objective function by taking the weighted sum [40,41]. When one encounters a problem with two competing objectives, one naturally thinks of minimizing the weighted sum of these objectives. It is thus no surprise that over 70% of the papers reviewed use this method. The main advantage is that after aggregation, the composite objective function can be optimized using SOO algorithms. In its most simplistic (and most used) form the composite function is defined by Equation (5).

$$\begin{array}{c}\hfill F(f_1\left(x\right),\dots ,f_k\left(x\right))=\sum _{i=1}^k{w}_i{f}_i\left(x\right)\\ \hfill s.t.\sum _{i=1}^n{w}_i=1\\ \hfill w_i\ge 0\end{array}$$

(5)

In principle, by systematically varying the weights and solving the corresponding series of SOO problems one could obtain the Pareto front. However, this has some disadvantages. First, a uniform distribution of weights does not necessarily result in a uniformly distributed set of solutions on the Pareto front [41]. Moreover, only convex Pareto fronts can be generated (see more below). Therefore, better alternatives for finding the Pareto Front, like the $ϵ$-constraint method, described in Section 4.3.1, exist.

Essentially, when one uses the Weighted Sum method, there are three modeling choices to be made. First, the constraints have to be specified. Secondly, the mathematical form of the equation has to be chosen, and finally, the weights have to be decided. All of these choices have certain implications for the optimization problem.

• Constraints—Normalization and Weakly Dominated Points
  The first constraint: ${\sum }_{i=1}^n{w}_i=1$ does not impose any constraints on the optimization problem. Any weight vector could be normalized to satisfy the constraint. However, restricting the DM to choose weights according to this constraint can cause unnecessary complications. Generally, the weight vector is interpreted to reflect the relative importance of the objectives [42]. In the case where one objective is five times more important than the other, it would be much easier for a DM to set the weights as $\overrightarrow{w}=(5, 1)$ than $\overrightarrow{w}=({\displaystyle \frac{5}{6}}, {\displaystyle \frac{1}{6}})$. This becomes especially troublesome when there are many objectives, hindering the interpretability of the weight vector. Therefore, it is better to normalize the weights automatically after weight specification rather than restricting the DM in their weight selection.
  The second constraint $w_i\ge 0$ can pose a problem when one of the objective weights is equal to 0. When setting $w_3$ to 0, a solution with objective values $(f_1\left(x\right), f_2\left(x\right), f_3\left(x\right)=(1, 1, 100)$ is equivalent to $f_1\left(y\right), f_2\left(y\right), f_3\left(y\right)=(1, 1, 1)$. The equivalence of these two objective vectors introduces weakly-dominated points (i.e., points like x for which a single objective can be improved without any deterioration in another objective) to the Pareto Front. When the DM truly does not care about $f_3$ this is not a problem, however, when one would still prefer y over x, one could set the weight of $f_3$ to a very small value $ϵ$ instead of 0 [43]. This allows the method to differentiate between $f\left(x\right)$ and $f\left(y\right)$ using the third objective $f_3$, removing weakly dominated points from the Pareto front.
• Mathematical Form—Imposing Linearity
  The mathematical form of the equation refers to the functional composition of the composite objective function. In all papers identified, a linear combination of objective functions, as defined in Equation (5), has been employed. However, there is no reason, besides simplicity, why the equation has to be a simple linear combination. In fact, because Equation (5) is a linear combination of the objectives, it imposes a linear structure on the composite objective function. Preferences, on the other hand, generally follow a nonlinear relationship [44]. A DM might approve a 20km detour to reduce the probability of detection from 0.2 to 0 but might be less willing to do so to reduce the probability of detection from 1 to 0.8. Furthermore, imposing a linear structure on the preferences leads to favoring extreme solutions whereas DMs generally favor balanced solutions [42]. This is particularly troublesome when the feasible objective space is non-convex, as can be seen in Figure 9 (compare Figure 9a to Figure 9b). The linear weighted sum is not able to find any solution between the two extremes. Points on this non-convex section of the Pareto Front cannot be identified by a linear weighted sum. In discrete cases, it has been shown that up to 95% of the points can remain undetected [45].
• Weight vector—Relative Importance of Objectives
  This very simple method poses a not-so-simple question: how does one determine the weights? There is no unique answer to this, as the weight vector $\overrightarrow{w}$ needs to be determined by the DM and it differs per scenario. The most common interpretation of the weights is to assign them based on the relative importance of the individual objectives. For example, in a bi-objective problem case, setting $w_1=1$ and $w_2=2$ assumes that objective 2 is twice as important. However, as shown by Marler & Arora [42] the value of a weight is significant relative to both the weights of the other objectives and all the objective function’s ranges. In particular, if objective 1’s range is much higher than objective 2’s range, objective 1 will still dominate regardless of the weight set. Therefore, normalizing the objectives is necessary. Although the normalization of objectives is a necessary precondition to interpret weights as relative importance, only a very small portion ($\approx 19\%$) of the literature has done so, as can be seen in Figure 10. Alternatively, one can interpret the ratio of the weights as the slope of the line crossing the mathematical optimum on the Pareto Front. By changing the slope, one can identify other points on the Pareto front. However, as one does not know the shape of the Pareto front a-priori, it is hard to relate the slope of the line to the final MPS. For example, in Figure 9b, a ratio of 1 does not actually result in a balanced solution.
  Additionally, it is hard for a DM to set the weights before having information about the possible trade-offs. Why should the weight vector $\overrightarrow{w}=(0.6, 0.4)$ be chosen? Why not $\overrightarrow{w}=(0.59, 0.41)$? How much would the solution and its objective values change when slightly changing the weights? These questions are important, as in a practical setting a DM will need to trust that the weights set lead to the most preferred solution and that that solution is robust to a small weight change.

In conclusion, the weighted sum method provides an easy way to deal with multiple conflicting objectives. Its scalarizing nature ensures that SOO methods can be used. However, when implementing the method, several points have to be taken into account. First, weights should only be set to 0 if DMs truly do not care about the objective. Alternatively, setting the weight to a very small amount will ensure that no weakly non-dominated points are found without degrading the performance. Secondly, to allow DMs to set weights intuitively, the weights should not be restricted to add up to 1, but rather be normalized after setting the weights. Thirdly, objective values should be normalized to allow for interpreting the weights as the relative importance of the corresponding objectives. Fourthly, the mathematical form of the equation should be representative of the DM’s preferences. If a linear combination is chosen, the preferences need to be linear as well. Moreover, a linear combination will only be able to find points on the convex hull of the Pareto Front. Finally, even if all these points are taken into account correctly it is still hard to determine the weights without any information regarding the possible trade-offs and solutions of the scenario at hand.

4.2.2. Reward Shaping

In reinforcement learning, a reward function (analogous to an objective function) is maximized. At each timestep t, a reward $R_t$ is calculated based on the state of the agent. The state is a representation of the environment that the UAV is in at time t. For instance, it includes the location of the UAV, the velocity of the UAV and the threat distribution. Both positive and negative rewards can be used. For example, to minimize the time spent in the air a negative reward ($R_t^D$) is awarded for being in the air. Additionally to remain undetected, a negative reward $R_t^T$ can be awarded for being within the line of sight of a radar. To force the UAV to the goal, reaching it can give a large positive reward. Equation (6) is one possible, exemplary way to model this reward function.

$$R_t^{total}=R_t^T+R_t^D$$

(6)

The reward values have an interpretation similar to that of the weights of the weighted sum method. For example, when $R_t^T=-10$ and $R_t^D=-1$ being in a threat zone at time t is 10 times worse than being in the air in general. Rewriting the reward values by scaling them to be −1, and 1 and integrating weights in the equation gives us Equation (7) below.

$$R_t^{total}=w_T{R}_t^T+w_D{R}_t^D,\mathrm{where}{R}_t^T, R_t^D\in [-1, 1]$$

(7)

The similarity between Equation (7) and Equation (5) is striking. In particular, one can view the composite reward function $R_t^{total}$ as a linear weighted sum of the individual rewards (in this case $R_t^T$ and $R_t^D$). Each of the individual rewards aligns with one of the optimization objectives. For example, in Equation (7) to minimize the time spent in the air the negative reward $R_t^D$ is added. Similarly to minimize danger, the negative reward $R_t^T$ is added. Therefore, the individual rewards are scalarized in the same way as the linear weighted sum method described in Section 4.2.1. Indeed, using a linear weighted sum of the individual rewards, as employed by Erlandsson [46], Yan et al. [47], Yuksek et al. [48], Alpdemir [49], Zhao et al. [50] is equivalent to the linear weighted sum method [51,52]. The implications for this method of rewards shaping are thus the same as the implications of the linear weighted sum method described in Section 4.2.1. For example, because the total reward is a convex combination of the individual rewards, policies that lie in concave regions of the Pareto Front will always have a lower total reward than the policies in convex regions of the Pareto Front [53,54]. In practice, this also means that by designing the reward function, the engineer includes assumptions about the DM’s preferences in the model [51]. Therefore, the DM would need to be included in the early stages of the development of the model.

4.2.3. Decision-Making Based on Goals and Priorities

Another a-priori preference handling method requires the DM to categorize each objective into a priority level and allows them to set goals for each objective. This method, formalized by Fonseca & Flemming [55], has quite a lot of flexibility. For example, Pareto optimality is represented by putting all objectives at the same priority level and not specifying any goals. Lexicographic optimization (first optimizing the most important objective, then the second, and so on) is represented by putting all objectives into different priority levels. Different interpretations of goal programming can be represented by setting the goals and priority levels in the desired ways. For example, in a bi-objective optimization problem, one objective can be set as a constraint (high priority with goal), and the other objective can be set as an optimization objective (low priority, no goal). Note that depending on the choice of priorities and goals, the method can result in a Pareto front rather than a single most preferred solution. Therefore, it can require additional preference information to reach the final most preferred solution.

Within the route planning literature, 10 papers have used variants of this method. Yang et al. [56], and de la Cruz et al. [57], later extended by Besada-Portas et al. [58] directly applied the method from Fonsesca & Flemming [55]. In particular, they define three different priority levels: constraints, important objectives, and less important objectives. First, the solutions must satisfy all constraints. Secondly, the routes are minimized with respect to the important objectives. When they are equivalent (in a Pareto sense), they are minimized with respect to the less important objectives. Finally, a simple heuristic is used to choose the final solution from the Pareto front. Although not directly based on Fonseca & Flemming’s [55] method, Ogren [59], proposed the use of Behavior Control Lyapunov Functions (BCLF). BCLF requests the DM to provide a priority table where each column is a set of goals for each objective. Each subsequent priority column imposes stronger limits on the objective values than the one before. Ultimately, the method aims to reach the highest priority column (the far right column), which is attained by satisfying each individual goal of said priority column. Kabamba et al. [60], and Savkin & Huang [61] both use lexicographic optimization, where they first minimize the maximum danger and when it is at its minimum they minimize the route length. Finally, Dogan [62], Chaudhry et al. [63], Ogren & Winstrand [64], and Zabarankin et al. [65] set one objective as a constraint and optimize the other objective, similar to setting a goal for one objective in the highest priority level and moving the optimization objective to the lower priority level.

Clearly, many different variants of methods using priorities and goals exist. Which method is better suited than the other is highly dependent on the DM and the specific problem. However, when designing a preference handling method based on goals and priorities one can take the following implications on using them into account:

• Priorities—Lexicographic Ordering
  Categorizing objectives into priorities imposes an absolute ordering on the objectives. Even an infinitely small improvement in a higher-priority objective is preferred over an infinitely large improvement in a lower-priority objective. This ordering is called a lexicographic ordering and its sequential optimization process is called lexicographic optimization [8]. First, the highest priority is optimized. Only when there are ties or when all goals are achieved, subsequent priority levels are considered. For each lower priority level, one thus need more ties or more achieved goals, making it less likely to even reach the lower priority levels. Indeed, the presence of an excessive amount of priority levels, causes low-level priorities to be ignored [66]. Moreover, even when only two priority levels are used, such a strong lexicographic ordering has strong implications. It makes sense when a clear natural ordering is available. For example, satisfying a UAV’s flight constraints is always preferred to optimizing an objective, as a route that cannot be flown is always worse than a route that can be flown. However, for optimization objectives, this is usually not the case. It is hard to argue that there is no trade-off possible between criteria. Surely, a huge decrease in the route length warrants a tiny increase in the probability of detection. Therefore, when setting priority levels, objectives should only be categorized differently when one is infinitely more important than the other.
• Goals—Satisficing vs Optimizing
  Setting goals has a profound impact on the resulting final solution—it can have as large an impact, if not more, as the mathematical definition of the objective functions themselves [67]. In fact, the entire underlying philosophy of the optimization process can change when a goal is set to an attainable value versus an unattainable value. When the goals are set at attainable values, goal programming essentially follows a satisficing philosophy. This philosophy, named after the combination of the verbs to satisfy and to suffice, introduced by Simon [68], argues that a decision-maker wants to reach a set of goals, and when they are reached this suffices for them and hence they are satisfied [69]. Mathematically, this means that objectives are optimized until they reach a goal value. Any further improvements are indifferent to the DM. For constraints, this makes sense, satisfying the goals imposed by a UAV flight constraints suffices. There is no need to make even tighter turns than the turn constraint imposes. For optimization objectives, however, this contradicts the entire concept of optimizing, for which “less is always better” (i.e., a monotonic belief). Even when the goal value for the danger levels is satisfied, a solution that has less danger is preferred. Note that this does not align with the principle of Pareto-optimality and can thus result in dominated solutions. This is precisely the main disadvantage of setting goals for optimization objectives. Indeed imposing attainable goals on monotonic optimization objectives should not be done. Instead, for monotonic objectives, goals should be set to unattainable values [70] to ensure the optimization process optimizes rather than satisfices. Alternatively, techniques for Pareto-efficiency restoration can be used (e.g., see Jones & Tamiz [71]). Finally, when objectives in the same priority level all have goals, the objective will be to minimize the sum of unwanted deviations from the goals. For this sum to weigh objectives equally, objectives need to be on the same order of magnitude (i.e., commensurable). Therefore, when they are incommensurable, the objectives need to be normalized [66].
  Clearly, setting goals properly is of the utmost importance. So, how would one reliably set goals? To determine whether a goal value is realistic and attainable, one needs information regarding the ranges of the objective functions as well as the trade-offs in objective space, and the beliefs of the DM about the objectives (e.g., is it a monotonic objective?). a-priori, such information is not present. Therefore, setting goals is usually not done a-priori, but rather it is an iterative (i.e., interactive) process.

In conclusion, handling preferences with goals and priorities can be a flexible method to deal with initial DM preferences. Priorities can be set when a strong hierarchical (i.e., lexicographic) ordering between objectives exists. This is surely the case between constraints and optimization objectives, but it is questionable whether this holds between optimization objectives. Secondly, when setting goals, it is important to understand the effect of the goal set. An attainable goal will lead to satisficing solutions, whereas unattainable goals will lead to optimizing solutions. Which one is preferred depends on the DM’s beliefs about the function of each objective (e.g., monotonic function). Finally, setting goals requires information about the ranges and trade-offs of the objective shape. Therefore, setting them should happen interactively, rather than a-priori.

4.3. A-Posterior Preference Handling

In a-posterior preference handling methods, the preferences of the DM are handled after the optimization. In particular, the Pareto front is generated, after which DMs need to choose their MPS. It thus consists of an optimization stage and a preference handling stage.

• Optimization: Generating the Pareto front
  To generate the entire Pareto front, the UAV route planning literature has used two distinct methods. In two papers [72,73], the Pareto front is approximated directly using metaheuristics, in particular, evolutionary algorithms. These optimization algorithms use an iterative population-based optimization process to get an approximation of the Pareto front. As the scope of the paper is on preference handling techniques, evolutionary algorithms will not be discussed. Instead, they are extensively covered in existing surveys identified in Table 1 as well as by Yahia & Mohammed [74] and Jiang et al. [75]. The two other papers: Dasdemir et al. [76] and Tezcaner et al. [77] reformulated the problem using the $ϵ$-constraint method. This method is a general technique that reformulates the problem into a series of SOO problems. Although it is not directly a preference handling method, it is a general method to deal with multiple criteria and can be combined with any optimization algorithm. Therefore, it is covered in Section 4.3.1.
• Preference Handling: Selecting the most preferred solution
  None of the identified papers that generate the entire Pareto front cover the preference handling stage after the optimization. Instead, they simply assume that the DM can choose their most preferred solution based on the Pareto front. However, as explained at the start of Section 3, choosing a solution from the Pareto front is not a straightforward task. For example, when the Pareto front is high dimensional, the DM will face cognitive overload. One paper [72] addresses this by first linearly aggregating multiple objectives into two composite objective functions to lower the dimensionality. Afterwards, the authors assume that the DM can choose their most preferred solution from the two-dimensional Pareto front. However, to linearly aggregate the objectives, the method requires a-priori preference information (i.e., a weight vector), thereby, introducing the same challenges of a-priori preference handling and in particular of the weighted sum. Clearly, the focus of the aforementioned papers is on the optimization process and on the creation of a Pareto front. The subsequent step of selecting the MPS, however, is neglected. The DM is not aided in this process, which hinders the usability of the method. In particular, the decision-aid aspect of optimization is neglected.

4.3.1. $ϵ$-Constraint Method

The $ϵ$-constraint method sets one arbitrarily chosen objective as an optimization objective and sets the other objectives as constraints. Mathematically, this means the problem is reformulated to Equation (8).

$$\begin{array}{cc}\hfill & \underset{x}{min}{f}_j\left(x\right)\hfill \\ \hfill & s.t.f_k\left(x\right)\le {ϵ}_{k}k=1,\dots ,pk\ne j\hfill \end{array}$$

(8)

At first, these constraints are very loose bounds. By systematically tightening the bounds, one can get the entire Pareto front. A visualization of this process can be seen in Figure 11. In this example, $f_1$ is arbitrarily chosen to be constrained and $f_2$ is minimized. The first constraint, ${ϵ}_1^1$ results in the far right solution on the Pareto front ($f{\left(x^1\right)}^{*}$), as $f_2$ is at its minimum there. By systematically moving the red line to the left (tightening the bound) one can obtain all the solutions. For example, at ${ϵ}_1^2$, one gets the green solution $f{\left(x^2\right)}^{*}$ and at ${ϵ}_1^3$, one gets the blue solution, $f{\left(x^3\right)}^{*}$. Of course, in practice, one would not use such a coarse $ϵ$ grid. In particular, the more fine-grained the $ϵ$ grid is, the more detailed the approximation of the Pareto front is.

The most notable advantage of using the $ϵ$-constraint method over the systematic weighted sum method is that also non-convex parts of the Pareto front can be generated, as can be seen in Figure 11. Besides the applicability to non-convex Pareto fronts, there are four components that have several implications for the optimization problem. First, the introduction of constraints changes the problem structure. Secondly, the $ϵ$-constraint method can generate weakly efficient solutions. Thirdly, in order to generate the entire Pareto front, one needs to know the range of the objectives (i.e., the start and end point of the varying of constraints). Finally, one needs to choose how to systematically tighten the constraint.

• Constraint Introduction—Problem Complexity
  The introduction of constraints can greatly complicate the structure of the optimization problem. In particular, for the shortest path problem, introducing constraints changes the complexity of the problem from polynomially solvable to NP-complete [78]. Therefore, for larger problem instances, an exact $ϵ$-constraint method for the UAV route planning problem is not possible. Instead, for example, metaheuristics can be used to solve the formulated SOO problem.
• Weakly Dominated Points
  The $ϵ$-constraint method can generate weakly dominated points. In particular, the method will generate all points that satisfy the constraint on $f_2\left(x\right)$ and for which $f_1\left(x\right)$ is at its minimum. When there is no unique solution (i.e., multiple solutions x for which $f_1\left(x\right)$ is at its minimum), all of these solutions can be generated. Of course, in this case, one would prefer to only keep the solution with minimal $f_2\left(x\right)$. Therefore, one could add a second optimization problem that optimizes w.r.t. $f_2\left(x\right)$ given the optimal $f_1\left(x\right)$ value. Note that this is lexicographic optimization [79]. Alternatively, one could eliminate these weakly efficient solutions by modifying the objective function slightly. In particular, in the augmented $ϵ$-constraint method, the problem is reformulated to include a small term meant to differentiate between weakly efficient and efficient solutions [43]. For example, in the bi-objective case, we add the factor $\rho f_2\left(x\right)$ to get Equation (9). When $\rho$ is sufficiently small to only differentiate between solutions with equal $f_1\left(x\right)$, we remove weakly efficient solutions from the equation. Note that this happens without the need for the formulation and solving of additional optimization problems, thereby saving computational resources. Alternative solutions to remove weakly efficient solutions are available (e.g., Mavrotas [80]).

  $$\begin{array}{cc}\hfill & \underset{x}{min}{f}_1\left(x\right)+\rho f_2\left(x\right)\hfill \\ \hfill & s.t.f_2\left(x\right)\le ϵ\hfill \end{array}$$

  (9)
• Start and End Points of Tightening—Objective Range
  In order to get the full Pareto front, one needs to know where to start systematically tightening the bound and when to stop tightening the bound. In particular, the loose bound (starting point) is the nadir point. The ending point of the method is when the bound is at its tightest and only one solution remains. This is at the ideal point, the point with the best possible objective values of points on the Pareto front. The ideal point is easy to estimate; one simply needs to solve a single-objective minimization problem for each objective i and the resulting optimal objective value will be the ideal point $I_i$ of dimension i [8]. For the nadir point, one can take the worst objective values found by solving the series of SOO problems. In the bi-objective case, this will find the exact nadir point. However, when there are more than two objectives, this can greatly over- or under-estimate the nadir point [8,80]. Many different estimation methods exist (for a survey see Deb & Miettinen [81]) but no efficient general method is known [8].
• Systematically Tightening the Weights
  Originally, the epsilon constraint method tightens the bounds of each constraint i by a value ${\delta }_i$. The value of ${\delta }_i$ relative to the range of the objectives $f_i$ determines how detailed the generated Pareto front will be. A large ${\delta }_i$ results in a coarse Pareto front, whereas a small ${\delta }_i$ results in a fine-grained Pareto front. In particular, this ${\delta }_i$ can be seen as the stepsize of the method. Increasing it makes bigger steps, leading to a coarser Pareto front. At the same time, it decreases the computational complexity. A bigger stepsize, reduces the amount of steps needed to reach the ideal point, hence fewer optimization problems need to be solved. Therefore, ${\delta }_i$ needs to be fine enough to get a good approximation of the Pareto front but coarse enough to prevent a lot of redundant computational effort [82]. Alternatively, one can tighten the weights adaptively. When the bound is at a non-Pareto efficient part of the objective space, larger steps can be taken, whereas when the bound is at a Pareto efficient point, small steps can be taken. An example of this can be found in Laumans et al. [82].

In conclusion, one can use the $ϵ$-constraint method to generate both convex and non-convex Pareto fronts. To do so, one needs to find (or approximate) the ideal and nadir point, and systematically tighten the bound between these points. By changing the step-size $\delta$ of the tightening, one can increase the detail of the Pareto front, at the cost of an increased computational complexity. Alternatively, one can set the step-size adaptively based on objective space information. Moreover, to guarantee efficient solutions, one can include a factor ${\rho }_i{f}_i\left(x\right)$ into the optimization objective per constraint i (${\rho }_i$ being sufficiently small). Finally, note that the introduction of constraints changes the complexity of the single-objective shortest path problem from polynomially solvable to NP-complete for the constrained single-objective shortest path problem. Therefore, for large problem instances, exact methods will not be possible and metaheuristics can be used.

4.4. Interactive Preference Handling

Interactive methods consist of an iterative cycle of preference elicitation, preference modeling, optimization, and feedback [83]. Additionally, the interaction pattern, the point at which a DM can supply feedback is important. Essentially, the DM provides some preference information upon which a preference model is built. Based on this model the region of interest (ROI) of the Pareto front is identified after which the optimization model generates solutions close to the ROI. Finally, feedback (like trade-off information) is provided to the DM such that the preferences can be updated and a novel iteration can start. Alternatively, one can distinguish a learning phase and a decision phase [84]. In the learning phase, a DM learns the possibilities and limitations of the problem and in the decision phase, the DM focuses on the ROI and tries to find their most preferred solution. Within the UAV route planning literature, one paper, by Dasdemir et al. [85], uses an interactive method. In particular, they use a reference point-based evolutionary algorithm.

Interactive Reference Point-Based Methods

Interactive reference point-based methods require the DM to supply them with a reference point. This reference point is a vector in objective space with certain goal levels (similar to the goals defined in Section 4.2.3). It provides the optimization model with information on what the ROI is. Finally, the optimization model uses this information to approximate the ROI and subsequently, it provides the DM with feedback on the possibilities, limitations, and solutions of the problem. Note that an interactive system has many more interdependent components than an a-priori or a-posterior method. Additionally, the components depend on the interaction with the DM, which is, of course, individual-specific. It is thus impossible to fully determine the implications arising from certain design choices. Therefore, the discussion below is aimed more toward explaining the general implications than toward covering very specific design choices.

• Preference Information—Reference point
  Preference information is supplied by means of a reference point. Providing reference points is a cognitively valid way of providing preference information [86], as DMs do not have to learn to use other concepts. Instead, they can simply work in the objective space, which they are familiar with. However, as discussed in Section 4.2.3, using goals can result in dominated solutions due to the satisficing philosophy. Therefore, it is important to handle dominated solutions when using interactive reference point-based methods. Dasdemir et al. [85] prevent dominated solutions by keeping track of a set of nondominated solutions throughout the entire optimization process. After the optimization is done, the set of nondominated points that are closest to the reference point are returned as the ROI. Therefore, assuming the evolutionary algorithm has found nondominated solutions, only nondominated solutions will be returned.
  Additionally, when achieving one goal is more important than achieving another goal, it can be difficult to represent this with a reference vector. Therefore, a weighted variant [87] exists that introduces relative importance weights in the distance function (see preference model below). Note that although this gives the DM the possibility to include preferences regarding the importance of achieving certain goals, specifying weights is not a trivial task (as described in Section 4.2.1).
• Preference Model—Distance Function
  Based on the provided reference point, a preference model is constructed. The preference model captures the subjective preferences of the DM and uses them to identify the ROI on the Pareto front. In interactive reference point-based methods, the preference model is usually fully determined by the combination of a scalar distance function and the reference point [87,88]. This distance function is sometimes called an Achievement Scalarizing Function, which essentially does distance minimization [89]. The choice of distance function has an influence on the chosen ROI. For example, by using a quadratic distance function, points far away from the reference point are punished more than by using a linear function. Moreover, by choosing a nonlinear distance function, one introduces nonlinearity, alleviating the issues of the weighted sum methods, described in Section 4.2.1. In Dasdemir et al. [85] the distance function is the normalized euclidean distance (which is nonlinear) to the reference point. The solution closest to the reference point will be ranked 1, the second closest will be ranked 2, etc. Whichever solution is closer (based on the chosen distance function) to the reference point is thus preferred by the DM.
  Instead of finding a single most preferred solution, interactive reference point-based methods aim to identify an ROI. To identify this ROI, one needs to balance specificity and diversity. In particular, the ROI should not collapse into a single point because it is too specific, but it should also not be too diverse such that it is spread around the entire Pareto front. Therefore, to tune the spread of the ROI, an $ϵ$-niching method is commonly used [83] (e.g., in R-NSGA-II [90] and RPSO-SS [91]). Dasdemir et al. [85] also employ such an $ϵ$-niching method. In particular, one representative is constructed and all solutions within an $ϵ$-ball to the representative are removed. Subsequently, the next representative outside of the $ϵ$-ball is created and the process repeats. By increasing $ϵ$, more solutions will be deemed equal and the ROI will be wider and vice versa. Setting this design parameter, however, is not trivial [83]. For example, in the exploration phase a DM could want a more diverse spread of the ROI compared to when the DM tries to find the final most preferred solution. Therefore, adaptively changing the $ϵ$ based on whether the DM is exploring or selecting his MPS might be useful.
• Feedback to the DM—Pareto Front Information
  One of the main advantages of using interactive optimization methods is that the DM can learn from the possibilities and limitations of the problem at hand by receiving feedback from the optimization process. Based on the learned information, a DM can better identify the ROI of the Pareto front. The feedback provided to the DM can take many forms. For example, possible trade-off information or a general approximation of the Pareto front can be provided. For an interactive system to work correctly, it is vital that the interaction is not too cognitively demanding for the DM (i.e., cognitive overload). At the same time, the information needs to be rich enough to provide the DM with the necessary capabilities to correctly identify their ROI. Moreover, different DMs have different skillsets and affinities with optimization, therefore, interaction should be designed in close collaboration with the relevant DMs.
  In Dasdemir et al. [85], the feedback provided to the DM is a general approximation of the Pareto front (although no guarantees are given that the provided points are actually nondominated points). In the bi-objective case they evaluate, this is a cognitively valid way of information communication, but when the Pareto front becomes higher dimensional it can cognitively overload the DM [92]. Therefore, alternative visualization methods exist (e.g., Talukder et al. [93]). Besides that, the interaction is not designed nor evaluated with the relevant DMs, therefore, it is hard to say whether, in this context, Pareto front information is an appropriate way to communicate information to the DM.
• Interaction Pattern—During or After a Run
  Finally, when designing an interactive optimization method, one needs to decide at what point the DM is allowed to provide preference information. In particular, Xin et al. [83] have identified two interaction patterns: Interaction After a complete Run (IAR) and Interaction During a Run (IDR). Reference-point based methods generally follow the IAR method. IDR methods, on the other hand, give the DM more chances to guide the search process, thereby making it more DM-centric. Alternatively, one could allow the DM to intervene at will, either waiting for a complete run or stopping the run. Most importantly, there needs to be a balance between requesting preference information and optimization. We do not want to strain the DMs too much by asking them to provide a lot of preference information, but at the same time, we need to be able to build an accurate preference model and allow the DM to be in control.

In conclusion, interactive methods consist of an iterative cycle of preference elicitation, preference modeling, optimization, and finally feedback to the DM. Moreover, interactive methods can either request preference information after a complete optimization run or during an optimization run. Reference point-based methods use a reference point (goal vector) as preference information, which is a cognitively valid approach for the DM. Usually, they employ a nonlinear distance function that functions as a preference model. Additionally, $ϵ$-niching methods are used to tune the spread of the identified ROI. After doing a complete run (IAR), the DM is supplied feedback, usually some information on the Pareto Front, to base their new reference point on.

5. Comparison of Preference Handling Techniques

To compare all the identified preference-handling techniques, key evaluation criteria have been established. In particular, we evaluate them by the point at which the DM includes their preferences (e.g., a-priori or a-posterior), the computational complexity, the number of solutions generated, the optimality of the solution(s), the ease of preference specification, and finally by the preference model used. These criteria have been chosen to represent both the decision-aid aspect (number of solutions, ease of preference specification, and preference model) as well as the optimization aspect (complexity, optimality). A comparison based on these evaluation criteria can be found in

Table 3. Summary of Preference Handling Techniques.

Method	Class	Complexity	Number of Solutions	Optimality	Ease of Preference Specification	Preference Model
Nash Product	No-preference	1 SOP	Single solution	Pareto optimal	Not Applicable	Equal importance of objectives Favors Balanced solutions
Weighted Sum/Reward Shaping	a-priori	1 SOP	Single solution	(Weakly) Pareto optimal	Hard to provide weights and potentially misleading results	Overly simplistic Linear aggregation of preferences Favors extreme solutions
Priorities	a-priori	k SOPs	Single solution	Pareto optimal	Easy to provide ordering but questionable it exists	Imposes absolute ordering on objectives with very strong implications
Goals (reference points)	a-priori	1 SOP	Single solution	Not Pareto optimal	Specifying goals is cognitively valid but requires information on objective space range and trade-offs which are usually not available a-priori	Distance to reference point
$ϵ$-constraint	a-posterior	n SOPs	Entire Pareto front	Pareto optimal	Hard to select the final solution from Pareto front if it is high-dimensional	Not Applicable
Interactive Reference Point	Interactive	m SOPs	ROI of Pareto front	Not Pareto optimal	Cognitively valid and interaction allows for the required information to be provided	Distance to reference point

SOP = Single-objective Optimization Problem, $k=$ number of objectives, $n=$ number of steps in constraints, m = number of iterations

and is further discussed below. Additionally, the advantages, disadvantages, and suitable application areas are highlighted. Note that for the sake of clarity, priorities and goals are discussed separately, although many possible combinations exist.

• Nash Product
  The Nash product, a no-preference method, transforms the original MOO problem into an SOO problem by aggregating the k-objectives into a single composite objective function via the product rule (see Equation (3)). Subsequently, it allows the use of SOO techniques to solve the formulated problem. The resulting single Pareto Optimal solution assumes equal importance of objectives and finds a balanced tradeoff between the objectives. One of the advantages of using the Nash product is the low computational complexity, allowing the use of SOO methods. Additionally, the use of the product implies the method is scale-invariant which means no normalization of objectives is needed. The main disadvantage of the method is that it is impossible to tailor to different scenarios, which is much needed due to the large variety of military missions. Suitable applications for this method are applications where one does not have access to a DM and/or requires a single balanced compromise solution.
• Weighted Sum Method and Reward shaping
  The weighted sum method, an a-priori technique, transforms the original MOO problem into an SOO problem by linearly aggregating the k-objectives via a weighted sum (see Equation (5)). The Reward Shaping method, as used by the reviewed papers, handles preferences equivalently to the weighted sum method. By using regular SOO techniques one can solve the formulated problem. The resulting solution is weakly Pareto optimal, but when all the weights are set higher than 0 ($w_i>0,\forall i\in \{1\dots k\}$) the solution is guaranteed to be Pareto Optimal. The main advantage of using the Weighted Sum method or Reward shaping is that it greatly reduces the computational complexity of the problem. A disadvantage is that it can only find points on the convex hull of the problem. Besides that, it can favor extreme solutions even when weights are set equal. Because of this, setting weights is non-trivial, and the interpretation of the relative importance of objectives might not hold. Additionally, it requires the normalization of objectives. Finally, specifying preference information a-priori might result in suboptimal results. Suitable applications are those with convex Pareto fronts (e.g., linear objective functions), requiring a single optimal solution, and having access to objective trade-off information before optimization.
• Priorities
  Setting priorities transforms the original MOO problem into a series of j SOO problems (assuming j priority levels are used) via the use of lexicographic optimization. Subsequently, SOO techniques can be used to solve the formulated SOO problems. The resulting solutions are Pareto optimal, assuming the objectives all have their own priority level. The main advantage of using priorities is that it greatly reduces the computational complexity of the original MOO problem. Besides that, when a natural absolute ordering is available, it is easy to provide preferences. However, it is questionable whether such an ordering exists between optimization objectives. Applications suitable for this method are thus applications where a natural absolute ordering between objectives exists and a single optimal solution is needed.
• Goals
  Specifying goals (i.e., reference points) transforms the MOO problem into a single SOO problem. In particular, the composite optimization function is the sum of unwanted deviations from the goal values. Traditional SOO methods can be used to solve the formulated problem. The resulting solution is not guaranteed to be Pareto optimal. The main advantage of using goals is the reduction in computational complexity. Additionally, when information on the range and trade-offs in the objective space is available, setting goals is a cognitively valid way of expressing preference information. The main disadvantage is that such information is usually not available a-priori. Besides that, solutions found by these methods are not guaranteed to be Pareto optimal. Therefore, techniques for Pareto-efficiency restoration have to be used. Suitable applications are thus characterized by having access to objective range and trade-off information a-priori and needing a single optimal solution.
• $\mathbf{ϵ}$-constraint method
  The $ϵ$-constraint method transforms the MOO problem into a series of n (dependent on stepsize of tightening) SOO problems. The solutions to these problems form the approximated Pareto front. To formulate the SOO problems, one objective is kept as an optimization objective, and all the others are set as constraints. First, the constraints are set loosely, and in every iteration, they are tightened. The main advantage of this method is that it provides the DM with Pareto front information before requiring them to make a choice. Additionally, it provides the DM with multiple solutions. Finally, as opposed to the systematic weighted sum method, this method also finds points on the concave parts of the Pareto front. The main disadvantage of the method is the computational complexity. To get a reasonable level of detail of the Pareto front a very large amount of SOO problems need to be solved. Therefore, this method does not scale well to many objectives or complex SOO problems, like the resource-constrained shortest path problem. Finally, when the Pareto front is high-dimensional, it is hard for the DM to select the final solution. Suitable applications for this method are thus applications where multiple alternative solutions are preferred, few optimization objectives exist, and the formulated SOO problems are easy to solve.
• Interactive Reference point-based method
  Interactive reference point-based methods allow the DM to specify their reference points iteratively during the optimization process. By doing so they aim to approximate parts of the Pareto front close to the reference points. The resulting approximations are not guaranteed to be Pareto optimal by default. The main advantage is that the interactive nature of the method allows the DM to specify preference information under better-informed circumstances thereby simplifying the preference specification step and consequently improving the likelihood that the final solution identified is the MPS. Moreover, since preference information is supplied during the optimization process, only parts of the Pareto front have to be generated, saving computational resources. The main disadvantage of this method is that it is difficult to find the right balance between overloading the DM and getting enough preference information. Moreover, since the solutions are not guaranteed to be Pareto optimal, Pareto-efficiency restoration techniques or similar methods have to be used. Suitable areas of application are those with intensive access to DMs, who are willing to participate in the decision process.

6. Conclusions and Future Research

In this paper a comprehensive survey on preference handling in the military UAV route planning literature is presented. We provided a categorization in four classes: no-preference, a-priori, a-posterior, and interactive methods. The review of the literature has shown that, despite its weaknesses, an absolute majority (90%) of the papers uses a-priori preference handling. The literature predominantly focuses on the optimization aspect of the problem, neglecting the decision-aid aspect. This is also substantiated by reviewing how the current literature has evaluated the performance of their method. Many of the papers exclusively evaluate algorithmic performance and overlook the evaluation of the degree of decision-aid for the DM. In particular, 66% of the papers simply assumed the preferences to be a given known fact. Only one paper uses a subject matter expert as DM to extract the preferences. Clearly, many of the papers exclusively focus on the optimization aspect, but fail to ensure that the resulting mathematical optimum aligns with the most preferred solution of the DM. The proposed methods neglect to validate whether the chosen preference handling methods align with the DM’s capabilities and beliefs. To mitigate these issues, we have provided a critical discussion of the used preference handling methods, stating their advantages and disadvantages as well as providing several ways of mitigating undesired implications. By reviewing the literature, we identified several future research directions:

• Aligning the mathematical optimum with the MPS via accurate preference models
  The goal of the route planning problem is to aid the decision-maker in finding their MPS. Currently, most of the research is focused on improving optimization algorithms. No doubt, it is important to ensure that a mathematical optimum can be found. However, when this mathematical optimum does not result in the DM’s MPS, the optimization is useless. Therefore, more effort should be put into aligning the mathematical optimum with the DM’s MPS. For instance, the weighted sum method, used by 70% of the reviewed papers, employs an overly simplistic preference model, potentially leading to a mismatch between the mathematical optimum and the MPS. Lexicographic optimization methods assume an absolute ordering between optimization objectives, however, it is questionable whether such an ordering ever exists. Therefore, future research should employ accurate representations of the DM’s preferences.
• a-posterior and interactive preference handling
  As our literature review has shown, nearly 90% of the papers employ a-priori preference handling techniques. These methods have the major disadvantage that preference elicitation happens before any knowledge of the possibilities, limitations, and capabilities of the problem are known. As a result, the final identified solution might not be the MPS. Additionally, these methods provide a single optimal solution, whereas in some scenarios, particularly in military missions, providing alternative solutions is extremely useful. Therefore, it would be interesting to further investigate a-posterior and interactive preference handling techniques, as these techniques include preference elicitation in a later, better-informed stage, and provide multiple optimal solutions, mitigating the disadvantages of a-priori methods.
• Cognitive validity of preference elicitation
  Most of the literature has neglected the cognitive validity of the preference elicitation stage. For example, providing weights as preference information is a nontrivial task, potentially misleading the DM. For the method to work in practice, a DM needs to be able to trust the method. Therefore, future research should be geared towards using cognitively valid preference elicitation techniques.
• DM-centric evaluation
  Finally, as the goal of the route planning problem is to help DMs arrive at their MPS, the performance should be evaluated in this regard as well. In particular, methods should be evaluated based on their decision-aid capabilities and not just on algorithmic performance. Therefore, evaluation with expert DMs is an interesting direction for future research.
• Preference Handling in cooperative path planning
  Our paper has focused on the handling of preferences w.r.t. the optimization objectives for individual UAVs. When one focuses on cooperative path planning (e.g., Pan et al. [94] and Liu et al. [95]), one not only has multiple conflicting objectives per UAV, but each UAV’s optimal path might conflict with the others. Therefore, there is another layer of multi-objective optimization. For future research, it would be interesting to further investigate the way preferences are handled in such cases.
• Standardization of evaluation of methods
  Currently, research papers propose an optimization algorithm and evaluate it using self-defined scenarios. Since every paper defines its own scenario, it is difficult to compare the results of different methods with each other. For future research, it would be interesting to define a set of benchmark scenarios, on which each method can be tested. This would facilitate a better comparison of methods.
• Definition of optimization objectives
  Many different mathematical definitions of optimization objectives are employed throughout the literature. For example, to minimize the probability of detection, the literature has maximized the distance from the radar [96], minimized the radar power with various different functions [97,98,99], minimized a probability defined by a Gaussian [62], minimize time in threat zone [100], etc. Similarly for the flight time objective, many different definitions exist. All these different definitions have different effects on the resulting mathematical optimum.