Networked Control of a Small Drone Resilient to Cyber Attacks

Abstract

With increasing advances in networked systems and networked control systems in everyday life, the problem of cybersecurity becomes crucial. Moreover, in some applications like small UAVs, the safety and integrity of the system and its surroundings are highly susceptible to cyberattacks. In this context, the current paper proposes a resilient networked control approach. The control structure is split into an inner and an outer loop. The outer position control loop uses measurements from motion cameras connected to a remote computer, while the commands are sent through the network. We consider the resilience problem for two types of cyberattacks: denial of service (DoS), emulated as an increase in the network transmission delay, and man in the middle (MitM), emulated as additive input disturbances. The mitigation for the DoS attack is performed through the help of a reference governor (RG), which uses the delay estimates and the system’s model to predict future safety violations and adapts the reference accordingly. The MitM attack is mitigated by an unknown input disturbance observer (UIDO) together with a RG. Experimental results on a Parrot Mambo drone show that both types of attacks are rejected successfully, ensuring a safe and stable flight.

1. Introduction

Unmanned aerial vehicles (UAVs) are increasingly being used in a wide range of applications, from surveillance and disaster management, to infrastructure inspection, precision agriculture, or even emergency medical services [1]. As a special subclass, small UAVs, which weight less than 1 kg and fly at velocities less than 10 m/s, are very popular due to their compact size and small cost, being used in several industries, from commercial and education, to research and military [2]. Among the different types of small UAVs, the quadcopter is the most often encountered, due to its easy-to-understand flight principles, simple architecture, high maneuverability and vertical landing/take-off. Despite these advantages, there are several challenges and open problems that limit the performance of these type of drones in real life applications. First, the quadcopter is an unstable and nonlinear process, so the control design for stability and tracking is not a trivial matter [3]. Second, not all of the states of the system are available, so different estimation algorithms need to be designed, depending on the available sensors, and whether the applications are indoor or outdoor [4]. The controller is highly dependent on the accuracy of these estimates. Third, there are computational limitations, both at a software and hardware level, while the autonomy is extremely limited [1]. Lastly, there is the problem of path planning, which is usually treated at a higher level, separated from the control problem [2]. This involves the task of generating the sequence of waypoints from the drone’s current position to the target (goal) position (the reference trajectory sent to the controller), ensuring collision avoidance [5], synchronization with other agents (robots or even manned vehicles) [6], or the optimization of some resources like battery autonomy or wireless transmission signal strength [7]. The interrelation between the control loop and path re-planning is an area of active research, and still represents a challenging task in practice.

As the systems in everyday applications become more and more interconnected, we can say that networked systems, and in particular networked control systems, are becoming ubiquitous [8]. This brings multiple benefits like increased reliability and scalability, ease of maintenance and higher performances overall. However, the network can induces extra problems like packet loss (information loss), time varying transmission delays, or bandwidth limitations [9]. Research in this field has focused on two areas: control over networks (i.e., to control one or multiple processes with a remote controller) [10] and control of networks (i.e., optimize the network performance, e.g., in terms of Quality of Service parameters) [11]. Networked control systems are highly used in applications with mobile robots (in particular drones), involving wide spatial areas, due to their increased autonomy. In such applications, using the already available computer networks infrastructure helps reduce costs and increases the overall performance. For small UAVs, moving a part of the control structure remotely helps in addressing the computational limitations mentioned before. Moreover, this permits a better integration between the control algorithm and the path planning algorithm, and facilitates extensions to cooperative control of multiple agents (robots).

One of the biggest challenges recently in using mobile robots in real life applications, involving interactions with other robots but also human–robot interaction, is ensuring safety and resilience [12] (Ch. 4). Safety, from a Control Theory perspective, involves that the system is not only stable, but also that it operates in a certain predefined safe region, where the integrity of the robot and of the surrounding environment is ensured. Although safety has been traditionally addressed in the robotics community at a higher level, a multi-level approach is needed for the safety of complex autonomous system. To this end, mid-level and low-level solutions for safety are subject to intensive research, with notable solutions in this direction being the safety filter [13] or the reference governor [14]. Besides safety, it is important that the system is resilient to less frequent but severe disruptions, like cyberattacks, natural disasters, etc. Resilience is different from the classical concept of robustness through the fact that it considers major perturbations, which can be identified in real-time, and the nominal performances can be recovered in a non-conservative manner. Both from a safety and resilience perspective, cybersecurity of network control systems involving mobile robots represents an emerging research field [15].

In this context, the current study proposes a networked control approach for small UAVs resilient to cyberattacks. We focus on indoor drone applications, where motion camera systems can provide position feedback, thus alleviating the sensing problem. The safety and resilience become of high priority, due to possible human–robot interactions and the constraint management needed in narrow closed environments. The original contribution consists of a mid-to-low-level approach in addressing the cybersecurity problem, by using a combination of a reference governor (RG) and a unknown input disturbance observer (UIDO). We focus on two often encountered types of cyberattacks: denial of service (DoS) and man in the middle (MitM). The DoS attack involves a sudden variation in the network time delay. Different from previous studies involving RG with time delays, like [16], we consider here that the time delay variation can be estimated through a time stamping technique. We extend the RG to include the time delay model, such that it uses the online delay estimate to predict future safety violations, and adapts the reference accordingly. The MitM attack involves false data injection, and since our focus is on stealthy attacks, we model the attack as an additive slow varying input disturbance. For the detection and mitigation of such an attack, we use a combination of a RG with an UIDO. The disturbance is estimated by the UIDO and rejected in steady state. The RG is extended to predict possible transients during disturbance compensation that can cause safety violations, and again adapts the reference in accordance. We show real time experiments on a Parrot Mambo drone that validate our solution.

The paper is structured as follows. Section 2 presents some preliminaries on the cybersecurity problem in networked control systems. Section 3 presents the baseline networked control structure for small drones used, and how the controllers are designed. Section 4 describes the resilient networked control structure, detailing the design of the RG and UIDO for detecting and mitigating the two types of attacks—DoS and MitM. The experiments on the real Parrot Mambo drone are presented in Section 5, while Section 6 discusses some final conclusions.

Notation. We denote by $\mathbb{R}$ and $\mathbb{Z}$ the sets of real and integer numbers, and ${\mathbb{Z}}_{+}$ represents the set of positive integers. For a continuous signal f(t) sampled with a sampling period $T_s$, we denote by $f\left[k\right]$ the value at the kth sampling instance, i.e., $f\left[k\right]=f\left(k{T}_s\right)$. $I_n$ denotes the identity matrix of dimension $n\times n$, while $0_n$ represents the zero matrix of dimensions $n\times n$. $s(·)$ and $c(·)$ are shorthand notations for $sin(·)$ and $cos(·)$. Throughout this paper, we use the dot convention for the time derivative ($\dot{x}:={\displaystyle \frac{dx}{dt}}$), and the time dependence ($x:=x\left(t\right)$) is omitted whenever the meaning is obvious.

2. Cybersecurity in Networked Control Systems

Security is defined in computer science as having the following goals: confidentiality, integrity and availability [17] (Ch. 1). Confidentiality refers to keeping information or resources concealed from unauthorized access. Integrity means how much the information can be trusted that it has not been modified by an unauthorized third party (attacker). Availability can be regarded as the access to information in a timely manner (availability to the information can be withhold or denied through a security breach). These concepts can be transferred to the security of cyber-physical systems, and in particular to networked control systems, but their meaning holds greater weight. A security breach in cyber-physical systems involving control loops on real physical process can have disastrous consequences, affecting the stability and safety of the system. This can be seen more clearly from an attacker’s perspective, in terms of disclosure attacks, deception attacks, and disruption attacks, which correspond to the three security goals mentioned above [18].

Although disclosure attacks are of greater interest in information-theoretic studies [19] (e.g., eavesdropping), in networked control systems, this can also have dire consequences because a malicious third party can have access to data involving the dynamics of the physical process, which can be further used in more complex cyberattacks in the future (inference attacks). In discussing deception and disruption attacks, we will make use of the generic networked control structure from Figure 1, where the cyberattacks are at the network level, either on the direct-actuator path ($a_{dp}$) or the feedback-sensing path ($a_{fb}$). In a deception attack the signals send through the network are corrupted, for example through false data injection (this is sometimes referred to as a man in the middle attack—MitM). Such attacks can be modeled as

$$u_n\left[k\right]=u\left[k\right]+a_{dp}\left[k\right],y_n\left[k\right]=y\left[k\right]+a_{fp}\left[k\right],$$

(1)

and represent the strongest attacks from a control perspective, because they have the greatest impact on closed loop stability [18]. In a disruption attack, the signals are either blocked or delayed (e.g., denial of service—DoS), which induces time varying delays or packet loss in the control loop. If we consider packet loss, the attack can be modeled as

$$u_n\left[k\right]=u\left[k\right]a_{dp}\left[k\right],y_n\left[k\right]=y\left[k\right]a_{fp}\left[k\right],$$

(2)

where $a_{dp}\left[k\right]$ and $a_{fp}\left[k\right]$ take the value 0 in case of an attack, and 1 otherwise. When considering a delay attack type, we can use the model

$$\begin{array}{c}\hfill u_n\left[k\right]=u[k-{\tau }_n-{\tau }_{adp}]=u[k-{\tau }_n]+\underset{a_{dp}\left[k\right]}{\underbrace{(-u[k-{\tau }_n]+u[k-{\tau }_n-{\tau }_{adp}])}},\\ \hfill y_n\left[k\right]=y[k-{\tau }_n-{\tau }_{afp}]=y[k-{\tau }_n]+\underset{a_{fb}\left[k\right]}{\underbrace{(-y[k-{\tau }_n]+y[k-{\tau }_n-{\tau }_{afp}])}},\end{array}$$

(3)

where ${\tau }_n$ represents the nominal network transmission delay and $\{{\tau }_{adp},{\tau }_{afp}\}$ the attack induced delays. Note that if the nominal delay is less than the sample period, we can simplify the above model by letting ${\tau }_n=0$. Also, the distinction between the packet loss and the time delay variation case depends on the packet handling strategy at the receiving end: if the delay exceeds a certain threshold, the packet can be considered lost [9].

These three types of attacks are not mutually exclusive; different combinations can be imagined and implemented by an attacker. The damage of the attack depends also on factors like duration (how long are $\{a_{dp},a_{fp}\}$ active), attack profile (profile of the signals $\{a_{dp},a_{fp}\}$), or stealthiness (how long it takes to detect the attack) [19].

Next, we move our discussion towards the defense mechanisms for cyberattacks. We can group them according to the function they serve (prevention, detection or mitigation), or according to the level of intervention (high-level methods, from computer science and information technology, or low-level methods, from control and systems theory).

The high-level solutions for cyberattacks have been adapted from the information technology field [17]. Different cryptographic methods are used to assure the confidentiality, integrity, and authenticity of the data transmitted through the network. The main challenge here is the high computational needs of the algorithms, which makes them unsuitable for an important class of network control systems with energy constraints or with limited computational power and memory. Thus, the research community has started to develop new lightweight methods to meet those constraints, for both symmetric and asymmetric key cryptography. Symmetric key cryptography uses basic logical operations and is inherently more suitable for cyberphysical systems. Several lightweight stream and block ciphers have been designed, like CHACHA, EXPRESSO, Humming Bird, SIMON, or different AES-based versions [20]. Lightweight asymmetric methods have also been the focus of recent research; for example, in [21], the authors propose a key exchange protocol for Industry 4.0 that uses digital certificates based on lightweight Elliptic Curve Qu-Vanstone (ECQV) for trust building and key generation. In [22], a new source anonymous message authentication (SAMA) scheme is presented that allows online/offline computation, making it more versatile for devices with constrained computation power. The implementation of cryptographic methods offers significant protection against MitM and Replay attacks. On the other hand, DoS and distributed denial of service (DDoS) attacks are harder to mitigate using high-level methods, as they overload the computational and network resources that are already limited. Another category of cyber attacks aims to exploit the software and hardware vulnerabilities of the elements composing the network control loop. In these cases, the mitigation solutions also adopt specific techniques from the information technology domain like the use of firewalls and intrusion detection and prevention systems. For example, ref. [23] evaluates the performance of an industrial application-layer firewall, as its use may have an impact on the network delay and jitter and can severely alter the network control performance and stability. The authors develop and validate simple models for the firewall that can be used by engineers for control loop design. Modern methods, using deep packet inspection, have also been proposed to identify malicious traffic and filter attacks. For example, ref. [24] evaluates the mitigation performances of three open source next-generation firewalls (Snort, Suricata, Bro), by addressing known attacks exploiting Modbus/TCP vulnerabilities. Finally, ref. [25] proposes a machine learning method for intrusion detection in industrial control systems (ICS) to cope with unknown attacks, which cannot be detected based on specific signatures. The authors present a detection model that addresses the imbalanced nature of the ICS datasets, by filtering out anomaly traffic from normal one, and identify unknown attacks, by comparing anomaly traffic with known attack traffic. Experimental results show high detection rate of unknown attacks. To sum up, although high-level solutions are available from computer science and information technology, these may not always be suitable or available for networked control systems that have real-time constraints with limited software and hardware capabilities.

Low-level approaches have been the focus of the control community, especially on detection and mitigation, as alternatives with lower resource needs, and which integrate better with the baseline control structure. The most often encountered prevention approach is based on randomization methods [18]. For example, in addressing confidentiality attacks, ref. [26] proposed a randomization approach by switching between different stabilizing control gains from a given set. On detection and mitigation, there are several approaches proposed in the literature, usually specific to different attack types or class of systems [27]. In ref. [28], an event triggering resilience mechanism is proposed for sensor DoS attacks. The event based system, as an alternative to a sample based system, is regarded as being more tolerant to network delays and packet loss, and leads to less communication consumption. The resulting event based controller, together with the DoS attacks modeled as time delays, are tested and validated in simulations. A different approach is proposed in ref. [29] for MitM attacks, called physical authentication or watermarking. The idea refers to injecting a known noisy input to the system, of which the attacker has no information. The effect is expected to be visible at the level of the measured outputs. The approach is tested on a linear discrete plant, with an LQG controller. The watermarking algorithm proposed ensures a trade off between detection and control performances. A robust control approach is explored in ref. [30], where unknown time delays and stochastic malicious attacks are considered. An $H_{\infty }$ controller is designed in terms of linear matrix inequalities (LMIs). Although the results are validated in simulations on simple linear plants, the LMIs could prove to be conservative in real life applications. An interesting idea is given in [31], where the cyber security problem is represented as a two player stochastic game, between the controller and an attacker, which have antagonistic objectives. A model predictive controller which adapts its receding horizon depending on the attack, and calls at each prediction step a particle swarm optimization algorithm, is designed and tested in a formation control simulation scenario. In ref. [32], the authors designed an adaptive neural network for detection of deception attacks. The online tuning of the neural network was performed through a Kalman filter. The technique is validated on UAV cyberattack scenarios.

Most cyberattack mitigation approaches encountered in engineering practice are high-level. However, there are some situations when high-level methods do not work, or they need to be complemented by low-level methods. The cybersecurity of networked control systems with UAVs is an emergent field [27], with relatively few studies in the literature on the cyberattack mitigation of networked control of drones with real life experiments. The most important attacks to consider for networked control of low cost drones are disruption and deception attacks, as these can directly affect the stability and safety of the system. Consequently, in this paper, we will present a new low-level approach to detect and mitigate DoS and MitM attacks, in order to ensure the resilience of the system. Our approach has the advantage that it is relatively straightforward and intuitive to design, requires low online computational power, and proves to be efficient in experiments on small drones.

3. Networked Control for Small Drones

We start from the baseline network control structure from Figure 2. The control structure is split into an inner loop and an outer loop. This relies on the assumption that the inner loop is much faster than the outer loop [2]. The inner loop controller is onboard the drone and controls the attitude (Euler angles—pitch, roll, yaw) and the altitude, based on onboard estimates from a Kalman filter. The outer loop control runs on a remote computer, uses the measurements from a motion capture system, and controls the $XY$ positions. It is important to note that the network is only on the direct path, because the position feedback is performed through the motion capture cameras. The networked control structure has the following benefits: (i) it addresses the drone hardware limitations, by moving a part of the control on a remote computer (outer control loop); (ii) it permits accurate position measurements from a motion capture system like OptiTrack (as alternative to onboard measurements and estimates); and (iii) the transmission delays and packet loss due to wireless transmissions are minimized by sending smaller data packets only one way.

In the next subsections, we describe the mathematical model of the drone, the inner control loop, and the outer control loop.

3.1. Drone Model

We define the pose $P={\left[{\xi }^T{\eta }^T\right]}^T={\left[xyz\varphi \theta \psi \right]}^T$ (see Figure 3), where the position in the world frame is given by $\{x,y,z\}$, and the orientation by the Euler angles $\varphi$ (roll), $\theta$ (pitch), and $\psi$ (yaw). The dynamic model of the drone, developed using the Euler–Lagrange formalism, is given by

$$\left\{\begin{array}{c}\ddot{x}=[c\left(\varphi \right)s\left(\theta \right)c\left(\psi \right)+s\left(\varphi \right)s\left(\psi \right)]{\displaystyle \frac{U_{coll}}{m}}\hfill \\ \ddot{y}=[c\left(\varphi \right)s\left(\theta \right)s\left(\psi \right)-s\left(\varphi \right)c\left(\psi \right)]{\displaystyle \frac{U_{coll}}{m}}\hfill \\ \ddot{z}=-g+c\left(\varphi \right)c\left(\theta \right){\displaystyle \frac{U_{coll}}{m}}\hfill \\ \left[\begin{array}{c}\ddot{\varphi }\\ \ddot{\theta }\\ \ddot{\psi }\end{array}\right]=J^{-1}\left(\eta \right)\left(\left[\begin{array}{c}{U}_{\varphi }\\ U_{\theta }\\ U_{\psi }\end{array}\right]-C(\eta ,\dot{\eta })\dot{\eta }\right)\hfill \end{array}\right.$$

(4)

where $J\left(\eta \right)$ denotes the Jacobian matrix and $C(\eta ,\dot{\eta })$ the Coriolis matrix. The full expressions are not given here, due to space constraints (for more details, see [33]). The control inputs are the torques on the three rotational axes $U_{\varphi }$, $U_{\theta }$, $U_{\psi }$, and the collective force $U_{coll}$. The drone parameters are: mass m, gravitational acceleration g, and inertia moments on the X, Y, and Z axis ($I_x,I_y,I_z$).

The nonlinear model (4) can be written in the state space form as

$$\dot{\mathbf{x}}=\mathbf{f}(\mathbf{x},\mathbf{u}),$$

(5)

with the state vector $\mathbf{x}={\left[xyz\varphi \theta \psi \dot{x}\dot{y}\dot{z}\dot{\varphi }\dot{\theta }\dot{\psi }\right]}^T$ and the input vector $\mathbf{u}={\left[U_{coll}{U}_{\varphi }{U}_{\theta }{U}_{\psi }\right]}^T$. Let the equilibrium point in hovering mode be ${\mathbf{x}}^e={\left[x^e{y}^e{z}^e{0}_{1\times 9}\right]}^T$, with inputs ${\mathbf{u}}^e={\left[U_{coll}^{e}000\right]}^T$. The linearized model can be determined as

$$\left\{\begin{array}{cc}\hfill \ddot{x}& =\theta g\hfill \\ \hfill \ddot{y}& =-\varphi g\hfill \\ \hfill \ddot{z}& ={\displaystyle \frac{\Delta U_{coll}}{m}}\hfill \end{array}\right.\left\{\begin{array}{cc}\hfill \ddot{\varphi }& ={\displaystyle \frac{U_{\varphi }}{I_x}}\hfill \\ \hfill \ddot{\theta }& ={\displaystyle \frac{U_{\theta }}{I_y}}\hfill \\ \hfill \ddot{\psi }& ={\displaystyle \frac{U_{\psi }}{I_z}}\hfill \end{array}\right.$$

(6)

where $\Delta U_{coll}=U_{coll}-mg$. This model can also be written in standard linear state space form as

$${\dot{\mathbf{x}}}_l=A{\mathbf{x}}_l+B{\mathbf{u}}_l$$

(7)

where ${\mathbf{x}}_l=\mathbf{x}-{\mathbf{x}}^e$ and ${\mathbf{u}}_l=\mathbf{u}-{\mathbf{u}}^e$, and the expressions for the matrices A and B are omitted due to space restrictions.

3.2. Inner Control Loop

Here, we design the inner loop for attitude (Euler angles) and altitude (z position) control. We first define the state vector for the inner loop ${\mathbf{x}}_i={[z,\varphi ,\theta ,\psi ,\dot{z},\dot{\varphi },\dot{\theta },\dot{\psi }]}^T$. Based on the linear model (6), the subsystem involving attitude and altitude can be extracted as

$${\dot{\mathbf{x}}}_i=A_i{\mathbf{x}}_i+B_i{\mathbf{u}}_l$$

(8)

A linear state feedback control will be further designed for tracking:

$${\mathbf{u}}_l=-K_i{\mathbf{e}}_i=-K_i({\mathbf{x}}_i-\left[\begin{array}{c}{z}_r\\ {\varphi }_r\\ {\theta }_r\\ {\psi }_r\\ 0_{4\times 1}\end{array}\right]),$$

(9)

where $z_r$ is the altitude reference, which for simplification purposes will be considered constant (in the general case, when the altitude reference is not constant, it would need to be provided through the network). ${\psi }_r$ is the yaw orientation reference, which is considered zero for the purpose of position tracking control (we assume that the drone keeps the same orientation during position tracking). ${\varphi }_r$ and ${\theta }_r$ are virtual control inputs provided by the outer control loop through the network (Figure 2): ${\mathbf{u}}_{oc}^n={\left[{\varphi }_r{\theta }_r\right]}^T$. The feedback gain $K_i$ can be designed using a linear quadratic regulator (LQR) approach [34], which implies minimizing the following objective function:

$$J_i={\int }_0^{\infty }({\mathbf{x}}_i^T{Q}_i{\mathbf{x}}_i+{\mathbf{u}}_l^T{R}_i{\mathbf{u}}_l)dt,$$

(10)

where $Q_i$ and $R_i$ are diagonal matrices, representing state and control inputs weights.

3.3. Outer Control Loop

For the outer control loop of the structure from Figure 2, we consider ideally that $\varphi \to {\varphi }_r$, $\theta \to {\theta }_r$, $\psi \to {\psi }_r=0$ and $z\to z_r=z^e$ (the inner control loop is faster than the outer loop control). Consequently, the inner loop can be modeled in a simplified manner, by also using the first two equations from (6):

$${\dot{\mathbf{x}}}_o=A_o{\mathbf{x}}_o+B_o{\mathbf{u}}_{oc}$$

(11)

where

$${\mathbf{x}}_o=\left[\begin{array}{c}x\\ y\\ \dot{x}\\ \dot{y}\end{array}\right],A_o=\left[\begin{array}{c}0010\\ 0001\\ 0000\\ 0000\end{array}\right],B_o=\left[\begin{array}{c}00\\ 00\\ 0g\\ -g0\end{array}\right],{\mathbf{u}}_{oc}=\left[\begin{array}{c}{\varphi }_r\\ {\theta }_r\end{array}\right].$$

At this point, we consider that there are no delays or packet loss: ${\mathbf{u}}_{oc}={\mathbf{u}}_{oc}^n$. We further design a state feedback controller for this linear state space model,

$${\mathbf{u}}_{oc}=-K_o{\mathbf{e}}_o=-K_o({\mathbf{x}}_o-\left[\begin{array}{c}{x}_r\\ y_r\\ 0_{2\times 1}\end{array}\right]).$$

(12)

The gain $K_o$ can again be designed for tracking using an LQR approach, i.e., by minimizing the objective function

$$J_o={\int }_0^{\infty }({\mathbf{x}}_o^T{Q}_o{\mathbf{x}}_o+{\mathbf{u}}_{oc}^T{R}_o{\mathbf{u}}_{oc})dt,$$

(13)

where $Q_o$ and $R_o$ are state and control inputs weights. $x_r$ and $y_r$ are the reference positions in the horizontal plane at a given altitude.

4. Resilience to Cyber Attacks

In this paper, we propose the networked control structure resilient to cyberattacks from Figure 4. As discussed in Section 2, we mainly consider two types of cyberattacks: DoS and MitM. We interpret the DoS attack as an increase in the network time delay $\tau$. If the delay increases over a predefined threshold ${\tau }_{max}$, we consider that the communication is interrupted. Thus, we define a manageable DoS attack when $\tau \left(t\right)<{\tau }_{max}$, and a major attack when $\tau \left(t\right)>{\tau }_{max}$. We interpret the MitM attack as an additive disturbance on the command sent through the network. If the disturbance is slow-varying or piece-wise, with amplitude smaller than the saturation limit $|u_{oc}^n\left(t\right)|<u_{max}$, we define the attack to be sneaky, but manageable. If the disturbance is fast varying, and pushes the command $u_{oc}^n\left(t\right)$ in saturation, we consider again that in this case the communication is interrupted (major attack).

For major cyberattacks, either DoS when $\tau \left(t\right)>{\tau }_{max}$ or MitM when $|u_{oc}^n\left(t\right)|>u_{max}$, for a number of consecutive sample instances greater than a predefined threshold $k_a$, we consider the communication as interrupted. In this case, the practical solution is to activate a back-up position controller onboard the drone (not shown in Figure 4), that stabilizes the drone in hovering, and eventually ensure a safe landing procedure. This gives time for higher level security responses to the cyberattack.

For manageable cyberattacks, our solution consists of using a combination between a RG and an UIDO. The RG changes the reference signals such that the systems remains in a safe region (the states satisfy some predefined safety constraints) [14], while the UIDO estimates unknown inputs and helps the controller to compensate their effect [35]. We consider also that the states ${\mathbf{x}}_o$ are available: the positions are directly accessible from OptiTrack measurements, while the velocities we can calculate through numerical differentiation.

The next subsections will present different designs for this approach, depending on the attack type—DoS or MitM. Although the attacks are treated separately for reasons of simplicity and clarity, the control structure from Figure 4 can be designed to be resilient to both types of attacks.

4.1. Reference Governor for DoS Attack

Here, we consider the DoS (manageable) attack as a change in time-varying network transmission delay that affects the control signal from Figure 4:

$${\mathbf{u}}_{oc}^n\left[k\right]={\mathbf{u}}_{oc}[k-{\tau }_d],$$

(14)

where ${\tau }_d$ is the discrete delay, considered as a multiple of the sampling period $T_s^o$ (${\tau }_d={\displaystyle \frac{\tau }{T_s^o}}$). We assume the baseline outer loop controller can handle some delay variation, such that the systems remains stable (if this does not hold, one can design an additional time delay compensator for a nominal/average value of the delay).

In addressing the resilience to a DoS attack based on the approach from Figure 4, we will restrict our attention only to the X axis movement, since our outer loop model assumes that the movement on the X and Y axes are decoupled. The whole discussion can also be adapted easily to the Y axis. We first extract from (11) just the part related to the X axis movement:

$$\begin{array}{cc}\hfill {\dot{\mathbf{x}}}_{ox}& =A_{ox}{\mathbf{x}}_{ox}+B_{ox}{u}_{ocx}\hfill \\ \hfill y_x& =C_{ox}{\mathbf{x}}_{ox}\hfill \end{array}$$

(15)

where

$${\mathbf{x}}_{ox}=\left[\begin{array}{c}x\\ \dot{x}\end{array}\right],A_{ox}=\left[\begin{array}{c}01\\ 00\end{array}\right],B_{ox}=\left[\begin{array}{c}0\\ g\end{array}\right],u_{ocx}={\theta }_r,C_{ox}=\left[\begin{array}{cc}1& 0\end{array}\right].$$

Because the RG framework is usually in discrete-time, we discretize (15) through the zero order hold method [34], and add the delay model (14):

$$\begin{array}{cc}\hfill {\mathbf{x}}_{ox}[k+1]& =A_{oxd}{\mathbf{x}}_{ox}\left[k\right]+B_{oxd}{u}_{ocx}[k-{\tau }_d]\hfill \\ \hfill y_x\left[k\right]& =C_{oxd}{\mathbf{x}}_{ox}\left[k\right],\hfill \end{array}$$

(16)

where k the index of the current sampling instance. The control law for the X axis is

$$u_{ocx}\left[k\right]=-K_o^x({\mathbf{x}}_{ox}\left[k\right]-{\mathbf{x}}_r\left[k\right]),$$

(17)

with ${\mathbf{x}}_r\left[k\right]={\left[x_r\left[k\right]0\right]}^T$. As indicated in ref. [36], we can define ${\tau }_d$ additional states ${\mathbf{x}}_i^d\left[k\right]=x[k-({\tau }_d-(i-1))]-{\mathbf{x}}_r[k-({\tau }_d-(i-1))]$ (where $i=1,\dots ,{\tau }_d$), such that closed loop system can be written as

$$\begin{array}{cc}\hfill x_{eox}[k+1]& =A_{eox}{\mathbf{x}}_{eox}\left[k\right]+B_{eox}{\mathbf{x}}_r\left[k\right]\hfill \\ \hfill y_{ex}\left[k\right]& =C_{eox}{\mathbf{x}}_{oex}\left[k\right],\hfill \end{array}$$

(18)

with the extended state vector ${\mathbf{x}}_{eox}={[{\mathbf{x}}_{ox}{\left[k\right]}^T{\mathbf{x}}_1^d{\left[k\right]}^T\dots {\mathbf{x}}_{{\tau }_d}^d{\left[k\right]}^T]}^T$ and

$$A_{eox}=\left[\begin{array}{ccccc}{A}_{oxd}& -B_{oxd}{K}_o^x& 0_2& \dots & 0_2\\ 0_2& 0_2& I_2& \dots & 0\\ ⋮& ⋮& ⋮& \dots & ⋮& \\ 0_2& 0_2& 0_2& \dots & I_2\\ I_2& 0_2& 0_2& \dots & 0\end{array}\right],B_{eox}=\left[\begin{array}{c}{0}_2\\ ⋮\\ -I_2\end{array}\right],C_{eox}=\left[\begin{array}{cccc}{C}_{oxd}& 0_2& \dots & 0_2\end{array}\right].$$

We further design the RG for (18) considering that all of the states are accessible (a buffer can be used to store the delayed state values). Instead of applying directly the reference $x_r$, we will use the RG to calculate at each time instance the virtual reference $v_x$ which, if held constant from the current time onward, will make the output satisfy some safety constraint. With this idea in mind, we define the polytopic constraint for the output

$$y_{ex}\left[k\right]\in {\mathbf{Y}}_s:=\{y_{ex}:S{y}_{ex}\le s\},$$

(19)

where s is the safety limit, while S provides additional freedom for linear combinations of outputs. The maximal admissible set (MAS) is defined as the set of all initial states and constant inputs, such that the outputs constraint (19) is satisfied for all future times [14]:

$$O_{\infty }:=\{({\mathbf{x}}_{eox}^0,v_x^0)\in {\mathbb{R}}^{3+2{\tau }_d}:{\mathbf{x}}_{eox}\left[0\right]={\mathbf{x}}_{eox}^0,v_x\left[k\right]=v_x^0,y_{ex}\left[k\right]\in {\mathbf{Y}}_s,\forall k\in {\mathbb{Z}}_{+}\}.$$

(20)

Next, the output $y_{ex}$ can be calculated explicitly depending on the initial conditions ${\mathbf{x}}_{eox}^0$ and constant input $v_x^0$:

$$y_{ex}\left[k\right]=C_{eox}{A}_{eox}^k{\mathbf{x}}_{eox}^0+C_{eox}{(I-A_{eox})}^{-1}(I-A_{eox}^k)B_{eox}{v}_x^0$$

(21)

Consequently, the MAS (20) can be defined by an infinite number of inequalities:

$$O_{\infty }:=\{({\mathbf{x}}_{eox}^0,v_x^0)\in {\mathbb{R}}^{3+2{\tau }_d}:H_x{\mathbf{x}}_{eox}^0+H_v{v}_x^0\le h\},$$

(22)

with $H_x=\left[C_{eox}{A}_{eox}^k\right]$, $H_v=\left[C_{eox}{(I-A_{eox})}^{-1}(I-A_{eox}^k)B_{eox}\right]$, $h={[s^T{s}^T\dots ]}^T$. However, to make the computation tractable, we want the matrices $H_x$, $H_v$ and h to be finite dimensional. It is proved in ref. [37] that we can always calculate an inner approximation of $O_{\infty }$ through the use of the steady state value of $y_{ex}$:

$$y_{ex}^{ss}:=\left(C_{eox}{(I-A_{eox})}^{-1}{B}_{eox}\right)v_x^0\in (1-ϵ){\mathbf{Y}}_s,$$

(23)

where $0<ϵ\ll 1$ is a steady state margin. Thus, by introducing (23) in (22), there always exists a finite prediction time $k^{*}$ for which all future time steps ($k>k^{*}$) make the inequalities become redundant [14]. This gives us the finitely determined inner approximation of $O_{\infty }$, denoted further as ${\tilde{O}}_{\infty }$.

The delay ${\tau }_d$ is actually time varying in network transmissions. Here, we assume that the delay is piecewise constant (when this does not hold in practice, we can always adopt an averaging approach in order to eliminate jitters), and that it can take values in the set $D_{{\tau }_d}=\{1,2,3,\dots ,{\tau }_d^{max}\}$. Through a so-called time stamping technique, we can actually determine online an estimation of the time delay variation [9]. Because the MAS can be calculated offline, we can calculate a priori several MASs for (18) for each delay values from $D_{{\tau }_d}$. Then, we just have to switch online between the different MASs corresponding to the current estimated delay ${\tau }_d$.

We can now introduce the scalar RG that calculates online the virtual reference $v_x$, based on the equation

$$v_x\left[k\right]=v_x[k-1]+k(x_r\left[k\right]-v[k-1]),$$

(24)

where k is determined by solving the following linear program:

$$\begin{array}{cccc}\hfill & \underset{k\in [0,1]}{\mathrm{maximize}}\hfill & \hfill & k\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\hfill & \hfill & ({\mathbf{x}}_{eox}\left[k\right],v_x\left[k\right])\in {\tilde{O}}_{\infty },\hfill \\ \hfill & \hfill & & v_x\left[k\right]=v_x[k-1]+k(x_r\left[k\right]-v[k-1]).\hfill \end{array}$$

(25)

Note that when $k=0$, we obtain $v_x\left[k\right]=v_x[k-1]$. This means that, in order to keep the system safe, we can not change the reference for the current time moment. When there are no violations of the constraint, we obtain $k=1$ and thus $v_x\left[k\right]=x_r\left[k\right]$.

To sum up, in case of a DoS attack, we assume step changes in ${\tau }_d$. This change can be estimated online, and the MAS is switched accordingly, such that the RG can predict possible safety constraint violations and adapt the virtual reference sent to the system.

4.2. Unknown Input Disturbance Observer and Reference Governor for MitM Attack

We assume that the MitM (manageable) attack ca be modeled as an additive slow-varying or piece-wise disturbance

$${\mathbf{u}}_{oc}^n\left[k\right]={\mathbf{u}}_{oc}\left[k\right]+\mathbf{d}\left[k\right].$$

(26)

As in the previous subsection, the resilience to a MitM attack, based on the approach from Figure 4, is discussed only for X axis movement (but this can also be easily adapted to the Y axis). The zero order hold discretization of (15) along with this new input can be written as

$$\begin{array}{cc}\hfill {\mathbf{x}}_{ox}[k+1]& =A_{oxd}{\mathbf{x}}_{ox}\left[k\right]+B_{oxd}{u}_{ocx}\left[k\right]+B_{oxd}{d}_x\left[k\right]\hfill \\ \hfill y_x\left[k\right]& =C_{oxd}{\mathbf{x}}_{ox}\left[k\right],\hfill \end{array}$$

(27)

First, we want to design the UIDO such that we can estimate the disturbance (i.e., to detect the attack). Considering the disturbance model $d[k+1]=d\left[k\right]$, the observer can be designed based on (27) extended with this additional state [38]:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{x}}}_{dox}[k+1]& =A_{dox}{\widehat{\mathbf{x}}}_{dox}\left[k\right]+B_{dox}{u}_{ocx}\left[k\right]+L_x(y_x\left[k\right]-C_{dox}{\widehat{\mathbf{x}}}_{dox}\left[k\right])\hfill \\ \hfill {\widehat{d}}_x\left[k\right]& =C_{dod}{\widehat{\mathbf{x}}}_{dox}\left[k\right],\hfill \end{array}$$

(28)

with

$${\widehat{\mathbf{x}}}_{dox}=\left[\begin{array}{c}{\widehat{\mathbf{x}}}_{ox}\\ {\widehat{d}}_x\end{array}\right],A_{dox}=\left[\begin{array}{cc}{A}_{oxd}& B_{oxd}\\ 0_{1\times 2}& 1\end{array}\right],B_{dox}=\left[\begin{array}{c}{B}_{oxd}\\ 0\end{array}\right],C_{dox}=\left[\begin{array}{cc}{C}_{oxd}& 0\end{array}\right],C_{dod}=\left[\begin{array}{ccc}0& 0& 1\end{array}\right].$$

The observer gain $L_x$ can be determined through pole placement. The control law (17) for the X axis can now be extended with a disturbance compensation term,

$$u_{ocx}\left[k\right]=-K_o^x({\mathbf{x}}_{ox}\left[k\right]-{\mathbf{x}}_r\left[k\right])-{\widehat{d}}_x\left[k\right].$$

(29)

Through this approach, the disturbance can be compensated completely at least in steady state, and keep the system stable, despite the attack. However, due to the convergence time needed by the observer, we can still have transients that can violate our safety constraints. Thus, we proceed by adding a RG on top of the closed loop system formed by coupling (27) and (28) with (29):

$$\begin{array}{cc}\hfill {\mathbf{x}}_{eox}[k+1]& =A_{eox}{\mathbf{x}}_{eox}\left[k\right]+B_{eox}{\mathbf{x}}_r\left[k\right]+B_{eoxd}{d}_x\left[k\right]\hfill \\ \hfill y_{ex}\left[k\right]& =C_{eox}{\mathbf{x}}_{oex}\left[k\right].\hfill \end{array}$$

(30)

The state vector for the closed loop system is ${\mathbf{x}}_{eox}={\left[{\mathbf{x}}_{ox}^T{\widehat{\mathbf{x}}}_{dox}^T\right]}^T$.

Now, we have to adapt the RG design presented in the previous subsection for the case of an input disturbance $d_x$. Because $d_x$ is not accessible for measurements, when adopting model (30) for the RG, we will actually use an upper bound of the estimated disturbance ${\widehat{d}}_x$. We first split the disturbance set into several intervals,

$$\{(-d_m,-d_{m-1}],\dots ,(-d_2,-d_1],(-d_1,-d_0],(-d_0,d_0),[d_0,d_1),\dots ,[d_{m-1},d_m)\}.$$

The number of intervals is limited by the maximum disturbance $d_m$ which, in our case, is given by the command saturation limit $d_m=u_{max}$. Whenever the estimated disturbance is in one of these intervals, the RG will use the upper bound of the absolute value (e.g., for $[d_1,d_2)$, the bound is $d_2$, and for $[-d_3,d_2)$, the bound is $-d_3$). The only exception is the middle interval $(-d_0,d_0)$, for which we impose the “bound” 0 for practical reasons (in practice the UIDO can estimate additional effects like model uncertainty, noise, or other unmodeled disturbances). Thus, we will work with piece-wise disturbance bounds, switched depending on the disturbance online estimate, which are fed to the RG.

Let us denote the discrete disturbance bound as ${\overline{d}}_x$, which can take values in the set $D_d=\{-d_m,\dots ,-d_1,0,d_1,\dots ,d_m\}$. The MAS for constraint (19) can be recalculated (without the delay) as

$$O_{\infty }:=\{({\mathbf{x}}_{eox}^0,v_x^0,{\overline{d}}_x^0)\in {\mathbb{R}}^6:H_x{\mathbf{x}}_{eox}^0+H_v{v}_x^0+H_d{\overline{d}}_x^0)\le h\},$$

(31)

based on the output response with

$$\begin{array}{cc}\hfill y_{ex}\left[k\right]& =C_{eox}{A}_{eox}^k{\mathbf{x}}_{eox}^0+C_{eox}{(I-A_{eox})}^{-1}(I-A_{eox}^k)B_{eox}{v}_x^0\hfill \\ \hfill & +C_{eox}{(I-A_{eox})}^{-1}(I-A_{eox}^k)B_{eoxd}{\overline{d}}_x^0.\hfill \end{array}$$

(32)

Because we assume that, in a steady state, the disturbance is completely compensated (i.e., the disturbance vanishes in steady state), for the finite inner approximation ${\tilde{O}}_{\infty }$ we will use the same steady state condition (23). Several MASs can be computed offline for each disturbance bound from $D_d$. We can further use the same RG online as the one defined by (24), but now the MASs are switched depending on corresponding bound ${\overline{d}}_x$ of the estimated disturbance.

Thus, in case of a MitM attack, we assume step changes in $\mathbf{d}$. The disturbance is estimated and compensated in steady state through UIDO and controller, while the transients that can cause safety constraints violations are taken into account by the RG, which changes the virtual reference accordingly.

5. Experiments

This section will present some experimental results for the two cyberattack scenarios discussed throughout the paper: DoS and MitM. The experimental setup is given in Figure 5. The Parrot Mambo drone (Figure 3) communicates wireless via Bluetooth with a remote computer (Intel i7 processor, 16GB RAM, Windows 11). The Parrot Mambo drone has $0.18\times 0.18$ m, weights only 0.063 kg, and the propellers are actuated by four DC motors. As onboard sensors, the drone has an accelerometer, a gyroscope, an ultrasound sensor, a barometer, and a temperature sensor. The software interface with the sensors, actuators and network communication is made possible due to a firmware specifically developed for Matlab [39]. An OptiTrack motion camera system with four cameras is connected to the remote computer. As software running on the computer, we used MATLAB and Simulink Real Time for networked control, and the OptiTrack Motive 1.8.0 software for data capturing, which communicates with Simulink through a middleware client-server application.

The parameters of the drone are given in

Table 1. Drone parameters.

Parameter	Notation	Value	Units
Mass	m	0.063	kg
X-axis inertia moment	$I_x$	$0.5829·{10}^{-4}$	kg m2
Y-axis inertia moment	$I_y$	$0.7169·{10}^{-4}$	kg m2
Z-axis inertia moment	$I_z$	$1.000·{10}^{-4}$	kg m2
Gravitational acceleration	g	9.8	m/s2

. In the control structure from Figure 2, we first design the baseline inner loop and outer loop controllers. The following tuning matrices were adopted for the two LQRs:

$$\begin{array}{cc}\hfill Q_i& =diag\left(\right[10005005001100101010\left]\right)·0.001,\hfill \\ \hfill R_i& =diag\left(\right[10/1510000100001000\left]\right),\hfill \\ \hfill Q_o& =diag\left(\right[100010002020\left]\right),\hfill \\ \hfill R_o& =diag\left(\right[50005000\left]\right).\hfill \end{array}$$

The weights for $Q_i$ and $Q_o$ were designed empirically through experiments, considering position weights larger than the velocity weights (it is more important for position to converge faster to equilibrium). The weights $R_i$ and $R_o$ were chosen such that, during experiments, under nominal conditions (when there are no cyberattacks), the control signals do not reach saturation. Through this design, we obtained in nominal operating conditions (i.e., when there is no cyberattack), an overshoot of less than $10\%$, a settling time of less than 2 s, and a maximum steady state error of $0.1$ m. For the inner loop control, the sampling period is limited by the hardware to $T_s^i=5$ ms, while for the outer loop, the sampling period is limited by the Bluetooth transmissions and drone hardware. Our experiments revealed that a sampling period too small leads to a large amount of packet loss and large time-varying transmission delays of several hundreds milliseconds (this was also confirmed in [40]). On the other hand, a large sampling period makes the control miss important transient behavior of the real drone, with control performance deteriorating significantly. After multiple experiments with the Parrot Mambo drone, we arrived empirically at a compromise solution for the outer loop sampling period of $T_s^o=30$ ms. The outer loop command saturation values are chosen as $u_{max}=0.5$ rad. For the inner loop, the states are estimated using a steady-state linear Kalman filter. For the outer loop, the states are obtained from the OptiTrack measurements.

5.1. DoS Attack

We adopt the maximum number of samples for the delay ${\tau }_d^{max}=10$ (i.e., ${\tau }_{max}=0.3$ s), and then calculate offline several MASs for different possible transmissions delays, from 1 to ${\tau }_d^{max}$. The delay bound was adopted considering maximum network latency values under high channel load, receiving buffer size, but also the stability margin of the system. A scalar RG is designed and implemented for X axis, and one for Y axis, according to the methodology from Section 4.1. For the steady state approximation of the MASs, we adopt $ϵ=0.001$, such that we obtain a good enough approximation with reasonable computational effort. The time delay is estimated online using time stamps. As the time delay changes, due to communication disturbances or cyberattacks, the MAS is switched accordingly.

We emulate a DoS attack by considering a sudden increase in the network transmission delay. Several experiments were conducted for different delay values. Here, for illustrative purposes, we will show two representative DoS attacks: one for a delay of six samples (DoS attack 1) and one for a delay of eight samples (DoS attack 2).

For DoS attack 1, we increase the network transmission delay from one sample to six samples, at time moment $t=23$ s. In this scenario, the drone moves on the X axis, tracking multiple step references (equivalent results could have been shown for the Y axis). The constraint on X axis is $s=1$, i.e., the safe space on X is $[-1,+1]$ m. Figure 6 shows the results using the baseline network control structure without the RG. The first 10 s are the take off initialization phase. The tracking controller exhibits some steady state error due to modeling uncertainties, but we consider the control performances satisfactory for the purpose of this paper—i.e., resilience to cyberattacks. It can be noticed that due to the increase in the network transmission delay (DoS attack), the step reference at time $t=25$ s induces oscillations that violate the constraint (dashed green horizontal line). This can be seen more clearly in the zoomed version from Figure 7. Figure 8 shows that when using the RG approach proposed in this paper (see the control structure from Figure 4), the safety constraints are no longer violated, thus ensuring resilience to this type of cyber attack. This is performed through the fact that the RG anticipates the safety constraint violation based on the model and the time delay information, and as response it increases the reference more gradually (slowly), as shown in Figure 9, thus reducing the oscillations. The reference is first increased to a value of about $0.86$ m, and it gradually reaches the desired reference value after $0.5$ s.

DoS attack 2 implies a delay of eight samples, with the same scenario described previously. When we use the baseline control structure, the safety constraint are violated as before. If we add the RG for low-level cyberattack mitigation, we obtain the result from Figure 10, where we can see that response is with damped oscillations, and it does not violate the safety constraint. The RG modifies the reference as shown more clearly in the zoomed version from Figure 11: the reference is first increased to a value of about $0.82$ m, and only after $0.85$ s it reaches the desired reference value.

So, the RG changes the references more gradually depending on the delay induced by the DoS attack such that the safety constraints are enforced. The safety constraints are important because they could represent an obstacle or a wall in real life applications.

5.2. MitM Attack

We emulate the MitM attack by an additive piecewise disturbance on the control signal. The disturbance input is detected by the UIDO, and then compensated in steady state. The gain of the UIDO was designed through pole placement: $L_x={\left[0.23760.62590.0559\right]}^T$. A RG is designed to account for the transient regime during compensation. Several MASs are computed offline for different intervals of the disturbance: $[-0.5,-0.3]$, $[-0.3,-0.1]$, $[-0.1,0.1]$, $[0.1,0.3]$, $[0.3,0.5]$. For each interval, the worst case values are actually computed for the MAS: e.g., a disturbance in the interval $[0.1,0.3]$ is adopted as $0.3$. Because the UIDO estimates also the effect of model uncertainties and network delay effect, the interval $[-0.1,0.1]$ is neglected, i.e., the MAS is computed for a zero disturbance. The RG switches the MAS according to the estimated disturbance. We assume the disturbance is slow varying or piecewise constant. Because the saturation limit is $\pm 0.5$, the disturbance can not be larger than this limit.

We consider a scenario when the drone follows step reference signals on the X axis (an equivalent scenario could have been shown on the Y axis). Several experiments were conducted for different disturbance amplitudes and timings. Here, for illustrative purposes, we will show two representative MitM attacks: one of disturbance amplitude 0.3 (MitM attack 1) and one of amplitude 0.4 (MitM attack 2).

The MitM attack 1 consists of a step disturbance of amplitude $0.3$. Figure 12 shows the effect of the attack (disturbance) on the baseline control, when there is no UIDO or RG. The vertical dashed green line indicates the onset of the attack. Although the control manages to stabilize the drone, the steady state offset is very large, and there is no way of controlling the drone such that it remains in a safe region.

Figure 13 shows the systems response to the attack when we add the UIDO. The disturbance effect is compensated in steady state, but the transient regime, due to the observer convergence time, may still cause safety problems. The problem with the transient regime can not be eliminated by further tuning the observer gains (increasing the gains further actually induces oscillations in the control loop). Figure 14 illustrates the scenario when the reference changes after the onset of the attack, which causes damped oscillations in the systems response. Due to these oscillations, the drone violates the safety constraint shown with dashed green horizontal line. This can be seen more clearly in the zoomed version (Figure 15). This shows that using just an UIDO can not always ensure safety in case of MitM cyberattacks, due to the observer convergence time.

When we add also the RG, which uses the disturbance estimate provided by the UIDO, and thus use the control structure from Figure 4, we obtain the results from Figure 16. The RG anticipates the transient regime due to the UIDO, and increases the reference in slower steps (first to a value of about $0.57$ m, and it gradually reaches the desired value after $0.17$ s), thus reducing the oscilations. The fact that the the safety constraints are now enforced by the combination of RG and UIDO can be seen more clearly in the zoom version from Figure 17.

The MitM attack 2, when the disturbance has amplitude 0.4, also violates the safety constraints when we use only the baseline control structure. When we adopt our low-level mitigation approach, with UIDO and RG, we obtain the results from Figure 18. As seen also in the zoomed version from Figure 19, the RG now changes more drastically the reference by keeping it constant for a few samples, and then increasing it slowly to its desired values (reaching the desired value after $0.34$ s), in such a manner that the safety constraints are not violated.

So, one can conclude that the RG modifies the reference slower, depending on the severity of the MitM attack and possible safety violations, and may even keep the reference constant (i.e., keep the previous value) for several sample instances. Although in above presented scenarios the RG still managed to increase the reference to its final desired value, in case of major cyber attacks, depending on the scenario, the RG may choose to keep the reference constant (i.e., hold the previous reference value), until the attack passes or it is mitigated at a higher level.

6. Conclusions

In this paper, we have proposed a new resilient networked control architecture for small drones. Because of the sensing and computational limitations of such drones, the control structure was split in an inner loop and an outer loop. Our focus was on the outer loop which controls the positions in the horizontal plane. We considered the case when the network is only on the direct path, while the sensing needed for the feedback path is provided by OptiTrack cameras connected to a remote computer. Two types of cyberattacks were considered: DoS, emulated as a switch in the network transmission delay, and MitM, emulated as additive piecewise input disturbances. The DoS attack, i.e., the change in the network time delay, can be detected through time stamping.

The main contribution of the study consists of the low level mitigation approach to these attack types, with application to low cost small UAVs, based on the structure from Figure 4. The mitigation for the DoS attack implies adopting a RG that uses the delay estimate and the system’s model to predict future safety violations, and adapts the reference accordingly. The MitM attack is detected by an UIDO. The attack is rejected in steady state through a disturbance compensation loop, while the possible transients that can cause safety violations are predicted by a RG, and the reference is again adapted in concordance. We show real time experimental validations of both approaches on a Parrot Mambo drone.

The results with the DoS attack illustrate that our approach ensures stability and safety for network transmission delays of up to 0.3 s. For the MitM attack, the attack bound is given by the saturation limit, which in our case was 0.5 rad. The experimental results illustrate that the safety constraints are enforced despite this type of attacks. In both attack cases, our approach implies to temporally sacrifice reference tracking in case of cyberattacks, in order to preserve stability and safety. The low-level mitigation approach with RG and UIDO is independent of the controller, so it can be augmented on other drones with different baseline controllers. In case of major cyberattacks, when these bounds are violated for a number of consecutive samples, we considered that we actually have an interruption in the network transmissions, and the drone switches to a back-up onboard position controller that ensures safe landing.

As a future work, we will focus on increasing the control robustness to model uncertainty, and extend the whole approach to cooperative resilient control of multiple drones.