Next Article in Journal
Piezoresistive Porous Composites with Triply Periodic Minimal Surface Structures Prepared by Self-Resistance Electric Heating and 3D Printing
Next Article in Special Issue
VA-LOAM: Visual Assist LiDAR Odometry and Mapping for Accurate Autonomous Navigation
Previous Article in Journal
Feedback Beamforming in the Time Domain
Previous Article in Special Issue
Constrained Cubature Particle Filter for Vehicle Navigation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment

Department of Research and Multidisciplinary Studies, Program of Aerial and Submarine Autonomous Navigation Systems, Center for Research and Advanced Studies of the National Polytechnic Institute, Mexico City 07360, Mexico
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Sensors 2024, 24(7), 2183; https://doi.org/10.3390/s24072183
Submission received: 26 February 2024 / Revised: 27 March 2024 / Accepted: 27 March 2024 / Published: 28 March 2024

Abstract

:
This paper proposes a new sensor using optical flow to stabilize a quadrotor when a GPS signal is not available. Normally, optical flow varies with the attitude of the aerial vehicle. This produces positive feedback on the attitude control that destabilizes the orientation of the vehicle. To avoid this, we propose a novel sensor using an optical flow camera with a 6DoF IMU (Inertial Measurement Unit) mounted on a two-axis anti-shake stabilizer mobile aerial gimbal. We also propose a robust algorithm based on Sliding Mode Control for stabilizing the optical flow sensor downwards independently of the aerial vehicle attitude. This method improves the estimation of the position and velocity of the quadrotor. We present experimental results to show the performance of the proposed sensor and algorithms.

1. Introduction

Research around Unmanned Aerial Vehicles (UAVs) has increased in recent decades due to their capacity for experimentation in different disciplines, low cost, and ease of repair compared to classical helicopter configurations. Sensor improvements and robust control algorithm developments allow for accuracy tasks in vehicles of different configurations such as fixed-wing aircrafts, rotorcrafts, and hybrids. Sensor-based navigation approaches are developed in specific environments and conditions for civil, military, or scientific applications.
In the early stages of development, optical flow-based navigation was limited by the range and field of view of the sensors. The frame-fixed optical flow module uses a downward-facing camera and a distance sensor for velocity estimation. Optical flow detection is affected due to angle changes caused by UAV maneuvers. In hover flights the maneuvers are reduced, and the velocity estimation is reliable.
Motion capture can be used for high-precision indoor flight. In [1], the position of the multirotor aircraft is estimated by a set of cameras in a room; the tracking of the movement of the UAV was controlled offline to follow a ground vehicle, but the cost of the motion capture system is highlighted. The works [2,3] present open-air flights using GPS in precise long-range missions, analyze the sensitivity of GPS to interference, and propose a return-to-home flight mode using the heading angle to home. In [4], vision-based navigation is used for indoor and outdoor flights with high-cost computational requirements and lighting changes sensitivity. Moreover, navigation using optical flow and distance sensors requires low-cost computational and provides good accuracy in local environments.
In [5], the authors present an eight-rotor UAV using optical flow, and this rotorcraft uses the main four rotors for altitude and attitude displacement and another four rotors for horizontal maneuvers. This prototype uses the optical flow for estimating the horizontal position avoiding angular tilt. In [6] in the authors proposed improved motion compensation using feature block selection, look-ahead rotation, fault case detection and filtering using PX4FLOW hardware by tunning the camera resolution, the interval of adaptive boxes and advanced search algorithm. In [7] presents a UAV position estimation using an optical flow approach using a Gated Recurrent Unit (GRU) network-based pointing angles and Magnetic, Angular Rate, and Gravity (MARG) sensors improving robustness and performance in real-time experimentation.
Optical flow sensors used for position control in quadrotor aircraft are facing downwards and fixed to the airframe. In a flight based on the optical flow sensor, the roll and pitch maneuvers are restricted to ± 15 , while maneuvers in flights without optical flow sensors are up to ± 35 . The bounded angles reduce the horizontal displacement velocity of the vehicle. Quadrotor rotorcraft in an ideal hover flight would maintain the Euler angles of pitch and roll aligned to the horizontal and with a linear displacement velocity near zero. However, in practice (even in hover flight), a quadrotor rotorcraft vehicle is sensitive to disturbances and the responses of the attitude controller can be considered as optical noise. Furthermore, it is well known that tilting an optical flow module produces positive feedback on vertical position estimates and linear velocity estimates.
The rotorcraft in Figure 1a shows an optical flow module mounted directly on the quadrotor’s frame. When the quadrotor tilts it causes a distortion in the estimation of the altitude and horizontal velocity. Figure 1b shows an optical flow module stabilized by a gimbal. The gimbal feedback control stabilizes pitch and roll angles on the horizontal plane, providing reliable altitude and horizontal velocity estimations with respect to the ground and avoiding optical distortions.
The main contribution of the present paper is improving the optical flow sensing to obtain a reliable set of image data using a gimbal that in general is used to stabilize camera videos for recording or First-Person View (FPV) flights. The optical flow sensor is improved for trajectory control in GPS-Denied environments. This approach allows for flight maneuvers with large changes in the attitude angles; in other cases this produces positive feedback that destabilizes the orientation of the vehicle. Furthermore, we use a robust control technique to enable hover flights and trajectory following with tracking velocity without affecting the sensed images.
The paper is organized as follows: Section 2 describes the system mathematical models. Section 3 is devoted to the control algorithms. Materials and Methods are given in Section 4. The experimental results are shown in Section 5. A brief discussion about our approach is presented in Section 6. Final remarks are given in the conclusions.

2. Mathematical Models

Mathematical models are approximations of the real systems. The subsystems include the dynamics, estimations, control inputs, and degrees of freedom. In the quadrotor aircraft, the forces and angular moments are induced by four actuators to move in the inertial reference frame ( x , y , z ) and rotate in their Euler angles ( ϕ , θ , ψ ) . In the gimbal, the angular moments are induced by its three actuators that allow it to tilt around its Euler angles ( ϕ g , θ g , ψ g ) .
The optical flow module estimates the lateral and longitudinal displacement velocity and altitude of the complete system ( x ˙ , y ˙ , z ) . The following subsections present the mathematical models of the quadrotor aircraft, gimbal, and optical flow module to describe the complete system.

2.1. Quadrotor Aircraft Model

Figure 2 shows the representation of the quadrotor aircraft. Let us define the total thrust u as follows:
u = ( f 1 + f 2 + f 3 + f 4 )
where f i = k ω i 2 is the force produced by i-rotor where i = 1 , 2 , 3 , 4 . k represents a set of aerodynamic constants, and ω i is the angular velocity. We obtain the Euler angles rates as
ϕ ˙ θ ˙ ψ ˙ = T p q r
where
T = 1 sin ϕ tan θ cos ϕ tan θ 0 cos ϕ sin ϕ 0 sin ϕ sec θ cos ϕ sec θ
The rotor torque τ ( ϕ , θ , ψ ) is defined as
τ ϕ = ( f 1 f 3 ) l τ θ = ( f 2 f 4 ) l τ ψ = i = 1 4 τ i
where l is the distance from the rotors to the center of the gravity and τ i is the torque produced by the i-rotor. Therefore, according to [8] the quadrotor aircraft model can be defined as
m x ¨ = u sin θ m y ¨ = u cos θ sin ϕ m z ¨ = u cos θ cos ϕ m g I η ¨ + C ( η , η ˙ ) η ˙ = τ ˜
where m is the mass, g is the acceleration due to the gravity, I represents the inertia matrix, C is the Coriolis matrix, η = ( ϕ , θ , ψ ) T is the attitude vector, and τ ˜ is the control input vector.

2.2. Gimbal Model

Figure 3 shows the gimbal diagram, where the G = ( x g , y g , z g ) represents the frame fixed to the optical flow sensor. In [9,10], the authors propose the gimbal mathematical model as follows:
J x ϕ ¨ g + ξ 1 ϕ ˙ g = υ ϕ + d ϕ J y θ ¨ g + ξ 2 θ ˙ g = υ θ + d θ J z ψ ¨ g + ξ 3 ψ ˙ g = υ ψ + d ψ
where J = D ( J x , J y , J z ) is the gimbal rotational inertia matrix, and η g = ( ϕ g , θ g , ψ g ) represents the torques in the pitch, roll, and yaw gimbal angles. ξ = ( ξ 1 , ξ 2 , ξ 3 ) are the friction coefficients for each angle. Moreover, υ = ( υ ϕ , υ θ , υ ψ ) is the control input, and d = ( d ϕ , d θ , d ψ ) is the nonlinear dynamic effect of the couplings that are considered as a disturbance.

2.3. Optical Flow Model

Optical flow is a monocular vision method that uses images of a moving object to estimate the displacement velocity.
Figure 4 shows the optical flow diagram, where the O = ( x o , y o , z o ) represents the frame fixed of the optical flow sensor. P is a direct point on the ground from the optical axis, f is the focal length from the centre of projection O to the image plane, and h is the altitude from the image plane to the ground. θ o is the optical axis angle and the z i axis. The authors of [11] present the approximate pixel displacements as follows:
V x = T z x T x f h ω y f + ω z y + ω x x y ω y x 2 f V y = T z y T y f h + ω x f ω z x + ω x y 2 ω y x y f
where ( V x , V y ) are the estimate optical flow translation velocities, ( ω x , ω y , ω z ) are the angular velocities, and ( T x , T y , T z ) are the linear velocities.

3. Control Strategies

This section presents a feedback control algorithm implemented to stabilize the quadrotor aircraft altitude and the robust control algorithms that guarantee finite-time convergence for the quadrotor aircraft pitch and roll angles and for the gimbal pitch and roll angles. The main goal of the algorithms based on Sliding Mode Control is to stabilize the quadrotor aircraft attitude and the optical flow module attitude independently. The quadrotor aircraft altitude and attitude are controlled by the aircraft’s four rotors angular velocity, and the gimbal is controlled by its two rotor torques that stabilize the optical flow module on the horizontal line.

3.1. Quadrotor Aircraft Control

The quadrotor aircraft robust control approach enables stable flight by damping external disturbances or attitude tilts for horizontal vehicle displacement. This behavior is very important for the entire aerial vehicle system.

3.1.1. Vertical Displacement and Yaw Control ( z , ψ )

A feedback control algorithm has been selected to stabilize the vertical position, and it is obtained using the following input:
u = c 1 + m g cos ϕ cos θ
where c 1 is defined as
c 1 = k p ( z z d ) k d z ˙
where k p , k d > 0 and z d is the desired altitude. Introducing (6) into (3)
z ¨ = k p ( z z d ) k d z ˙
When the quadrotor aircraft reaches the desired altitude z z d then c 1 0 .
The yaw angle ψ is controlled by
τ ψ = k a ( ψ ψ d ) k b ψ ˙
where k a , k b > 0 and ψ d means the desired yaw angle.

3.1.2. Lateral Displacement and Roll Control ( y , ϕ )

By applying (6) into Equation (3), after a convergence time, the lateral dynamics become
y ¨ m g tan ϕ
In order to guarantee the robust tracking in y-axis, differentiating (10) twice gives
y ( 4 ) = m g ( ϕ ¨ + 2 ϕ 2 ˙ tan ϕ ) sec 2 ϕ
Introducing (3) into (11) and applying a variable change τ ˜ = τ + C ( η , η ˙ ) η ˙ and inertia are considered the identity matrix
y ( 4 ) = m g ( τ ϕ + 2 ϕ ˙ 2 tan ϕ ) sec 2 ϕ = τ ϕ m g sec 2 ϕ + 2 m g ϕ ˙ 2 tan ϕ sec 2 ϕ
Let us define the following f a and f b as
f a ( ϕ ) = ( m g sec 2 ϕ ) , f b ( ϕ , ϕ ˙ ) = 2 m g ϕ ˙ 2 tan ϕ sec 2 ϕ
where f a ( · ) , f b ( · ) represents terms obtained after a second derivative. We have considered
f a ( ϕ ) > 0
Then, the y dynamics are
y ( 4 ) = τ ϕ f a ( ϕ ) + f b ( ϕ , ϕ ˙ )
In order to reach the desired trajectory, we propose the robust control based on the Sliding Mode Control algorithm. Let us define the following variables:
y ˙ 1 = y ˙ y ˙ 2 = y ¨ y ˙ 3 = y ( 3 ) y ˙ 4 = y ( 4 ) = τ ϕ f a ( ϕ ) + f b ( ϕ , ϕ ˙ )
and the following errors as
e 1 = y y d ( t ) e 2 = y ˙ y ˙ d ( t ) e 3 = y ¨ y ¨ d ( t ) e 4 = y ( 3 ) y d ( 3 ) ( t )
Differentiating
e ˙ 1 = e 2 e ˙ 2 = e 3 e ˙ 3 = e 4 e ˙ 4 = y ( 4 ) y d ( 4 ) ( t )
The sliding surface is defined as a function of error
s = ( k 1 e 1 + k 2 e 2 + k 3 e 3 ) + e 4
where k 1 , k 2 , k 3 > 0 . The objective is to reach the surface s = 0 . We propose the following positive function:
V = 1 2 s 2
we have
V ˙ = s ( k 1 e 2 + k 2 e 3 + k 3 e 4 + τ ϕ f a ( ϕ ) + f b ( ϕ , ϕ ˙ ) )
By choosing τ ϕ like
τ ϕ = 1 f a ( ϕ ) f b ( ϕ , ϕ ˙ ) + ( k 1 e 2 + k 2 e 3 + k 3 e 4 ) + ν ϕ
where ν ϕ is defined as
ν ϕ = ρ ϕ s i g n ( s )
where ρ ϕ > 0 and ν ϕ denote the Sliding Mode Controller, then V ˙ = ρ ϕ s s i g n ( s ) . Then, s 0 in a finite time. Then, from (19)
e 4 = ( k 1 e 1 + k 2 e 2 + k 3 e 3 )
this is an error linear system, where the Hurwitz dynamics characteristic polynomial is
p 3 + k 3 p 2 + k 2 p + k 1 = 0
then from (18) e 1 , e 2 , e 3 , e 4 0 .

3.1.3. Longitudinal Displacement and Pitch Control ( x , θ )

After a transient time, the longitudinal dynamics become
x ¨ m g tan θ cos ϕ
Consider it diagonal and compensate the torque. Considering a small angle ϕ , (22) becomes approximately x ¨ m g tan θ . Indeed, since the quadrotor aircraft configuration is a symmetrical frame, it is also possible to apply the Sliding Mode Controller to the pitch angle ( θ ) and to the longitudinal displacement in the x-axis.

3.2. Gimbal Control

The gimbal robust control approach ensures that the attitude of the optical flow module is stable and independent of the quadrotor aircraft attitude. This performance is very important for optical flow sensing.

3.2.1. Gimbal Pitch Control ( θ g )

An algorithm based on Nonsingular Terminal Sliding Mode Control (NTSMC) is selected to stabilize the gimbal pitch angle ( θ g ) due to the robustness to disturbances.
Since the tracking error converges to zero, we propose the following sliding surface:
s g = θ g + α s i g n γ 1 ( θ g ) + β s i g n γ 2 ( θ ˙ g )
where s i g n γ ( θ g ) = s i g n θ g | θ g | γ , θ g is the gimbal pitch angle, and the variables satisfying α , β > 0 , 1 > γ 1 > 2 , γ 1 > γ 2 and s i g n ( · ) are the signum function. Then, s g guarantees the convergence to zero in finite time. The algorithm control law υ θ is defined as
υ θ = J y 1 β γ 2 s i g n 2 γ 2 θ ˙ g 1 + α γ 1 | θ g | γ 1 + β θ ˙ g + K 1 s i g n ( s ) + K 2 s
where K 1 , K 2 > 0 . The control law υ θ for the pitch ( θ g ) angle guarantees the finite-time convergence of the tracking error to zero. The stability proof can be found in [12].

3.2.2. Gimbal Roll Control ( ϕ g )

Indeed, since the gimbal configuration is symmetrical and orthogonal in x y -axes, we applied the same control strategy to the angular displacement roll control ( ϕ ).
The algorithm control law υ θ is defined as
υ ϕ = J x 1 β γ 2 s i g n 2 γ 2 ϕ ˙ g 1 + α γ 1 | ϕ g | γ 1 + β ϕ ˙ g + K 1 s i g n ( s ) + K 2 s
The control law υ θ for the roll ( ϕ g ) angle guarantees the finite-time convergence of the tracking error to zero.

3.3. Chattering Avoidance

It is well known that the implementation of the control approach based on sliding modes brings with it the disadvantage of the vibration effect present in the real-time experimentation of the sign function. In implemented control systems, it is important to ensure the performance of the system in the face of high working frequency and that the electronic controller can command in real time considering the electronic and mechanical wear of its actuators. However, in [8] the authors propose replacing the sign function with a high slope saturation function s i g n ( · ) s a t ( · ) to solve this issue.
In our approach, in order to avoid the chattering effect caused by the signum function in robust control algorithms used in the quadotor attitude and gimbal attitude, the approximation function s i g n ( · ) tanh ( · ) is used to smooth the signal. The signum function and hyperbolic tangent function are shown in Figure 5.

4. Materials and Methods

In this section, we will describe the different hardware parts of the embedded control system, namely, sensors, actuators, pilot communication, and the flight controller board that links them together. The total weight of the Unmanned Aerial Vehicle is 2.5 kg; the weight of the gimbal is clearly considered as the payload. The setup presented can be scaled to a smaller quadrotor aircraft by using a smaller gimbal or using servomotors instead of brushless motors for stabilization purposes. However, as in the case of Unmanned Aerial Vehicles, the gains must be adjusted in flight tests.
In order to guarantee the stability of the aerial vehicle for the development of the experiments through the optical flow module, an advanced autopilot called: Pixhawk 6X is used, which provides the corresponding measurements (attitude, position) for the high-performance control of our quadrotor aircraft.
Finally, the optical flow sensor presented in this work uses a stabilized gimbal to avoid the positive feedback produced when the vehicle tilts. The vehicle platform was built with the components shown in Table 1.
Figure 6 shows the system diagram. A quadrotor aircraft and optical flow module are stabilized by gimbal, where I = ( x i , y i , z i ) represents the inertial frame, B = ( x b , y b , z b ) represents the body frame fixed to the center of mass of the quadrotor aircraft, and O = ( x o , y o , z o ) is the frame fixed to the optical flow module. Finally, h is the altitude and ( V x , V y ) are the horizontal velocity estimations.
The flight controller and custom PX4 firmware are used to control the quadrotor aircraft and stabilized gimbal. The modular flight controller has redundant embedded sensors (IMU’s), communication modules, and peripherals sensors. Figure 7 shows the experimental platform using an optical flow module stabilized by a gimbal. In this configuration, the experimental setup does not include GPS or MoCam.
The gimbal system configuration consists of two tilting joints around the x o and y o axis to control θ o and ϕ o close to zero. The gimbal mathematical model and robust control algorithm are important to ensure the performance through the torque stability of the motors. The gimbal robust control algorithm is implemented on a PX4 compatible controller board. In order to avoid the chattering effect caused by the signum function in robust control algorithms, we use the approximation function s i g n ( · ) tanh ( · ) .
The improved optical flow module placed on the gimbal ensures that the estimation of the vertical position and the horizontal displacement velocity are reliably estimated regardless of changes in pitch-roll angles caused by the attitude disturbances or by the quadrotor aircraft displacement. Therefore, the optical flow angular velocities are close to zero ω x , ω y 0 ; Equation (5) can be rewritten as
V x = T z x T x f h V y = T z y T y f h
Therefore, the distance and optical flow sensor estimates can be used to control the position of the quadrotor aircraft considering the altitude h z and the horizontal displacement as
V x x ˙ V y y ˙

5. Experimental Results

To validate the proposed control strategies for robust attitude control implemented in the quadrotor aircraft and the stabilized gimbal, we will summarize the various real-time experimental tracking tests at different stages of development, see Figure 8.
The tracking performance can be observed as it reaches the desired position. The controller parameters were adjusted by trial and error until the best performance was obtained. The parameters used are shown in Table 2.
Figure 9 shows an indoor flight tracking control test, and Figure 10 shows the trajectory in three dimensions of the quadrotor aircraft, which demonstrates the correct execution of the indoor flight mission using the optical flow sensor to track the programmed trajectory in the quadrotor aircraft at an altitude of 2 m and with a distance of 7 m.
In the following link, we share a video to show the disturbance rejection tests of the unmanned aerial system to demonstrate the quadrotor aircraft performance and the stabilized gimbal performance. Additionally, hover stability and a good performance in optical flow-based trajectory tracking control in a GPS-Denied environment are ensured.
https://youtu.be/SX1zxz5x9ko (accessed on 20 March 2024)

5.1. Quadrotor Aircraft Pitch and Roll Attitude ( θ , ϕ )

The attitude angles performance of the quadrotor aircraft during real time flight are presented below. Figure 11 shows the robust control law algorithm (20) behavior applied on the pitch angle stability of the quadrotor aircraft with disturbances added over the whole trajectory.
Figure 12 shows the robust control law algorithm (20) behavior applied on the roll angle stability of the quadrotor aircraft, with disturbances added over the whole trajectory.
The quadrotor aircraft attitude angles ( θ , ϕ ) back to the reference in a short period of time, achieving normal operation after disturbances, with some oscillatory behavior around the horizontal reference values. Therefore, the results obtained are satisfactory.

5.2. Gimbal Pitch and Roll Attitude ( θ g , ϕ g )

The attitude angles performance of the stabilized gimbal during real time flight are presented below.
Figure 13 shows the robust control law algorithm (24) behavior applied on the pitch angle stability of the gimbal, with disturbances added over the fight.
Figure 14 shows the robust control law algorithm (25) behavior applied on the roll angle stability of the gimbal, with disturbances added over the fight.
The gimbal attitude angles ( θ g , ϕ g ) back to the reference in a short period of time to normal operation after rejecting the angular disturbances induced by the quadrotor aircraft, with some oscillatory behavior around the horizontal reference values. Therefore, the results obtained are satisfactory.

5.3. Optical Flow Sensing ( z , V x , V y )

The altitude and displacement velocity sensing in real time flight are presented below.
Figure 15 shows the altitude estimate by the distance sensor to reach the desired altitude ( z d = 2 m).
Figure 16 shows the optical flow estimation of lateral velocity during indoor flight. An increase in speed of almost 0.04 m/s is observed during its lateral path of 7 m.
Figure 17 shows the optical flow estimates of the lateral and longitudinal velocities, respectively, during the indoor flight. A lateral velocity of 0.01 m/s during a longitudinal path of 1 m is observed.
It is possible to observe a peak in the horizontal displacement velocity at the start and at the end of the flight. This occurs when the quadrotor aircraft does not take off or land with its full landing gear (due to disturbances, like the ground effect, or because the terrain is not flat and horizontal); the optical flow can detect a small horizontal displacement that increases with short focal length.

6. Discussion

There are many approaches around GPS-Denied navigation; some are mentioned in Section 1 of this work. The stabilized optical flow avoids the positive feedback of the velocity and position estimation. Regardless of the inclination rate and attitude change of the quadrotor, the stabilized optical flow will always maintain the downward orientation; this is the main goal of this work. However, a handicap of using a stabilized optical flow is the extra weight of the gimbal itself. Normally, the use of conventional optical flow sensors considers limitations in the translational velocity and the altitude of the sensor with respect to the ground as this altitude increases the estimation of the velocity, which is more sensitive to changes in sensor orientation. In our proposal, we are avoiding these attitude changes due to the stabilization of the optical flow. To carry out safe flight tests, it is required to know the operating limits of the sensors to produce reliable local estimates of altitude and horizontal velocity. For example, the maximum distance sensed by the optical flow module is 2 m, so we use a distance sensor with a longer operating range (40 m).

7. Conclusions

This paper has proposed a novel technique for navigating in a GPS-Denied environment using an optical flow module stabilized by a gimbal system. The proposed approach solves the problem of having a positive feedback in the pitch angle when the optical module is attached to the body of the aircraft. The technique consists of a control algorithm that keeps the optical module facing downwards so that the optical flow is independent of the pitch angle of the aircraft. We have successfully tested the proposed approach in an experimental platform in outdoor and indoor environments. The control algorithms of both the aircraft and gimbal were designed robustly to reduce the effect of external perturbations. The experiments showed that in spite of disturbances, the aircraft Euler angles remained close to the origin.

Author Contributions

Investigation, J.F.; project administration, I.G.-H.; supervision, S.S.; data curation, R.L.; and conceptualization, C.R. All authors have read and agreed to the published version of the manuscript.

Funding

This project was funded by Department of Research and Multidisciplinary Studies of Research and Advanced Studies Center of the National Polytechnic Institute (CINVESTAV-IPN).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Dataset available on request from the authors.

Acknowledgments

The authors are grateful to the National Council of Science, Humanities, and Technology (CONAHCYT) for its support.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
UAVUnmanned Aerial Vehicle
VTOLVertical Take-Off and Landing
IMUInertial Measurement Unit
GPSGlobal Position System
GRUGated Recurrent Unit
MARGMagnetic, Angular Rate, and Gravity
FPVFirst Person View
SMCSliding Mode Control
NTSMCNonsingular Terminal Sliding Mode Control
NTSNonsingular Terminal Sliding Surface

References

  1. Mashood, A.; Dirir, A.; Hussein, M.; Noura, H.; Awwad, F. Quadrotor object tracking using real-time motion sensing. In Proceedings of the 2016 5th International Conference on Electronic Devices, Systems and Applications (ICEDSA), Ras Al Khaimah, United Arab Emirates, 6–8 December 2016; pp. 1–4. [Google Scholar]
  2. Okulski, M.; Ławryńczuk, M. A Small UAV Optimized for Efficient Long-Range and VTOL Missions: An Experimental Tandem-Wing Quadplane Drone. Appl. Sci. 2022, 12, 7059. [Google Scholar] [CrossRef]
  3. Van den Bergh, B.; Pollin, S. Keeping UAVs under control during GPS jamming. IEEE Syst. J. 2018, 13, 2010–2021. [Google Scholar] [CrossRef]
  4. Schmid, K.; Tomic, T.; Ruess, F.; Hirschmüller, H.; Suppa, M. Stereo vision based indoor/outdoor navigation for flying robots. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–7 November 2013; pp. 3955–3962. [Google Scholar]
  5. Romero, H.; Salazar, S.; Lozano, R. Real-time stabilization of an eight-rotor UAV using optical flow. IEEE Trans. Robot. 2009, 25, 809–817. [Google Scholar] [CrossRef]
  6. Sun, K.; Yu, Y.; Zhou, W.; Zhou, G.; Wang, T.; Li, Z. A low-cost and robust optical flow CMOS camera for velocity estimation. In Proceedings of the 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), Shenzhen, China, 12–14 December 2013; pp. 1181–1186. [Google Scholar]
  7. Liu, X.; Li, X.; Shi, Q.; Xu, C.; Tang, Y. UAV attitude estimation based on MARG and optical flow sensors using gated recurrent unit. Int. J. Distrib. Sens. Netw. 2021, 17, 15501477211009814. [Google Scholar] [CrossRef]
  8. González, I.; Salazar, S.; Lozano, R. Chattering-free sliding mode altitude control for a quad-rotor aircraft: Real-time application. J. Intell. Robot. Syst. 2014, 73, 137–155. [Google Scholar] [CrossRef]
  9. Altan, A.; Hacıoğlu, R. Model predictive control of three-axis gimbal system mounted on UAV for real-time target tracking under external disturbances. Mech. Syst. Signal Process. 2020, 138, 106548. [Google Scholar] [CrossRef]
  10. Mousavi, Y.; Zarei, A.; Jahromi, Z.S. Robust adaptive fractional-order nonsingular terminal sliding mode stabilization of three-axis gimbal platforms. ISA Trans. 2022, 123, 98–109. [Google Scholar] [CrossRef] [PubMed]
  11. Honegger, D.; Meier, L.; Tanskanen, P.; Pollefeys, M. An open source and open hardware embedded metric optical flow cmos camera for indoor and outdoor applications. In Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2013; pp. 1736–1741. [Google Scholar]
  12. Yang, L.; Yang, J. Nonsingular fast terminal sliding-mode control for nonlinear dynamical systems. Int. J. Robust Nonlinear Control 2011, 21, 1865–1879. [Google Scholar] [CrossRef]
Figure 1. (a) Optical flow without stabilization and (b) stabilized optical flow.
Figure 1. (a) Optical flow without stabilization and (b) stabilized optical flow.
Sensors 24 02183 g001
Figure 2. Quadrotor aircraft diagram.
Figure 2. Quadrotor aircraft diagram.
Sensors 24 02183 g002
Figure 3. Gimbal diagram.
Figure 3. Gimbal diagram.
Sensors 24 02183 g003
Figure 4. Optical flow diagram.
Figure 4. Optical flow diagram.
Sensors 24 02183 g004
Figure 5. Signum function and hyperbolic tangent function.
Figure 5. Signum function and hyperbolic tangent function.
Sensors 24 02183 g005
Figure 6. Navigation system: A quadrotor aircraft and a stabilized optical flow module.
Figure 6. Navigation system: A quadrotor aircraft and a stabilized optical flow module.
Sensors 24 02183 g006
Figure 7. Experimental UAV platform.
Figure 7. Experimental UAV platform.
Sensors 24 02183 g007
Figure 8. Stabilized gimbal in hover flight test with external perturbations.
Figure 8. Stabilized gimbal in hover flight test with external perturbations.
Sensors 24 02183 g008
Figure 9. Indoor flight tracking control test with stabilized optical flow.
Figure 9. Indoor flight tracking control test with stabilized optical flow.
Sensors 24 02183 g009
Figure 10. Three-dimensional trajectory tracking of the quadrotor aircraft ( x , y , z ).
Figure 10. Three-dimensional trajectory tracking of the quadrotor aircraft ( x , y , z ).
Sensors 24 02183 g010
Figure 11. Quadrotor aircraft pitch angle ( θ ).
Figure 11. Quadrotor aircraft pitch angle ( θ ).
Sensors 24 02183 g011
Figure 12. Quadrotor aircraft roll angle ( ϕ ).
Figure 12. Quadrotor aircraft roll angle ( ϕ ).
Sensors 24 02183 g012
Figure 13. Gimbal pitch angle ( θ g ).
Figure 13. Gimbal pitch angle ( θ g ).
Sensors 24 02183 g013
Figure 14. Gimbal roll angle ( ϕ g ).
Figure 14. Gimbal roll angle ( ϕ g ).
Sensors 24 02183 g014
Figure 15. Vertical displacement (z).
Figure 15. Vertical displacement (z).
Sensors 24 02183 g015
Figure 16. Lateral velocity ( V x ).
Figure 16. Lateral velocity ( V x ).
Sensors 24 02183 g016
Figure 17. Longitudinal velocity ( V y ).
Figure 17. Longitudinal velocity ( V y ).
Sensors 24 02183 g017
Table 1. UAV components.
Table 1. UAV components.
Carbon fiber airframe 550 mmPixhawk 6X Autopilot
Carbon fiber Propellers 1355 mmRadio Control Receiver DSM
Outrunner Brushless Motors 335 KVTelemetry Radio Sik
Brushless Electronic Speed Controllers 30 ARangefinder Lidar-Lite
LiPo Batery 5500 mAh 6SOptical Flow PMW3901 Based Sensor
Table 2. System parameters.
Table 2. System parameters.
ParameterValueParameterValueParameterValueParameterValue
k p 0.36 δ 1 1.45 k b 0.86 γ 1 1.23
k d 0.044 δ 2 1.05 α 0.71 γ 2 0.95
z d 2.0 m k 1 0.54 β 1.61 K 1 0.68
k a 0.084 k 2 0.33 ρ 1.23 K 2 1.33
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Flores, J.; Gonzalez-Hernandez, I.; Salazar, S.; Lozano, R.; Reyes, C. Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment. Sensors 2024, 24, 2183. https://doi.org/10.3390/s24072183

AMA Style

Flores J, Gonzalez-Hernandez I, Salazar S, Lozano R, Reyes C. Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment. Sensors. 2024; 24(7):2183. https://doi.org/10.3390/s24072183

Chicago/Turabian Style

Flores, Jonathan, Ivan Gonzalez-Hernandez, Sergio Salazar, Rogelio Lozano, and Christian Reyes. 2024. "Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment" Sensors 24, no. 7: 2183. https://doi.org/10.3390/s24072183

APA Style

Flores, J., Gonzalez-Hernandez, I., Salazar, S., Lozano, R., & Reyes, C. (2024). Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment. Sensors, 24(7), 2183. https://doi.org/10.3390/s24072183

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop