Comparison of Flight Parameters in SIL Simulation Using Commercial Autopilots and X-Plane Simulator for Multi-Rotor Models

Abstract

Modern aviation technology development heavily relies on computer simulations. SIL (Software-In-The-Loop) simulations are essential for evaluating autopilots and control algorithms for multi-rotors, including drones and other UAVs (Unmanned Aerial Vehicle). In such simulations, it is possible to compare the flight parameters achieved by flying platforms using various commercial autopilots widely used in the UAV sector. This research aims to provide objective and comprehensive insights into the effectiveness of different autopilot systems This article examines the simulated flight test results of a drone performing the same mission using different autopilot systems. The X-Plane software was used as an environment to simulate the dynamics of the drone and its surroundings. Matlab/Simulink r2023a provided the interface between autopilot software and X-Plane models. Those methods allowed us to obtain an appropriate comparison of the autopilot systems and indicate the main differences between them. This research focused on analyzing UAV flight characteristics such as stability, trajectory tracking, response time to control changes, and the overall effectiveness of autopilots. Various flight scenarios including take-off, landing, flight at a constant altitude, dynamic manoeuvrers and, flight along a planned trajectory were also examined. In order to obtain the most accurate and realistic results, the tests were carried out in various weather conditions. The aim of this research is to provide objective data and analysis to compare the performance of commercial autopilots. This method offers several advantages, including cost-effective testing, the ability to test in diverse environmental conditions, and the evaluation of autopilot algorithms without the need for real hardware. The findings of this study may have a considerable impact on how autopilot designers and developers choose the best platforms and technologies for their projects. Future research on this topic will compare the obtained data with flight test data.

1. Introduction

The use of commercial autopilots in Unmanned Aerial Vehicles (UAVs) has become a very common phenomenon. This is due, among other things, to their high availability, popularity, and broad community of users. These types of autopilots are widely used in many areas, such as agriculture, photography, surveillance, firefighting, personal use, infrastructure inspection, and much more [1]. Commercial autopilots are easy to use and install on the drone, regardless of its type. The more challenging aspects involve fine-tuning the PID ( Proportional-Integral-Derivative) controllers for a particular platform, designing the mission, and mastering the art of drone piloting [2,3]. Several pieces of literature have described and investigated the use of commercial autopilots for Unmanned Aerial Vehicles. Studies such as [4] described an approach for the implementation of controllers with COTS (Commercial Off-The-Shelf) autopilot for a simple kinematic model of aircraft dynamics. The open-loop controller was implemented by using output tracking in the study, with the COTS autopilot serving as the closed-loop interface. Typically, commercial autopilots are designed to provide only guidance loops for the purpose of waypoint navigation. However, the study performed by Kaminer, I. [5] presented a three-dimensional path following a control algorithm for off-the-shelf autopilots. The aim of implementing such an algorithm was to expand the possible range of UAV applications. Moreover, integrating commercial autopilots with UAV airframes provides several advantages and benefits, including enhancing safety by reducing human error. They aid in performing stable and precise manoeuvres, especially in dangerous and complex situations. In addition, autopilot integration provides an extension for UAV capabilities and data collection to enhance decision making, mission-planning improvements, and performance analysis [6].

In the development and testing of Unmanned Aerial Vehicles, integrating commercial autopilots with Software-In-The-Loop (SIL) simulations provides significant advantages. Such an approach was presented by [7], where the flight stabilisation system was designed and modelled by using Software-In-The-Loop (SIL) simulations [8]. The simulation environment provides a controlled environment for identifying and resolving issues early in the development cycle, minimising the need for expensive equipment and resources. Article [9] shows the usage of simulating an environment for testing the vision algorithms to determine the position of the drone. This method also enables risk mitigation by simulating various flight scenarios and potential failures in a simulated environment. In a related study focusing on UAV-based wireless sensor networks (WSNs), an improved k-means clustering algorithm was proposed to improve mission area decomposition and optimise UAV flight paths [10]. Identifying and resolving problems during tests helps minimise the risks associated with unpredictable real-world conditions, ensuring a safer and more reliable autopilot system. It is possible to create diverse and complex scenarios that may be difficult to test in the real world. Autopilots can be exposed to a wide range of environmental conditions, allowing developers to evaluate system performance and ensure robustness and adaptability. Detecting and fixing problems early reduces the likelihood of costly modifications later in the development cycle, leading to a more efficient and timely project delivery. This method allows developers to monitor and analyse system responses and quickly implement improvements. The existing literature has researched the integration of commercial autopilots with SIL simulations, such as the work presented in [11,12,13,14]. This approach was utilised by [15] to design a novel control structure that permits the explicit creation of tracking techniques for a system with an unchangeable autopilot in the loop. In real-life conditions, the presence of wind presents a significant disturbance in flight conditions; these disturbances must be taken into account during the development of the UAV system. Research taking into account wind disturbances for the design of a guidance controller was presented in [16].

This study aimed to test commercial autopilots in SIL simulations for an octocopter drone. A special environment was created to enable the comparison of two different autopilots. The configuration used allowed for a detailed comparison of the operation of different systems. Two commercial autopilots, Ardupilot and Veronte, were taken into account in this research. The strengths and weaknesses are described and the main differences between them are indicated. This study incorporates aerodynamics, flight dynamics, and motor models from the Xplane simulator, which offers a powerful and effective approach for the validation and testing of the autopilots under highly realistic flight conditions. A diverse range of environmental conditions are available for testing in the Xplane, along with sophisticated visualisations and the ability to record a wide array of flight data [17,18,19,20,21]. The navigation algorithms can be compared with different flight modes or the behaviour of failures during the mission. The errors of sensors can be simulated, such as signal blackouts, sensor failures [22], actuator failures, and navigation system failures. Performing such a study by integrating Ardupilot and Veronte autopilots with SIL simulations using the Xplane capability may aid in the selection of commercial autopilots to increase the effectiveness of UAV development and improve mission design. This in turn leads to a reduction in the necessary resources and development time. The usage of the X-Plane simulator for the visualisation of the mission and Matlab/Simulink for autopilot algorithms was presented in [23].

Regarding the organisation of this work, the following Section 2 of this study presents the multi-rotor octocopter test bed. This is followed by Section 3, which extensively discusses the methods used in this research with the selected commercial autopilots and the corresponding software used for planning the missions before the real flights or simulations. Then, Section 4 presents the concept of the SIL simulation. Finally, conclusions with remarks and future suggestions are highlighted in Section 6. In this study, our main contribution is a comprehensive evaluation and comparison of two widely used commercial autopilots, Ardupilot and Veronte, in Software-In-The-Loop (SIL) simulations for an octocopter drone. By carefully examining their strengths, weaknesses, and main differences, our goal is to provide valuable information on the selection of commercial autopilots in UAV development. The research used advanced models from the Xplane simulator, providing a realistic and robust testing environment. Additionally, we study the impact of environmental conditions and simulate various scenarios to assess the effectiveness of autopilots in various challenges. Through this extensive analysis, we aim to contribute to the development of UAV development practices and improve the decision-making process regarding autopilot selection.

2. Test Vehicle

A coaxial octocopter UAV was selected as the test bed for this research. The model consists of eight propellers with a diameter of 51.3 cm, and the arrangement of the engines is symmetrical with a wheelbase of 86.3 cm. The model of the drone is presented in Figure 1, with its inertial parameters listed in

Table 1. Mass and inertia parameters.

Parameter	Value	Units
m	16.2	[kg]
${\mathrm{I}}_{\mathrm{xx}}$	0.873	[$\mathrm{kg}·{\mathrm{m}}^2$]
${\mathrm{I}}_{\mathrm{yy}}$	0.909	[$\mathrm{kg}·{\mathrm{m}}^2$]
${\mathrm{I}}_{\mathrm{zz}}$	1.263	[$\mathrm{kg}·{\mathrm{m}}^2$]

.

The UAV multi-rotors were configured to rotate in opposite directions to balance the torque generated by each rotor; the rotation direction and notation of each propeller are presented in Figure 2. The rotation speed of the rotors was based on commanded orientation signals in the form of the Euler angles $\varphi$, $\theta$, and $\psi$. The commanded orientation signals were then converted to motor commands with the aid of a mixer matrix (See Equation (1)).

$$\left[\begin{array}{c}Motor1\\ Motor2\\ Motor3\\ Motor4\\ Motor5\\ Motor6\\ Motor7\\ Motor8\end{array}\right]={\left[\begin{array}{ccccccccc}-1& 1& 1& -1& 1& -1& -1& 1& \\ 1& 1& -1& -1& 1& 1& -1& -1& \\ -1& 1& -1& 1& -1& 1& -1& 1& \\ 1& 1& 1& 1& 1& 1& 1& 1& \end{array}\right]}^T·\left[\begin{array}{c}Roll\\ Pitch\\ Yaw\\ Thrust\end{array}\right]$$

(1)

3. Research Methods

For the purpose of this research, the curve of the earth and its rotation were neglected. Therefore, a flat earth model was used in this study as the global coordinate system. The x-axis of the global coordinate system is directed to the north, the y-axis is directed to the east, and the z-axis points down; those axes create the North East Down (NED) coordinate system. The body-fixed coordinate system and orientation Euler angles $\varphi$, $\theta$, and $\psi$ are shown in Figure 3.

The roll angle, denoted by the symbol $\varphi$, represents the rotation about the longitudinal axis of the object. A positive roll corresponds to a counterclockwise rotation when viewing along the positive x-axis toward the origin and negative to a clockwise rotation. The pitch angle, denoted by the symbol $\theta$, represents the rotation about the lateral axis of the object. The positive pitch corresponds to the rotation that points the object’s nose upward, and the negative angle points the nose downward. The yaw angle, denoted by the symbol $\psi$, represents the rotation about the vertical axis of an object. A positive yaw angle corresponds to a counterclockwise rotation when viewing along the positive z-axis toward the origin, while the negative sign corresponds to clockwise rotation. It is essential to be aware of the coordinate system used in Figure 3. The Euler angle conventions ‘ZYX’ define the sequence of rotations and the axis order. Different conventions may lead to different interpretations of positive and negative rotations.

The octocopter model was developed within the PlaneMaker 11.55 program, a tool used to design flying models in the Xplane simulator. It takes into account dimensions, mass, moments of inertia, and aerodynamic coefficients. Unlike traditional simulators, the X-Plane uses blade element theory in its aerodynamic model. Blade element theory enhances this simulation approach by modelling the forces and moments acting on the aircraft and individually evaluating its component parts [25]. This segmentation was carried out in accordance with design assumptions aimed at making the dynamic model as accurate as possible to the real object. Figure 4 shows the model prepared in the PlaneMaker 11.55 software used for simulations in the X-Plane.

For this study, two commercial autopilots were used: the Ardupilot [26], an open-source autopilot, and the second one, Veronte [27]. Both of them are easy-to-use autopilots that can be deployed on many flying and moving platforms. Operators can install additional devices such as external transponders, LIDAR, radars, or gimbals. Veronte was developed in accordance with aviation standards such as DO178C/ED-12, DO254, and DO160, while Ardupilot is an open-source project with a large community of users and more customisation options.

For both of the autopilots, the limitations such as the Euler angles and angular velocities were set. For the pitch and roll angles, the maximum desired value is 30 degrees and the maximum angular velocities were set to 90 deg/s. When it comes to the time constant of a drone, it is largely defined by the propulsion system dynamic. The time constant of the single motor is equal to 0.1 s, and this amount of time is needed by the motor to reach 63.2% of its final value. This characteristic influences the dynamic behaviour of the octocopter. Figure 5 illustrates the motor’s response to a step command. The maximum thrust of individual power units is approximately 60 Newtons.

Each of those autopilots has dedicated software for tuning the controllers and planning the mission. Both of them are intuitive and easy to use. Veronte provides Veronte Pipe 6.4.52 software, in which the operator has to design an autopilot structure with control loops and PID regulators. The design process involves arranging blocks in a designated area and connecting them to various signals, providing users with a detailed view inside the control loops. While this approach may be more time consuming compared to Ardupilot, it provides operators with a thorough understanding of the inner workings of the autopilot. An example of such an autopilot design is shown in Figure 6.

The grey blocks presented in the figure above represent PID controllers whose parameters are set by the user. Depending on the operator, it may be a simple PID controller or a more complex one. Figure 7 shows the content of this block.

Figure 8 represents the section of Veronte Pipe 6.4.52 software dedicated to planning the mission.

For Ardupilot autopilots, the MissionPlanner 1.3.80 software was used. This is ground control station software dedicated to planning, monitoring, and controlling UAV missions. It also provides tools for the real-time visualisation of flight data and the configuration of vehicle parameters. Figure 9 shows the part of the MissionPlanner responsible for planning the UAV mission. This is a user-friendly tool where waypoints can be placed on a specific point on the map by the operator or imported from a csv file.

Figure 10 presents the section where the operator can change and tune the parameters of the autopilot. Those parameters are responsible for flight and mission performance.

4. Simulation Environment

To replicate the environment of the mission, the X-Plane simulator was chosen. It was chosen because of the customisation options of the model and environment. The aerodynamic model of the octocopter was designed in PlaneMaker 11.55, so the aerodynamic forces can be calculated by the simulator. Parameters such as the mass and moments of inertia were taken into account during the design process. One of the drawbacks of this approach is that it is impossible to import the model from any of the CAD files; the user is obligated to redraw the model. Figure 11 presents the PlaneMaker tool.

The X-Plane simulator enables the application of weather disturbances such as wind, rain, and fog. The same environmental conditions can be replicated in many simulations. The graphics of the simulator are also an advantage; the real world can be accurately visualised, which is a big plus. Figure 12 shows the mission visualisation in the X-Plane simulator. For this purpose, version 11.55 of X-Plane was used.

Software-In-The-Loop simulations consist of autopilot software, ground control station software, a dynamic model of the UAV, and an environment simulator. The simple diagram of SIL is presented in Figure 13.

The communication between the components is conducted by using the UDP protocol. The frequency of the data coming from the X-Plane to the autopilot is set to the maximum possible value, equal to 100 Hz. The X-Plane sends data such as the acceleration of the UAV in the body frame, angular velocities, orientation, global coordinates, and position of the object. Those data are simulating the IMU, AHRS, and GPS measurements. The interface between the X-Plane and autopilots software was implemented in the Simulink Model. Figure 14 presents its structure; it consists of five blocks that exchange data between them. A block called ‘XPlane Data receive’ is responsible for receiving and parsing the data that the X-Plane sends during the simulation, and then those data go to the block called ‘Environment Data’, where the gravity acceleration and air density are computed, and ‘Veronte PWM Receiver’ and ‘Ardupilot PWM Receiver’, where the signals with states are sent to the autopilots software. The outputs from those two blocks are the PWM signals, which were calculated based on the simulation data. In the ‘APInterface block,’ those signals are translated and transmitted to the X-Plane simulator, where they are directed to the motors of the octocopter model.

5. Results

The comparison of the autopilots was conducted in three distinct mission scenarios. The first mission is a flight along a predefined trajectory, which is presented in Figure 15. The starting point and other initial conditions are the same for each mission. Those trajectories were imported into dedicated ground control software to perform the predefined missions. In the second mission scenario, the UAV follows the same trajectory as in the first scenario, with the additional inclusion of a weather disturbance, which is a northward wind with a magnitude of 5 m/s. The third scenario is a rapid deceleration from 25 to 0 m/s.

The results obtained from this research are presented in the graphs within the relevant subsections below.

5.1. First Scenario Results

In this section, the comparison of two sets of autopilot flight data from the first scenario is presented. Data such as the Euler angles, accelerations, and flight trajectory will be compared. The trajectory resembles a square, with each side measuring 300 m; in total, there are 10 waypoints. Figure 16 shows the comparison of two autopilot trajectories in the NED coordinate system; the axis is marked N points north and E points east.

The trajectory in blue corresponds to the mission test conducted with Ardupilot, while the red colour corresponds to Veronte. Circular symbols on the graph indicate “home” positions for the first two mission scenarios.

Figure 17 presents the orientation of the simulated object during the missions, containing data obtained from both missions using the Veronte and Ardupilot autopilot. The figure contains graphs on which the change in the roll, pitch, and yaw angles during the mission is demonstrated. The data in blue represent the Ardupilot data, and the data in red correspond to the Veronte data.

Figure 18 presents the accelerations relative to the body frame of the model that were obtained during the missions. It is the dynamic acceleration due to motion or external forces (including the gravitational components). This focus on variations in acceleration offers critical insights for analyzing motion, estimating orientation, and supporting various dynamic applications. The set of three figures is demonstrated; on the first graph, the acceleration along the x-axis in the body frame is presented. The second figure shows the acceleration along the y-axis and the acceleration along the z-axis in the third figure. On each of them, the data derived from the simulations using Ardupilot and Veronte autopilots are compared.

Figure 16 does not indicate any significant differences in the operation of both autopilots. The trajectories remain very close to each other. The Ardupilot autopilot faithfully reproduced the given trajectory. However, when performing missions with the Veronte autopilot, there were overshoots in turns, with errors not exceeding 3 m. The next two graphs show the difference in the mission completion time; Ardupilot completed the mission approximately 5 s faster than Veronte. This distinction becomes evident after the third turn. The values of the orientation angles and accelerations do not show significant differences in the operation of the two autopilots.

5.2. Second Scenario Results

This section includes the results from the second scenario tests in which the wind with a constant velocity was added. The trajectory planned for this mission is the same as in scenario 1, where the magnitude of wind is constant, equal to 5 m/s and in a north-to-south direction. Figure 19 presents the comparison of two trajectories; the blue colour corresponds to the trajectory of the object with the Ardupilot autopilot, while the red colour presents the data obtained from the simulation where the Veronte autopilot was used.

Figure 20 presents the orientation of the object during the mission; data from the simulations where the Veronte and Arudpilot were used is demonstrated on the same figures. The changes in the roll, pitch, and yaw angles are presented. In both cases, the differences in the orientation angles compared to the first scenario are evident. It is caused by the appearance of a constant wind velocity that either counteracts or facilitates the drone’s flight.

Similarly to Figure 18, the accelerations relative to the body frame of the drone during the missions are presented in Figure 21. The accelerations along the x-, y-, and z-axes in the body frame are demonstrated, as previously demonstrated in two simulations and compared in the figures.

Similarly to the previous scenario shown in Figure 19 with trajectories obtained during the missions, the current scenario does not show significant differences. During the Veronte autopilot mission, there were alternating overshoots, with deviations of no more than 4 m. The effect of constant winds did not significantly affect the performance of any autopilot during the mission. According to the above scenario, the Ardupilot autopilot completed the mission with a time advantage of approximately 5 s over Veronte. This consistent trend throughout the mission suggests a noticeable and repeatable difference in the performance of the two autopilots. The orientation angles and acceleration values provide evidence that the constant wind had no appreciable effect on mission performance.

5.3. Third Scenario Results

This subsection aims to present the data obtained from the final scenario, involving a rapid deceleration from 25 to 0 m/s. The flight data obtained from two simulations using two different autopilots are compared and visually represented on the graphs. Figure 22 shows the comparison of the change in velocity during the mission; in both cases, the deceleration was initiated just prior to the seventh s.

The data presented in blue correspond to the data obtained during the simulation, where the Ardupilot autopilot was tested, while the red colour represents data obtained during the Veronte autopilot testing. It can be observed from the graph that for both autopilots, the response times were approximately 3 s. After that, the velocity of the object is constant. It can be seen that the object controlled by the Veronte autopilot reached 0 m/s more quickly but was experiencing overshooting. Figure 23 shows the change in acceleration during the missions of Veronte and Ardupilot.

The colour scheme of the signals remains the same as before. The Veronte autopilot exhibited oscillations in its acceleration. For both cases, the maximum acceleration was about 15 [m/s²]. Overall, both autopilots performed the mission in a comparable way.

6. Conclusions

This section summarises the study of the comparison of the SIL simulations using commercial autopilots; the data obtained during the various simulation scenarios will be discussed, highlighting the differences between them. Additionally, an examination of the performance of the tested autopilots in distinct scenarios will be presented. The SIL simulations enabled a comprehensive assessment of each autopilot’s ability to handle a variety of scenarios, from routine operations to emergency situations. Each of the autopilots show strengths and areas for improvement looking at all stages from planning the mission.

Starting with the mission-planning stage, both autopilots demonstrated user-friendly interfaces and ease of use. No problems were encountered while importing the waypoints to the ground control software. In the context of designing and tuning autopilot algorithms, the Ardupilot demonstrated much more convenient methods and tools. The main autopilot parameters have to be set by the user, such as the proportional gain of the orientation control loop and proportional, integral, and derivative gains for the angular velocity control loop, while in the Veronte autopilot, the whole structure of the autopilot has to be created, including the inner and outer loops of the control system. This complexity contributes to its increased reliability and the operator’s awareness of what is happening within the control loops.

Moving on to the data obtained from the test scenarios, both of the autopilots successfully handled the three scenarios. The simulation data did not reveal significant differences in the performance of both autopilots. The trajectory flights aligned with the trajectory defined before the mission, and no significant overshootings or bias errors appeared. Any differences in the orientation angles or accelerations resulted from the autopilot algorithms, and the Ardupilot autopilot had a more complex structure, while the Veronte autopilot was based on closed-loops controllers in this research. The second scenario gave similar results, whereby both of the tested autopilots handled the presence of wind. The flight trajectory coincided with the desired one, and also the orientation angles and accelerations obtained from the simulations are similar. The third scenario showed the differences in the autopilots’ performance during rapid deceleration from 25 to 0 m/s. In this comparison, the Veronte autopilot had a better operational efficiency, the time in which the object achieved a velocity of 0 m/s was shorter than the time achieved by the Ardupilot autopilot, and the difference was equal to about 3 s.

In conclusion, choosing the Ardupilot autopilot is more beneficial for users looking for a simple and quick way to operate any object. This does not require an in-depth understanding of the design of the autopilot structure and also consumes significantly less time for preparation, tuning PID controllers, and executing flights on the respective object. There is a ready-to-use algorithm available for autotuning objects with the Ardupilot autopilot. On the other side, the autopilot Veronte is intended for users looking for a comprehensive understanding of the control algorithm for a specific object. It is also a preferable choice for users whose aim is to certify the autopilot and expect a high level of reliability from the autopilot system.

Overall, the SIL simulations provided a comprehensive platform to objectively evaluate and compare the performance of commercial autopilots. Future work may include more complex scenarios and extend the comparison to additional autopilot systems, providing a broader understanding of their capabilities and limitations in real-world applications. Furthermore, the comparison of specific autopilot systems in SIL and real flights could be conducted, and those results could show the differences between the model of the UAV and the real object and also evaluate the control algorithms on real hardware.