Automatic Jig-Assisted Battery Exchange for Lightweight Drones

Abstract

Drones utilize rechargeable batteries as a power source. Operating a drone requires human interaction with the exchange or recharge process of these batteries. This can provide limits for drones, which is why some use cases for drones could benefit from automated battery exchange. The purpose of this work was to research how a robotic arm could be used to automatically exchange flight batteries in flying drones without interference from operators and create a proof-of-concept system for evaluation. The devised method was based around a separate jig, which would hold the drone in place during the exchange operation, where battery exchange itself was handled by the use of a robotic arm. The constructed prototype could exchange flight batteries in drones in under two minutes with reasonable perpetuity and reliability.

1. Introduction

Unmanned aerial vehicles (UAVs) are types of flying drones that do not have a human operator on board. Such drones have found various possible use cases in multiple fields, from aerial photography to the delivery industry and even medical services [1,2]. Drones require a power source to operate. A variety of different methods of powering drones have been studied extensively regarding aspects of endurance and efficiency. While drones with tethered power sources do exist, drones often rely on portable power due to their remote controlled nature. Current drones often utilize removable batteries as a portable power source and can be equipped with additional payloads, both of which are usually manually installed by the pilot or other human operator. As drones can, by design, operate far away from their operators, the manual replacement of a battery by an operator can be a limiting and ineffective method that sets limits to the drone and its operational purpose. Therefore, a system designed to automatically and remotely handle these tasks could potentially benefit existing and emerging use cases for drones.

Research on different autonomous flight-time-extending solutions for drones is ongoing, as drones remain a popular topic across different industries. Examples of methods for automatic flight battery exchange include designs such as that of Lee et al., where the battery can be exchanged with a mechanical carriage, if the drone has successfully landed in the correct position [3]. Their system could exchange a flight battery in around one minute. However, it relies on the landing accuracy of the drone, which could be problematic when wind or other natural factors disturb the landing process. In their work, Fujii et al. [4] present a system where a landed drone could be mechanically guided in place before swapping the battery to compensate for inaccurate landings. Alternatively, Zhao et al. [5] disclose the use of a funnel and its counterpart attached to a drone for guidance before battery exchange. Other methods which could be more tolerant for inaccuracies in landing include methods with wireless or contact-based charging areas with guidance, but these can usually accommodate a singular drone for the duration of charging time [6,7,8]. Some solutions have emerged that have attempted to address both issues with mechanical containers for drones, often dubbed drone-in-a-box solutions, which can store a drone and provide either charging capabilities or a battery exchange for a singular drone, where they often rely on mechanical positioning of the drone in the landing area [9,10]. Similarly to those, some recent methods have utilized machine vision locating of drones and an extendable arm for exchanging batteries [11]. Using such an open-chained solution has created a possibility to house multiple batteries and possibly even service multiple drones while not relying on mechanical positioning systems or the pinpoint landing accuracy of the drone.

Table 1. Comparison of range-extending methods.

Method	Advantages	Disadvantages
Wireless charging	Lightweight	Services a singular drone while charging
	Simple	Relies on charging time
Contact-based charging	Lightweight	Services a singular drone while charging
	Simple	Relies on charging time
		Requires a guidance solution
Tethered drone	Does not rely on charging time	Limits the range of drone
Battery exchange, guidance by a drone	Does not rely on charging time	Relies on landing accuracy
	Less reliant on charging time	Complex
Battery exchange, guidance by an external solution	Does not rely on charging time	Complex
	Does not rely on landing accuracy
	Less reliant on charging time

summarizes the advantages and disadvantages that could be identified from methods to extend the flight time of drones.

This work presents a prototype system built around a robotic arm and a simple machine vision system to recognize and move the drones through a separate battery exchange jig, which can be used to store drones when not needed and hold them in place during the exchange process. Our system’s novelty and contribution can be summed up as follows: instead of using a more complex mechanisms that hold the drone and manipulate the battery simultaneously, we divide those into separate functionalities resembling how you could exchange the battery while holding it by hand by using separate jigs as a tool.

Background for this work begun as a master’s thesis of one of the authors, which is publicly available online [12]. In an effort to keep this paper short, some of the more trivial technical aspects that are shared between these works are not covered in this paper.

2. Methodology and Construction

One of the common challenges in flight-time-extending solutions for drones is the problem of accurately positioning the drone for a recharge or battery exchange. Our attempt to overcome this challenge was to locate the drone with machine vision and physically move the drone from the landing area to a separate battery exchange location with a robotic arm, which would also handle the battery exchange operation. Our presumption was that this would not require pinpoint accurate landings from the drone itself as the drone would only have to land in the general field of vision of the robot’s camera for the localization to work. Therefore, this concept would be central to the built system’s design.

The proposed system (Figure 1) was designed and constructed from sub-assemblies attached to an aluminum support frame. The Kinova Gen3 robotic arm was placed in position where it could reach the drones in the entirety of the landing area while maintaining a capability to operate within preset static points of interest (POIs). These POIs consisted of positions which would not move during operation and included the battery exchange jig, the battery chargers and a separate storage position for the drone. The construction also included a peripheral monitoring system for keeping track of occupied drone positions, a landing area to function as a good background for machine vision operation and for drones to land, and a support frame constructed from aluminum profile to fasten everything down. A list of the main parts in the final construction and their availability can be seen in

Table 2. List of main parts.

Part	Description	Availability
Kinova Gen3 7DOF	Robotic arm	Commercial
Robotiq Hand-E	Mechanical gripper interface	Commercial
RealSense L515	Camera	Commercial
Minitec 45 × 90 UL 4800 mm	Support frame material	Commercial
Corrugated plastic panel 598 × 598 mm	Landing area material	Commercial
DJI Tello drones products	Drones, batteries and chargers	Commercial
Gripper fingers	Gripper fingers fitted for Tello	3D printed
Exchange & storage jigs	Holding drones	3D printed
Miscellaneous parts	Funnels, custom fasteners, supports	3D printed
Actuonix PQ12-P	Linear actuator in exchange dock	Commercial
Microswitch D2VW-5	Used in battery exchange & storage dock	Commercial
STM32 Nucleo L433RC-P	Controller for microswitches	Commercial

.

The drones utilized in this project were DJI Tello drones equipped with a gripping surface and augmented reality (AR) marker. During a normal, manual battery exchange, the Tello drone is held between a person’s fingers (Figure 2, left) while the flight battery is removed by pulling it from the rear of the drone, and it is reinstalled in the opposite direction. A similar approach was used as a starting point for our automatic exchange, where the drone would be mounted on a jig instead of a hand.

A 3D-printed jig (Figure 2, right) was designed to attach directly to the feet of the Tello drone through the holes in the jig. These holes function similarly to a funnel and position the feet of the drone accurately when they are correctly pushed in. Once pushed in, movement of the drone is restricted, except for the same direction in which it was pushed in. In order to make sure that the feet, and by extension the drone, are pushed in the correct position and orientation, chamfered poles were designed next to each hole. These poles would act as preliminary guides and would also provide additional support during the manipulation of the drone. The jig also provided attachment points for a microswitch, which was used to detect the correct installation of the drone, and a small linear actuator, which could be used to toggle the power on the drone by pressing the power button of the drone.

The battery exchange jig was mounted in such a way that the robotic arm could reach partially underneath it in order to grasp the flight battery from the rear of the drone. As the movement of the drone is bound in this direction by the jig, the battery could simply be removed by pulling it away from the drone with the robot. For safety and monitoring purposes, a microswitch was placed on the jig as a fail-safe to detect the correct installation of the drones. This microswitch would not trigger unless the drone was installed correctly. The storage position was a simplified version of the exchange jig assembly (Figure 3), but without the linear actuator, as it was unnecessary.

Moving drones to a separate jig for battery exchange required a method of locating drones accurately from the landing area and a method to hold both the batteries and the drones. We used a Robotiq Hand-E gripper, which was fitted with a photogrammetrically calibrated RealSense L515 camera and proprietary gripper fingers with notches suitable for grasping both the batteries and the drones (Figure 4).

The machine vision system for locating the drone from the landing area used AR markers attached to a visible location on the drones. From these markers, the position and orientation of the drone in the landing area could be calculated by measuring the difference between the center point of the AR marker on the drone and the center point of the visual frame of the camera. This solution relied on the method introduced in the OpenCV library for Python [13]. The localization worked by driving the robot in a position where the arm of the robot would be perpendicular to the planar landing area and the attached camera would have a good field of view (FOV) of the landing area. This is illustrated in Figure 5, where FOV X and FOV Y resemble the edges of the FOV of the attached camera, and distance D resembles the height from the arm to the landing area. This height was assumed to be static as long as positions of the robot or landing area did not change, which eliminated the requirement of measuring it every time a drone was picked up.

The location of the AR marker in the frame of the FOV could be obtained using the detector feature built into OpenCV 4.2, yielding the positions of corners of the detected marker, which could further be converted to Euler angles with the pose estimation tools built into OpenCV. This method provided distances from the AR marker to the corner of the visual frame and rotation along the Z-axis of the marker. The difference between the AR marker and the center point of the camera could then be converted to Cartesian coordinates and angular values, which could further be passed to the robotic arm as corrections toward the pick-up point of the drone (Figure 6). Detection in the planar field yielding values for X and Y corrections and the angle of orientation was deemed sufficient, as the distance between the base of the robot and the top surface of the landing area, and by extension the height of the landed drone, was known.

Since the camera would already be used to locate the drones, it was also used for monitoring the status of the battery chargers. In this case, the chargers were a set of two chargers made for DJI Tello batteries, each capable of charging three batteries. The status of each charger position was determined by monitoring the position and the LED next to it with the camera, indicating the presence of a battery and readiness of the charge. The middle position in the battery chargers was too narrow to access it with the gripper and was therefore utilized as an attachment point to fasten the chargers to the support frame. This resulted in the system having a total of four possible charging positions. The unavailable middle position would also be used for attaching funnels directly above the remaining charger positions, which would guide batteries to the correct positions. As the system could identify empty slots and readiness of a charge in occupied positions, a simple pick-and-place action was deemed to be sufficient when returning or fetching batteries from the chargers.

As the points of interest were static, apart from the action to pick up the drone, movements of the robotic arm were created with positional sets containing Cartesian coordinates with angle/axis representations and angular position values. Cartesian coordinates were used in movements requiring finesse, such as the insertion of the drone to the exchange dock and the manipulation of the flight batteries within the drone, as Cartesian coordinates were easier to use in this application. Broader, more general movements, such as movements from one safe point above the exchange dock to another safe point above the drone storage dock, were made with angular position values to avoid unreachable intermediate points and high-joint-rate near-joint singularity in Cartesian paths.

The correct operation of the system was monitored with a separate peripheral monitoring system. This peripheral monitoring system consisted of an STM32 micro-controller that controlled an array of multiplexers that were connected to the microswitches in the storage and battery exchange jigs. The STM32 transmitted changes in the statuses of the microswitches to a main computer responsible for running the system. This main computer was a laptop computer responsible for monitoring data and controlling actions using Kinova’s Kortex application interface version 2.6.0 for robotic movements and the aforementioned OpenCV 4.2 for machine vision purposes.

Three modes of operation were programmed into the system (Figure 7). These modes were essentially looping sets of commands and checks based on parameters, such as the statuses of the battery chargers or the occupation of the drone docks. These modes were run in a loop until certain conditions were no longer met or the operator interrupted the program. The modes were called ‘Catch’, ‘Release’, and ‘Catch and release’. ‘Catch’ was used to find the drone in the landing area, remove its flight battery, and move the drone to storage. ’Release’ was the opposite mode, where the drone was moved from the storage to the landing area with a charged battery. ‘Catch and release’ was used to exchange the flight battery for a charged battery and immediately release the drone back to the landing area.

In ‘Catch’ mode, the system would check if any storage positions were available, and if they were, it would begin to look for AR markers from the landing area using the configured detection model (Figure 8, left). If any markers would be detected, the system would move to a better position for scanning the landing area for the position of those markers and, by extension, the drones (Figure 8, right).

If a marker was found and correction coordinates were successfully extracted, the arm would move to pick up the drone from the preset pick-up height (Figure 9 left) and transport the drone on top of the battery exchange dock (Figure 9, right).

The drone could then be inserted into the dock by pushing it in from above (Figure 10, left). If a correct installation was detected via the activation of the microswitch, the robot would move into position to remove the battery from the rear of the drone (Figure 10, right).

Once the battery was removed (Figure 11), the robot would then move to a position where the camera could see the battery chargers. The system would then attempt to detect the statuses of all battery chargers and insert the battery to the first empty charger position for charging. After leaving the empty battery at the charger, the robotic arm would move to pick up the empty drone from the exchange dock. The empty drone would then be transported to the storage position that was deemed empty in the first phase, and the loop began over again until all storage positions, which in our case comprised of just one actual storage position, were full. The ‘Release’ mode was configured to be the opposite of ‘Catch’, releasing drones from storage. The ‘Catch and release’ mode would insert a fully charged battery into the drone and then move it back to the landing area. By using static coordinates for the POIs, positions of the POIs did not change between different modes.

3. Tests

The system was tested by running the longest of the three modes, the ‘Catch and release’ mode, for 200 consecutive timed test runs. Tests were performed in an indoor laboratory with good lighting conditions, at around 20 degrees Celsius, and without wind or external vibrations. Between each run, the drone was manually placed by hand in various positions in the landing area. In every position, the orientation of the drone was set to be roughly between −90 and 90 degrees and generally facing the robot. The system also had three out of four possible battery charger locations occupied, leaving room for one battery from the drone. With this configuration, the location of the available empty battery position and the available fully charged batteries would rotate as the robot exchanged batteries between the drone and the chargers. Each test run was deemed successful if the drone was picked up from the landing area with an empty battery and released back to the landing area in correct orientation, facing away from the robot, with a charged battery and powered on, i.e., in a condition where it would realistically be ready for take-off.

During these 200 tests, the system failed this task five times. Two failures resulted from inaccurate readings of the battery charger LEDs, which caused the system to think that all battery charger positions were occupied, which in turn triggered an error state and halted operation. Three failures occurred from a poor pick-up accuracy of the drone, which led to a misalignment when inserting the drone to the battery exchange dock. However, it was also observed that the chamfered surfaces in the exchange dock managed to correct minor misalignments from poor pick-up action six times during all test runs by guiding the drone in the correct position. Therefore, even if the chamfered surfaces functioned as they were designed and provided correction when they were supposed to, the accuracy for the pick-up action would have been insufficient in a total of 9 times out of 200.

Durations of steps from all successful test runs were recorded with a Python 3.8.2 script, keeping log of when different steps were reached. The average time from the first detection of the drone to the robot returning the drone back to the landing area and moving back into the ready position was recorded to be 118.27 s, with most runs ranging roughly from 117 to 120 s. Average values of recorded times between steps are shown in

Table 3. Average times between steps.

Step	Average Time in Step (s)	Average Time from Start to End of Step (s)
Time to detect the drone	8.94	8.94
Time to locate the drone	5.75	14.69
Time to pick up the drone	25.96	40.65
Time to remove the battery	4.48	45.13
Time to move to the battery chargers	16.41	61.54
Time to deposit the empty battery & pick up the charged battery	11.43	72.97
Time to release the drone back to the landing area	39.73	112.7
Time to completion	118.27	118.27

. Test times were recorded with the highest available speed and acceleration settings for the Kinova Gen3 robot, which were 80 degrees/s and 297.9 degrees/s² for joints 1–4 and 70 degrees/s and 350 degrees/s² for joints 5–7, respectively.

Recorded times show that, on average, most of the time was spent on three steps. These steps were ‘Pickup the drone’ (25.96 s), ‘Move to the battery chargers’ (16.41 s), and ‘Release the drone back to the landing area’ (39.73 s).

The ‘Pickup the drone’ step incorporates the actions of picking up the drone from the landing area and moving it to the battery exchange dock, installing the drone in the dock, turning the power off in the drone using the small linear actuator in the exchange dock, and verifying correct installation of the drone with the peripheral system. As such, it is a fairly large step that could be considered movement-intensive. Therefore, the length of this step could likely be shortened by making movements of the robot either shorter or faster through optimization of the movements or via a different placement of the battery exchange dock. However, in the case of this system, these gains in performance would likely be close to negligible, as the movements were reasonably short in comparison to the speed settings of the robot.

The ‘Moving to the battery chargers’ step includes the short movement from the top of the landing area to above the battery chargers with the battery that was removed from the drone, as well as the measurement of the charger LEDs in each charger position. The latter spends five seconds measuring the frequency of the blinking LEDs in each occupied battery charger position to determine whether the battery in the charger position is charged or not. Therefore, with three batteries available in the chargers, this process takes at least 15 s in every test iteration, which is less than ideal. However, this was a placeholder solution, and while implementing a better solution could provide major gains in performance, it was considered to be outside the scope of this work.

The ‘Release the drone back to the landing area’ was another large step. It included transporting the charged battery to the drone, installing it in the drone, toggling the drone’s power on, removing the drone from the exchange dock, releasing the drone in the landing area, and finally looking for the presence of the marker to confirm that the drone has left the landing area. Similarly to the ‘Pickup the drone’ step, this is a movement-intensive step with multiple short movements. However, as the timing stops when the system can no longer detect the drone, with the drone was removed from the landing area by the operator in these tests, this step also relies on the speed of the operator removing the drone from the landing area. While the performance of the system in this paper could be increased by stopping the timing once the system considers the drone to be ready for take-off, the authors felt that this approach provided a more realistic scenario for testing.

Overall, for a system with similar functionalities, the speed of the system could potentially be increased by configuring it to require as few different steps of movement as possible and by conducting peripheral functions, such as battery monitoring, as parallel operations to the main system.

4. Conclusions

This article describes how an independent mechanical jig could be used to hold a drone in place during an automated battery replacement. As the drones were located with machine vision and handled by a robotic arm, the placement of a drone in the landing area could be flexible, allowing drones to land in the landing area with relatively loose precision. This was also tested during an evaluation by placing a drone in various positions and orientations in the landing area. Additionally, our system could be used to store and release drones using the same method. During the evaluation, the constructed prototype failed to exchange the battery 5 times out of 200 iterations, with the average time for a complete battery replacement being under two minutes. Compared to existing similar solutions, our method was neither the fastest nor the most reliable, as other comparable solutions could complete the flight battery exchange in even under one minute with near perfect reliability. Additionally, our setup was relatively complex and showed susceptibility to malfunctions during testing. However, our system offers the possibility for drones to land with relatively loose precision, and it could also be used as a storage for drones, which adds utility beyond battery exchange.

Therefore, the method of using independent jigs for handling drones during an automatic flight battery exchange shows promise as a concept but could benefit from additional research. The jigs could possibly be scaled up and redesigned for larger drones, as real-world use cases for DJI Tello drones are limited outside of research and hobbyist applications. Based on evaluations, other possible areas for future improvements include drones developed with such battery replacement methods in consideration, which could be beneficial for long-term operational reliability. It was observed that the localization of the drone in our system resulted in inaccurate pick-up actions, beyond what could be corrected three times, and pick-up actions within the margin of correction six times during testing. While the localization itself was likely the main cause, the mechanical guidance could possibly also be improved by design features such as centering gripping surfaces in drones for better pick up or by broader margins of guidance in the mechanical jig. Our system also took advantage of using static coordinates for non-moving POIs, which likely contributed to higher reliability once properly adjusted. However, using static coordinates may not be feasible in all use cases, as even a slight offset between the true POI location and its set coordinates can cause additional wear or even malfunctions due to the poor alignment of the robotic actions and the POIs.

Future development plans include designing scalable and modular features that could support various drone types to improve the overall usefulness of the system. The Tello drones win this study ere attached to jigs from their feet since they provided a sufficient support for the removal of batteries. However, using the same exact design may not feasible for larger drones that can require a greater surface area and a wider stance between their feet for stability. Scaling the jig to accommodate larger drones may require the development of separate attachment features for these drones, i.e., points of contact between a drone and a jig. These attachment features could possibly be designed to be universal between drones to increase compatibility. Similarly, batteries and battery housings could likely be designed in a way that would provide compatibility with different types of drones. Operational reliability could also benefit from having a method to recognize and correct for the misalignment of different POIs, i.e., a method of error correction instead of static coordinates, allowing for dynamic error correction rather that relying on static coordinates.