A Novel Gripper Prototype for Helical Bird Diverter Manipulation

Abstract

Energy grids represent a fundamental infrastructure of any country. These structures consist of many kilometres of power lines that must be periodically inspected and maintained. Among the necessary operations are installing and removing bird diverters to reduce bird strikes on power lines. These devices are intended to improve birds’ detection of power lines and reduce the risk of collision. Often, the installation and removal of bird diverters from power lines is accomplished by humans operating from helicopters or directly on the power lines. Apart from the considerable cost of these operations, working in elevated environments creates human safety risks. To reduce these risks, this paper proposes a novel solution to automatize these tasks. The proposed solution is a prototype gripper that can be mounted on unmanned aerial vehicles (UAVs) and remotely operated to install or remove bird diverters. This work presents a mechatronic device and software architecture system that is experimentally evaluated in a laboratory mock-up, which consists of a manipulator equipped with the proposed tool for removing a bird diverter. Future work is needed to deploy the proposed tool on a UAV.

1. Introduction

Overhead electricity transmission and distribution lines are expanding due to increased human population and energy demand. Power lines can create high risks for a wide range of bird species, affecting their habitats in breeding, nesting, and wintering areas [1]. Bird mortality caused by power lines is considerable in areas rich in avifauna for species with particular habits (e.g., eagle owl) or in situations favourable to the transit of migrants. Bird accidents have been estimated to affect about 350 bird species [2] and bird mortality caused by power lines affects 1 million birds per year in the Netherlands, 175 million per year in the United States [3], and around 1 billion per year worldwide [4]. Apart from bird deaths, collisions with power lines also present a problem for power companies due to potential outages following collisions [5]. For this reason, a set of devices known as bird diverters has been designed to make power lines visible to birds and keep them away.

Many bird diverters are available on the market. Some common bird diverters are depicted in Figure 1 including the clip-type diverter (left) and the helical bird diverter (right). These devices can be mainly classified into active (Figure 1 (left)) and passive (Figure 1 (right) diverters. An active bird diverter is composed of a part moved by the wind. One group of passive bird diverters are helical objects made of plastic or aluminium elements that must be wrapped around the power cable (see Figure 1).

Bird diverters are traditionally installed by highly specialized human operators, who reach the lines with aerial platforms, helicopters, or forklifts. These operations can be costly and dangerous because of the high heights (15–50 m). In addition, when the power line is not energized, operators can be in contact with electrified conductors. This requires additional safety protocols and conductive suits or insulated tools, further complicating the task.

In this work, we only consider the manipulation of helical bird diverters among the different kinds of diverters since their installation and removal processes are intrinsically complicated. This process can be split into the following steps.

• Slip the larger end of the bird diverter onto the conductor or strand (Figure 2a).
• Rotate the bird diverter until the first helix lies naturally along the power line (Figure 2b).
• Wrap the securing helix around the power line (Figure 2c).

In this context, to remove the diverter these steps must be executed in the opposite order.

Besides the installation process, bird diverters also need maintenance and periodic substitution. Bird diverter removal also presents difficulties. This paper proposes a novel mechatronic solution to carry out the removal process of helical bird diverters.

The envisioned solution considers an autonomous unmanned aerial vehicle (UAV) transporting a gripper that is designed to remove bird diverters. The UAV is teleoperated to move close to the operation site, where it can autonomously detect the helical bird diverter on the power line. This information is, in turn, used to detect the correct grasping point and start the automatic removal procedure. In this way, the human operator can safely stay on the ground while supervising this autonomous activity and bring the UAV back to land once the removal operation is concluded.

This work’s main contributions are the presentation of the designed prototype (Figure 3) and the machine learning algorithm to detect the bird diverter’s presence and orientation to plan the gripper motion during the removal process. This paper shows the preliminary results in a laboratory environment considering the gripper mounted on a standard robotic arm, whereas the power-line grid is emulated through a proper mock-up. In future work, a UAV should be used to continue the development process.

The remainder of the paper is organized as follows. Section 2 presents an overview of similar systems. Section 3 discusses the proposed prototype. In Section 4, the overall system architecture is presented and in Section 5, the system is evaluated in a realistic industrial environment. Finally, Section 6 presents the conclusions and future directions.

2. Related Works

Bird diverters are intended to make aerial lines visible to birds. To potentially improve effectiveness, they can have different colours that alternate on the three wires of the power line [6] (see Figure 1). Recently, the interest in aerial systems for installing such devices has grown. In general, aerial manipulators [7] are used to perform inspection and maintenance tasks in difficult or dangerous sites, such as oil and gas pipelines [8,9,10], chemical plants, or bridges [11,12], rather than the installation of devices, valve turning, and other maintenance operations [13,14,15,16]. Several platforms can transport massive payloads such as industrial arms or similar [17,18]. Related to our domain, the use of UAVs to preserve power grids has been discussed in different works. For example, in [19], the authors present a system that is able to land on power lines using a LIDAR sensor. In contrast, in [20], a pneumatic system is presented to deploy devices on power lines. Of course, considering the shape of helical bird diverters, the latter approach is unsuitable for the discussed application.

Another common type of diverter is based on a clip mechanism. The installation of this kind of bird diverter has already been tested in other works. In [21], an aerial robot is used to install the clip-type device, whereas in [22], a dual-arm lightweight system is presented to install helical diverters.

For helical bird diverter removal, a proper gripper mechanism must be designed. The literature on the gripper design and grasping mechanism varies from simple claw configurations [23] to more complex prototypes [24,25]. A comprehensive review of these technologies can be found in [26,27]. However, two arms are needed due to the movements needed to manipulate the helical diverter (see Figure 2). In this context, the most advanced technology is represented by the LiCAS A1 [28] dual-arm manipulator, a lightweight and compliant dual-arm manipulator [29] characterized by 4 degrees of freedom for each arm. This system has already been exploited to manipulate a helical diverter [22]. However, with respect to the latter work, our solution is simpler since no complex trajectory must be planned. In addition, in the above-mentioned work, the authors do not present the entire installation process of the diverter but only the early stages, excluding the more complex part of the handling, which, as pointed out by the authors, may require modifications to the clamps.

The main difference between the above-mentioned and proposed approaches is that, in our system, the manipulation step requires only one grasping point placed on the extreme side of the helical diverter. This is made possible thanks to the capacity of the tool to perform continuous rotations around the electric cable to install or remove the diverter. Moreover, we tested the proposed device in a laboratory mock-up with a fixed industrial arm instead of a real aerial manipulator. However, the use of this type of system to perform aerial manipulation and human–robot interaction for the installation of bird diverters has already been investigated in [30].

Finally, visual feedback is used to detect the helical tool to enable the autonomous execution of the manipulation task. In particular, the YOLO (You Only Look Once) V4 algorithm [31,32] was used to process the vision data received from an Intel Realsense depth sensor. YOLO is a widely used algorithm, that is used mainly for object detection. In this work, we properly train the YOLO algorithm to recognize the helical diverter.

3. Bird Diverter Manipulation Device

This section describes the mechanical design of the bird diverter manipulation tool.

Overall, the main idea behind the tool designed in this work is to use a single rotational mechanism to control the diverter and wrap the power cable using an integrated system. In this way, the complexity behind the manipulation of the helical diverter shown in Figure 2 can be overcome. Any other Degrees of Freedom (DoFs) needed to finalize the manipulation task (i.e., the linear motion needed to flow over the power cable) can be achieved using an aerial system. The designed gripper consists of one DoF.

The mechanical description of the gripper is presented below. The proposed prototype is composed of 41 standard mechanical and electronic components. Eleven customized parts were designed and realized with 3D printing systems, using PET-G material to guarantee the device’s integrity during the manipulation task. The overall size of the prototype is ($255\times 292\times 80$) mm and its weight is about $3.1$ kg.

The device’s mechanics were designed on a worm-toothed wheel to guarantee high torques at low speeds and a guillotine element to hold the diverter in the toothed wheel. All the internal components of the manipulation device are stored in an external chassis composed of three main parts: one upper, one lower, and one rear. The first two pieces contain circular housing to store the toothed wheel. Similarly, a circular opening of 30 degrees allows the electric cable to be grasped by the device, as shown in the left of Figure 4.

The internal housing of the device chassis was designed to store the motor and the driven shafts. Its assembly is depicted on the right of Figure 4.

In this context, each shaft comprises a central part, which is 16 mm long, and two extremities, which are 12 mm long. The supports to lock the rotation of the worm screw and hold the pulley are located on these components. The first component that is fit into the shafts is the worm screw. Moreover, the two shafts have two opposite screws to allow two rotations, one right-handed and one left-handed. At their extremities, there are two spacers with diameters of 20 and 18 mm, respectively, followed by ball bearings with a dynamic load rating of 6800 N and a maximum rotation speed of 22,000 rpm. These are the unique elements that interlock with the inside of the container and anchor the shafts and other mechanical elements. The last spacer and the pulley are fit after the ball bearing. The pulley is made up of 25 teeth for belts of a maximum thickness of 15 mm. A cavity also forms the key between the shaft and the pulley. The linear joint connecting the 12 mm motor shaft to the 6 mm electric gear motor is the last element that is keyed on the motor shaft. The two shafts rotate at the same speed thanks to the “pulley-belt” mechanism positioned in the internal housing in the back part of the device. The strap has a circumference of 525 mm and comprises 105 teeth.

Since the distance between the two shafts is smaller than 525 mm, a PET-G belt tensioner with ball bearings was created to favour the correct functioning of the “belt-pulley” mechanism. Two worm screws parallel to the sides of the toothed wheel are used to ensure the complete rotation of the wheel even when the 30-degree opening corresponds with one of the screws. In this case, the rotation movement will be momentarily supported only by the other screw, as shown on the left of Figure 5.

Furthermore, to facilitate the wheel’s movement, five ball rollers are inserted in each container, with an allowed load of 2 N each, which also perform the task of distancing the wheel from the containers, reducing the friction between the two moulded components in the PET-G.

The toothed wheel, with a diameter of 150 mm, is equipped with an opening of about 30 degrees to facilitate its placement on the cable to make the wheel’s rotation axis coincide with the axis of the cable. The internally hollow toothed wheel is divided into a container and a cover. It is externally characterized by teeth, the number and dimensions of which are calculated with reference, in particular, to the mechanical characteristics of the worm screw, that is, 64 teeth (see the right of Figure 5). An additional opening was also made to facilitate the entry of the diverter into the wheel. This opening has an egg-shaped maximum size of $36\times 26$ mm.

On one side of the wheel, there is a protrusion (Figure 6), which performs the levering work of the diverter during the initial phase of the removal process. The force impressed on the diverter was estimated to be approximately 58 N, and the gear motor was calibrated accordingly, considering the gear wheel–worm gear ratio and the friction of the various components.

Inside the container is a mechanism called a “guillotine” which, through linear guides fixed with screws, is operated by a linear actuator. The vertical movement of the component locks the diverter into the device, ensuring a rigid grip. The linear actuator pulls the guillotine towards itself while the two arms on the left, by inertia, move inside the compartment, blocking the diverter (see Figure 7). This mechanism is designed for diverters that do not exceed 12 mm in diameter. A protrusion was added to the wheel, which performs the lever work on the diverter during the rotation of the toothed wheel.

The rear casing is designed to incorporate electrical circuits that control the device. A protrusion is dedicated to the connection between the proposed device and the manipulator (i.e., a Kuka IIWA manipulator, in our case, see Figure 8 left, but it can be adapted to drones in the future). External to this cover, there is also the housing for the gear motor driver and, on its opposite side, the space needed to fit the microcontroller devoted to the system control. The goal of the microcontroller is to actuate the gripper, considering the input provided by an external program. To this end, the controller is connected to a companion computer via a serial communication bridge and receives the commands to activate/deactivate the stepper and the motor drivers. More information about the control program is provided in Section 4.

Finally, the gripper needs to be powered by a 12-volt power input to start the onboard microcontroller and actuate the internal mechanism. For this, a standard four-cell LiPo battery (which generates a nominal output voltage of 14.8 V (4 × 3.7 V) can be used to power both the avionics of the UAV and the gripper system. The complete prototype is shown on the right of Figure 8.

This device was designed for diverters that can be installed on a cable with a maximum diameter of 11 mm. This limitation mainly depends on the size of the toothed wheel. However, this was sufficient to demonstrate the functionalities of the proposed prototype.

4. System Architecture

The overall system architecture is depicted in Figure 9.

The system’s input is represented by the coloured image and the depth matrix generated by the Intel 435d depth camera. The data generated by the sensor are used by the Image Elaboration module that receives the coloured image matrix from the depth sensor and uses the YOLO library to identify the helical bird diverter. The YOLO algorithm uses neural networks to provide real-time and accurate object detection to identify an object and its position in an image. The YOLO algorithm uses convolutional neural networks (CNNs) to detect objects in real time, and the algorithm requires only a single forward propagation through a neural network to detect objects. This means that the prediction of the whole image is performed in a single execution of the algorithm. The Image Elaboration module additionally receives the depth matrix to convert the pixel points into 3D coordinates. To allow the identification of the grasping point on the diverter, the YOLO algorithm was trained to recognize the whole diverter (to assess its presence in the image) and two distinct parts on it, the LD (Line Diverter) back and LD front parts. The output of the trained system can be seen in the top image of Figure 10.

In the proposed case study, additional information is extracted from the diverter, exploiting the output of the YOLO library. In particular, this work considers a case in which a diverter that is already placed on the power cable must be removed from the line. In this context, two additional pieces of information are needed: the grasping point of the diverter and its orientation on the cable. For this reason, the output of the YOLO algorithm (i.e., the detected object and the region of the image in which it is contained) is used to threshold the image in order to distinguish the background from the diverter (see bottom-right of Figure 10). Then, the pixels belonging to the diverter are converted into 3D points and interpolated to obtain a line, depicted in pink in the bottom-left image of Figure 10. Starting from this line, two pieces of information are extracted. First, the extremity of the tool (at the other end of the giant spirals) represents the point where the gripper must be positioned at the start of the removal task ($p_{gp}$). This point is indicated by a yellow dot on the far left of the bottom-left image of Figure 10. The other point indicates the orientation of the diverter, and it is necessary to calculate the correct orientation to insert the cable in the guillotine mechanism. Overall, these data are represented by the inclination of the interpolation line with respect to the three Euler angles. Finally, the steps between the last two spirals of the LD front part are detected for future usage (yellow and blue dots in the bottom-left image of Figure 10).

Therefore, the output of the Image Elaboration module is represented by the objects detected in the image and their characteristic points (i.e., the grasping point and the diverter orientation). These data are sent to the Planner and Gripper Control module, which commands the manipulator and the gripper to accomplish the manipulation task. In this context, since the 3D coordinates of the salient points with respect to the depth camera are known, they can be converted into the manipulator base reference frame. With this information, the diverter grasping point on which the device acts is calculated. A combination of manipulator and gripper motions is used to remove the diverter, as described in Section 5. Moreover, the interface between the gripper and the system is established via a serial interface, which allows for communication between the companion computer and the microcontroller installed inside the gripper. Here, information, such as the guillotine tool’s desired actuation and the gripper’s overall state, is exchanged.

Finally, to manipulate the bird diverter, the manipulator must be actuated. The robotic manipulator employed for the experimental evaluation of the prototype is an LBR Kuka iiwa manipulator. It has seven joints and coupling torque sensors integrated into all the axes. The Kuka iiwa manipulator is modular and programmed using Java. The robot control is implemented using Kuka FRI libraries to directly control the desired position of the robot joints, solving the direct and inverse kinematic problems on the companion computer. Since the considered application requires that the gripper centres the power line cable with its guillotine mechanism, a force controller was implemented. In this way, the estimation of the pose of the diverter and, consequently, the power cable is imprecise. Physical contact with the cable will drive the mechanism into the correct housing. For this reason, the Admittance Control module implements a Cartesian admittance controller [33]. This controller is implemented considering the forces acting on the robot end effector. The dynamic relationship between the applied forces and the motion of the robot is established using the following admittance control scheme:

$$M_d(\ddot{X_c}-\ddot{X_d})+D_d(\dot{X_c}-\dot{X_d})+K_d(X_c-X_d)=F_c$$

(1)

where $M_d$, $D_d$, and $K_d$$\in {\mathbb{R}}^{3\times 3}$ are positive definite diagonal matrices representing the desired virtual inertia, damping, and stiffness, respectively. Following this formula, the output of the controller is the compliant position command $X_c\in {\mathbb{R}}^3$ for the end effector of the robot, given a destination $X_d\in {\mathbb{R}}^3$. Here, the input of this module is the Cartesian position $X_d$ selected by the Planner module and the estimated force acting on the end effector ($F_c\in {\mathbb{R}}^3$). This module is also in charge of solving the forward and inverse kinematic problems. In this context, this module receives the desired position in the operative space to convert them into the desired joint angles for the manipulator. At the same time, the pose of the end effector is calculated, starting from the current joint values of the arm. The core of this module is based on the well-known Kinematics and Dynamics Library (KDL).

5. Experiments

In this section, an autonomous bird diverter removal experiment is conducted to assess the effectiveness of the proposed prototype. Finally, the evaluation of the helical diverter detection system based on the YOLO algorithm is presented.

A sketch of the overall task is shown in Figure 11. In this context, the trajectory planned and executed during the task is indicated by a blue line. The estimated pose of the diverter is shown in front of the industrial manipulator.

The detailed process was carried out by the actuation of the manipulator and gripper. It is worth noting that to install/remove the diverter correctly, two rotations are needed. In the first part, where the diameters of the spirals are small, a clockwise rotation is needed. In contrast, in the second part, where the spirals are large, the rotation needed to remove the diverter must be counterclockwise (see Figure 2).

The removal process was split into the following steps, as shown in Figure 12. After the object was detected on the scene, the cable was approached. The approaching point was selected as an elevated point over the grasping point and detected thanks to the image elaboration algorithm described in the previous section.

In this context, the gripper was positioned vertically on the cable. Then, the opening of the tool was inserted into the cable (Figure 12a). At this point, the diverter was ready to be removed and the first rotation of the gear wheel of the gripper was performed (Figure 12b). During the wheel rotation, a linear motion of the manipulator towards the large spirals was planned and executed, continuing the rotation of the gear wheel. Thanks to the admittance controller, if the helical diverter part was not removed yet, the manipulator held its position (Figure 12c,d). Since the large part of the diverter was removed, the end effector was inclined with respect to the cable (Figure 12e) and then the guillotine mechanism was closed to grip the diverter. However, at this point in the task, only the large spirals fit into the cable. So, a counterclockwise rotation was actuated on the toothed wheel (Figure 12f,g). Finally, a vertical motion was executed to complete the task (Figure 12h).

To move the manipulator, some waypoints were generated starting from the pose of the diverter detected by the depth sensor. Therefore, a trapezoidal trajectory was planned from the waypoints. During the execution of the trajectories, the admittance controller was properly tuned so that the gripper was not forced until the next part of the diverter was unlocked from the cable.

Figure 13 shows the pilot case study for removing the helical bird diverter. During the same experiment, we recorded the Cartesian position and orientation of the end effector (Figure 13 and Figure 14) and estimated the forces acting on the robotic gripper (Figure 15). The cable was placed parallel to the x-axis of the robot’s reference frame in this experiment. Hence, the first motion of the robot was to reach the power line, as shown in the first graph in Figure 13. Then, the second graph in Figure 13 (the motion along the y-axis) shows the device’s progress over the cable. Similarly, the graphs in Figure 14 show a single change in the orientation of the end effector during the final step of extracting the diverter. In this context, the yaw angle of the end effector did not change since it was already aligned with the final orientation of the task.

Finally, the forces involved in the removal process acting on the robot’s end effector are shown in Figure 15. The force graphs are mainly characterized by peaks in contact with the cable and during some phases of the removal task, in particular, the removal of the widest spirals of the diverter.

Object Recognition Evaluation

To allow for the recognition of the helical diverter, the YOLO version 4 algorithm was adequately trained. Several pictures with different sizes and light conditions were taken in this context. In particular, 120 pictures were considered for the system training and the GoogleColab [34] hardware accelerator was used. The diverter and its subparts were labelled in each image, namely the diverter, LD front, and LD back.

The trained network was tested on an adequately made test set composed of 400 images, in which $80\%$ of the images showed the diverter placed on the floor or installed on power lines, whereas the other $20\%$ of the images showed power line cables without bird diverters. To assess the performance of the recognition system, the accuracy percentage, ${\alpha }_a$, was calculated considering the following formula:

$${\alpha }_a=\frac{T_P+T_N}{T_P+T_N+F_P+F_N}$$

(2)

where $T_P$ is the true positive, $T_N$ the true negative, $F_P$ the false positive, and $F_N$ the false negative. These values were calculated as the percentages of times that the detector correctly identified the diverter in an image. The results are shown in

Table 1. Object detection performed with YOLO version 4 evaluation.

Object	Accuracy	TP	TN	FP	FN
Diverter	0.87%	0.93%	0.81%	0.06%	0.18%
LD Back	0.92%	0.90%	0.93%	0.09%	0.06%
LD Front	0.84%	0.87%	0.81%	0.12%	0.18%

, where the accuracy percentage was calculated for all the recognizable elements from the YOLO library.

The results reported in Table 1 demonstrate that the vision system was able to detect both the whole diverter and its parts, allowing the control system to exploit the information about the diverter orientation (i.e., detect the grasping point of the diverter).

6. Conclusions

This work presented a novel robotic gripper to manipulate a helical bird diverter. These devices prevent collisions between birds and power lines and, consequently, the death of flying animals. Due to their complex shapes, the installation and removal of these tools are not trivial and are performed by human operators working in dangerous and critical conditions. Automatizing this task using an aerial manipulator represents an important step in improving the safety of human operators performing such tasks. For this reason, the structure of the diverter and the method of manipulating them have been analyzed. We proposed the mechanical design of a gripper for the removal task of a helical bird diverter and an automatic system to detect a diverter on power lines and reconstruct its position in the space. We integrated this into an industrial manipulator for preliminary testing. The proposed evaluation case demonstrated the effectiveness of the proposed gripper. Concerning the limitations of this work, it is worth noting that the proposed case study takes into account only the unhooking of the helical tool from the cable without considering the effect of the generalized forces involved in the real task, i.e., the effect of the diverter manipulation on a floating base. However, the current technology of UAVs allows for the development of an aerial system with a high payload and tiltable rotors to interact with the environment, impressing major forces on the interaction surfaces [35,36]. In addition, specific low-level controllers for aerial vehicles can be implemented to reject external disturbances such as the wind during the execution of the task [37,38].

In future work, the design of the gripper must be improved to decrease its total weight. At the same time, the gripper must be tested in a real-world test bed using an aerial manipulator.