Motion Signal Processing for a Remote Gas Metal Arc Welding Application

Abstract

This article covers the signal processing for a human–robot remote controlled welding application. For this purpose, a test and evaluation system is under development. It allows a skilled worker to weld in real time without being exposed to the associated physical stress and hazards. The torch movement of the welder in typical welding tasks is recorded by a stereoscopic sensor system. Due to a mismatch between the speed of the acquisition and the query rate for data by the robot control system, a prediction has to be developed. It should generate a suitable tool trajectory from the acquired data, which has to be a $C^2$-continuous function. For this purpose, based on a frequency analysis, a Kalman-Filter in combination with a disturbance observer is applied. It reproduces the hand movement with sufficient accuracy and lag-free. The required algorithm is put under test on a real-time operating system based on Linux and Preempt_RT in connection to a KRC4 robot controller. By using this setup, the welding results in a plane are of good quality and the robot movement coincides with the manual movement sufficiently.

1. Introduction

A decrease of times and costs in manufacturing using arc welding poses a continuous challenge, as well as the demand to establish new production chains in a short period of time. To achieve this, multi-robot systems become increasingly widespread [1,2,3,4,5]. They allow reducing strain, stress, and fatigue on workers by performing pre-programmed steps of joining, machining and assembly. However, economical use of those robot systems is restricted to high lot sizes because of the high programming effort and the low reusability of robot program code. Particularly in arc welding, it frequently occurs that welds must be pre- and post-processed and that even small geometrical deviations lead to undesired results in terms of quality. In the repair domain or for lot size one, the application of a robot is not profitable because the time required for the setup and the programming of the robot usually exceeds the time the work takes place by a skilled welder. Here, the human skills of dexterity, flexibility, and decision-making [6] outperform the robot. Thanks to his long lasting experience, the human welder is able by vision and the sense of hearing to take smallest deviations of the welding process into account and adjust for e.g., angle of attack, current, distance, feed rate, and movement. However, the welder is exposed to hazards, such as voltage, fumes, heat, radiation, and noise. Further on, detrimental long-term-effects on the musculoskeletal system as well as on internal organs are diagnosed regularly [7,8,9] on welders of higher age adding to illnesses or leading to the inability to perform any physical labor at all. In order to allow welders of higher age to continue working as well as to increase the job attractiveness for possible welding trainees, collaboration between a human welder and robot system is necessary, combining the particular strengths of both. Consequently, a lot of research and development is taking place to achieve such solutions. Erden et al. [10] introduce a system that relieves the worker from the weight and the torque of the welding torch by using a force–torque sensor attached to the welding tool. The sensor registers the hand movements and controls the robot. Forces and torques are directly fed back by each degree of freedom of the sensor in Cartesian space and regulated to zero. Consequently, the robot follows the motion of the hand. Undesired motions caused by, e.g., tremor, are suppressed by the inertia of the robot and the limited bandwidth of the control loops. In [11], this system is extended and a test series with experienced and unexperienced welders is performed. The results show that unexperienced welders have been able to significantly improve their results by using this robot system. However, the welder is exposed to welding specific hazards such as plasma arc and fume radiation. To overcome these expositions, this input principle has been used for a remote control in [12]. The operator uses a space ball to control the robot in the remote site approaching the welding seam by means of a stereo view augmented by a graphical simulation system. The experimental results show that this approach also is successful. In more recent research, the application of VR equipment has been put under examination. In [13], a very progressive system is introduced. It uses VR goggles and a hand-held VR controller input device. A hidden Markov model is used to reconstruct the intention of the welder. A very good overview on different approaches is given in Section 3 of [14]. The review summarizes that the modeling and calibration techniques of virtual environment in the field of remote welding technology have been researched quite thoroughly. However, there are few research activities focusing on simpler approaches. A setup requiring less signal processing is introduced in [15,16]. It uses an infrared optical motion sensor to track the motion of a dummy welding torch as an input device. The motion is forwarded to the robot controller and the first successful results have been achieved. The system allows for perform welding in hazardous environments and the working place for the welder can be designed with substantially improved ergonomics. However, based on experiments performed in course of this project, the applied Leap Motion-Sensor has shown to be prone to disturbance by scattered infrared light, and it has exhibited insufficient accuracy and repeatability in many setups. In this approach, a robust dummy tool motion tracking system making use of marker-based visible light stereoscopy is introduced. Details are given on the overall setup, and particularly on the signal processing for a responsive hand motion tracking. Solutions for the referencing problems are presented, accompanied by an application report on the integration of a suitable real-time signal processing system. In the last part, promising experimental results are discussed.

2. Materials and Methods

2.1. Experimental Setup

The setup for movement tracking consists of a stereoscopic measuring system which is directed towards the working area of the operator, a visual feedback display, mounted below the input device’s working space, and a dummy torch for metal active gas (MAG) welding applications. It determines the pose of the welding-dummy with a repeating accuracy of better than 0.3 mm. The current experimental setup, which is shown in Figure 1, provides a usable working area of 400 × 600 × 400 mm.

The mount, which can be seen on the right side of the setup, is used for referencing the tool center point of the dummy torch with respect to the marker attached to it. Furthermore, it determines the base coordinate system of the working area of the operator. Figure 2 shows the experimental setup incorporating the KR6 Agilus R900. The robotic welding system is integrated in a welding cell with an integrated exhaust system and a height-adjustable protection glass. As a result, the developer and the robot can coexist without hazardous vapours or radiations in the same room. The workspace provides a 1500 × 570 mm plane with means to attach workpieces in the correct orientation towards each other. Furthermore, a pneumatic tool change system is mounted to the robot to allow for the use of different tools besides the welding gun. In order to provide visual feedback from the welding process, a camera is attached to the robot. The image is displayed on a screen below the operator’s work surface, as shown in Figure 1.

The position repeatability of the robot, which is controlled by the KRC4 industrial robot controller, lies within ±0.03 mm, which is below the tolerance band for welding applications. A software component, the Robot Sensor Interface (RSI) module [17], allows for direct control of the robot via ethernet. First, results are generated with a robot attachable welding gun (WF 60i ROBACTA DRIVE CMT) and a programmable power source from Fronius International GmbH [18]. The control signals of the power source are connected to the robot controller’s DI/DO interface by means of an add-on bus-converter. To measure the orientation and to reference the working plane of the robot to the working area of the operator, a 2D laser scanner scanCONTROL 2610-25 from Micro Epsilon [19] is used, which features a measurement accuracy of ±2 $\mathsf{\mu }$m. Figure 3 shows the signal process chain of the entire setup.

To use the KUKA RSI module, a deterministic response time to update requests is required. If the query time interval is not met, the robot goes into a defined error state. The image processing system cannot guarantee this hard real-time request due to the operating system and the variable image data processing time. To connect the asynchronous context with the synchronous context of the robot controller, a real-time computer based on the Preempt_RT extension to the Linux kernel [20] is used as a communication interface between the robot and the image processing system. Moreover, all further signal processing covered in the next sections as well as the safety features are executed on that computer.

2.2. Motion Signal Analysis

Because of the internal data processing within the KRC4 robot control, either a $C^2$-continous reference trajectory or an internal second order low pass filter is required to achieve good tracking performance. To provide a direct and responsive operator experience, the overall phase lag in the control loop must be kept as low as possible. In order to achieve this, the raw sensor signal is analyzed in the frequency domain to obtain a bandwidth of the useful signal contents. During this work, an iterative approach using data acquired during successful test welds has been chosen. The feasibility of the results has been checked by welding in a weaving motion as well as in a Christmas-tree-shaped motion. To present the motion signal processing in detail, the data from the Christmas-tree-shaped sample weld in the y–z-plane shown in Figure 4 are discussed. The welding motion has a large periodic component, which can be analyzed in the frequency domain.

Figure 5 shows the spectrum of the main components of the torch motion and the useful upper cutoff frequency for hand welding motion signals has been determined to be at least $f_c=1$ Hz in order to obtain a reference for the subsequent filter design.

In Figure 6, the raw sensor signal over time is depicted. Next, to the camera‘s image acquisition rate of $f_{fps}=50$ Hz, a jitter in the imaging processing time and the network transport delay as well as an additional noise component are visible. Moreover, the signal sporadically shows missing data, resulting from failed optical marker detections. Because of those missing points and because of the control loop cycle time of $T_{KRC4}=4$ ms, intermediate reference trajectory points have to be created by prediction.

2.3. Prediction Filter Design

A third order linear Kalman filter approach [21] with zero acceleration input and double integrator model according to Equation (1) is chosen for the prediction of the velocity and the extrapolation of the position signal, while the system waits for a new input pose from the stereoscopic measurement system. It is derived from a continuous triple integrator system that is discretized to obtain a zero-order-hold equivalent with the control loop cycle time of the robot controller $T_{KRC4}$ and the states representing the position $x_{k,1}$, velocity $x_{k,2}$ and the assumed constant acceleration $x_{k,3}$ of the respective degree of freedom:

$${\widehat{x}}_k=\left(\begin{array}{ccc}1& T_{KRC4}& \frac{1}{2}{T}_{KRC4}^2\\ 0& 1& T_{KRC4}\\ 0& 0& 1\end{array}\right){\widehat{x}}_{k-1}$$

(1)

The measurement noise covariance is directly obtained from a stationary torch position experiment and the process noise covariance is tuned to match the cutoff frequency obtained from the preceding motion signal analysis. Since the updates of the measurement vector of the Kalman filter occur less frequently then the updates of the model, the confidence in the model decreases over time. Thus, a time-variant gain filter results. The application of this Kalman filter to the signal is shown in Figure 7, and it can be seen that the extrapolation task is performed satisfactorily. However, there are occurrences, especially after there have been missing data from the sensor system, where the measurements do not meet the predictions and a step in the signal results, caused by the update of the Kalman filter.

To avoid second order low pass filtering in the KRC4 robot control in order to obtain a $C^2$-continuous signal, which would impose a strong phase lag to the overall control loop performance, a subsequent disturbance observer for the double integrator is applied. The disturbance model is regarded as an unknown acceleration acting on a zero-input double integrator plant. Thus, the error of the first integrator representing the velocity error and the error of the second integrator representing the position error are fed back to the input of the first integrator in order to estimate the acceleration. If rearranged as a filter, the scheme shown in Figure 8 results. Equation (2) shows the mapping of the predicted states to the filter’s inputs.

$$\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)=\left(\begin{array}{c}{\widehat{x}}_{k,1}\\ {\widehat{x}}_{k,2}\end{array}\right)$$

(2)

The implementation of this scheme into the real-time system has been done by discretization applying the second-order Tustin approximation. The coefficients are tuned by weighting position and velocity errors as well as adjusting the cut-off frequency to $f_c=4.0$ Hz to find a good trade-off between hand tremor suppression and low phase lag. To keep the computational effort for the calculation of the Kalman gain low, the pose components are calculated independently, decomposed in three translational and six rotatory (real and imaginary Euler angles) 3 × 3 matrix inversion problems. Otherwise, a substantially more complex 27 × 27 sparse matrix inversion problem would result. Figure 9 shows the phase advantage of the applied signal processing (Kalman filter and disturbance observer) in comparison to a simple second order low pass filter, which has been set to the same poles as those of the disturbance observer filter. Note that, in regions of low acceleration, the extrapolation of the sensor signal is nearly exact.

The deviation observable at t = 2.15 s results from missing input data from the stereoscopic camera system leading to a prediction error caused by a low velocity covariance estimation in the Kalman filter.

3. Results and Discussion

By application of the presented approach, it is possible to reproduce welding motions geometrically fitting into the box of the stereoscopic sensor system on the robot. Successful seams of appropriate quality are achieved by using the visual feedback from the display on the bottom of the input device’s working area and the audio feedback only. No augmented reality or simulation software is required because the system is nearly free of lag. Field reports from welders compliment the responsive behavior of the robot. This behavior results from the Kalman filter prediction algorithm and subsequent filtering, so no further signal filtering in the robot control has to be performed. Figure 10 shows a working piece with a complex shape, which is completely created by using the presented remote welding system. The operators have not been affected in a negative manner by the setup and have been able to fully focus on the welding task itself.

The geometric deviations between the input signal and the dimensions of the weld seam are below 1.0 mm. The main error influence is due to non-optimal referencing of the markers and could be improved by lowering the tolerances of the referencing holder which allows the marker on the dummy welding torch to be freely adjusted. Given the fact that the melting bath does not form evenly, the real deviations are considered to be even lower. The automatic referencing of the input devices base plane to the workpiece plane by means of the 2D laser scanner works well and needs no further improvements. The operator does not need any means of protection, since there is no hazard present on the input device and, moreover, the operator can be offset by the length of an Ethernet cable (i.e., approximately 100 m). Future developments on network technology, such as 5G wireless networks [22], will provide long-distance low-latency communications over the internet. The presented solution could be more willingly accepted by traditional welders than VR based approaches and has been developed together with professional welders and a welding equipment manufacturer (Dinse GmbH). However, future research has to be done on how to achieve continuous welding when the workpiece dimensions exceed those of the input device’s acquisition box. First, results are introduced by Eto and Harry [23] and an extrapolation of a welding motion for a certain period of time is conceivable, during which the welder can change his posture, setting parameters consciously without haste or implementing other changes. The hard real-time requirements of the KRC4 robot controller in connection with the RSI interface can be successfully met by application of the Preempt_RT extension to the Linux operating system kernel. No missed or unanswered telegrams occur. The lower sample rate and the jitter of the stereoscopic measurement system are overcome by the Kalman filter approach. Even sporadic failed acquisitions of the sensor system do not affect the performance. Here, the quality of the update of the filters’ output and covariance prediction could be further improved by incorporating the sensor system’s internal quality estimation instead of using a fixed value for the measurement covariance.

4. Conclusions

In this paper, it is shown that real-time remote robot welding is possible by the application of appropriate motion signal processing, a referencing procedure, and visual and audio feedback only. It allows a welder to work ergonomically without being exposed to immediate hazards and physical stress. It enables a location-independent input of data and a remote execution of a seam. The problem of continuous welding has to be addressed by future work.