Visual Servoing in Robotics

1. Introduction

Visual servoing is a well-known approach to guide robots using visual information. Image processing, robotics and control theory are combined in order to control the motion of a robot depending on the visual information extracted from the images captures by one or several cameras. With respect to vision issues, different problems are currently under research such as the use of different kinds of image features (or different kinds of cameras), image processing at high velocity, convergence properties, etc. Furthermore, the use of new control schemes allows the system to behave more robustly, efficiently, or compliantly with less delays. Related issues such as optimal and robust approaches, direct control, path tracking or sensor fusion allows the application of the visual servoing systems in different domains.

In so-called image-based visual servoing systems, the control law is calculated using directly visual information. These last systems do not need a complete 3D reconstruction of the environment. For tasks that require high precision, speed or response, several works suggest that it may be beneficial to take into account the dynamics of the robot when designing visual servoing control laws. The type of visual control systems that consider the dynamics of the robot in the control law are often referred to as direct or dynamic visual control systems. However, for simplicity, indirect visual servoing schemes are mostly used in the literature.

Nowadays, the application fields of the visual servoing systems are very wide, and include research and application fields such as navigation and localization of mobile robots, guidance of humanoid robots, robust and optimal control of robots, manipulation, intelligent transportation, deep learning and machine learning in visual servoing and visual guidance of field robotics (aerial robots, assistive Robots, medical robots, etc.).

2. The Present Issue

This special issue consists of eight papers covering important topics in the field of visual servoing. In [1], an enhanced switch image-based visual servoing system for a 6 degrees of freedom industrial robot is proposed. An image feature reconstruction algorithm based on the Kalman filter is presented to handle feature loss during the tracking. Visual servoing approaches can be applied for the guidance of different kinds of robots such as mobile robots, aerial robots or parallel robots. The latter is the case described in [2], where an optical coordinate measuring machine is employed to stabilize parallel robots. In this case, the dynamic model parameters are identified by using a non-linear optimisation technique. In [3], a direct image-based visual servoing system is used for the guidance of a mobile manipulator. This approach considers not only kinematic properties of the robot, but also dynamic ones for guiding both the robot base and the manipulator arm. An optimal control approach is used in this paper.

This special issue also includes papers that describe new and interesting applications of the visual servoing systems such as the ones described in [4] or [5]. In [4], a visual servoing system is applied to an apple-picking robot. The image-based visual servo control method is adopted to control the manipulator in order to improve the grasping accuracy in the picking process. The joint control performance of the control system has been improved by the proposed adaptive fuzzy neural network sliding-mode control algorithm. Additionally, in [5], a spatial trajectory optimization method of a spray-painting robot is proposed.

Additionally, visual servoing approaches are very related with the necessity of estimating parameters or variables used in the control process. For example, in [6], a visual-based method to estimate robot orientation with RGB-D cameras is proposed. In [7], the reaction force of the end effector and second link of a three-degree of freedom hydraulic servo system with master–slave manipulators sliding mode control is determined with a sliding perturbation observer. Also, bilateral control is used to estimate the reaction force of the master device which is provided to the operator to handle the master device. Finally, in [8] a novel measurement system to visualize the motion-to-photon latency with time-series data in real time is proposed.

3. Future

While visual servoing systems have been an important field of research in the last years, several major challenges still remain. Tasks such as tracking, positioning, detection, segmentation, and localization play a critical role in visual servoing and different research is currently ongoing to increase the robustness of visual controllers. Additionally, new control approaches such as optimal control, robust control, dynamic control or predictive control will provide this kind of system with new dynamic properties. Furthermore, new computer vision systems, electronics, computers, etc., will offer new and interesting capabilities for the application of visual servoing in new kinds of robotics systems such as autonomous driving cars, humanoid robots, aerial robots, service robotics, UAV, parallel robots, space robotics, etc.