Applications of Computer Vision in Automation and Robotics

Computer vision applications have become one of the most rapidly developing areas in automation and robotics, as well as in some other similar areas of science and technology, e.g., mechatronics, intelligent transport and logistics, biomedical engineering, and even in the food industry. Nevertheless, automation and robotics seems to be one of the leading areas of practical applications for recently developed artificial intelligence solutions, particularly computer and machine vision algorithms. One of the most relevant issues is the safety of the human–computer and human–machine interactions in robotics, which requires the “explainability” of algorithms, often excluding the potential application of some solutions based on deep learning, regardless of their performance in pattern recognition applications.

Considering the limited amount of training data, typical for robotics, important challenges are related to unsupervised learning, as well as no-reference image and video quality assessment methods, which may prevent the use of some distorted video frames for image analysis applied for further control of, e.g., robot motion. The use of image descriptors and features calculated for natural images captured by cameras in robotics, both in “out-hand” and “in-hand” solutions, may cause more problems in comparison to artificial images, typically used for the verification of general-purpose computer vision algorithms, leading to a so-called “reality gap”.

This Special Issue on “Applications of Computer Vision in Automation and Robotics” brings together the research communities interested in computer and machine vision from various departments and universities, focusing on both automation and robotics as well as computer science.

The paper [1] is related to the problem of image registration in printing defect inspection systems and the choice of appropriate feature regions. The proposed automatic feature region searching algorithm for printed image registration utilizes contour point distribution information and edge gradient direction and may also be applied for online printing defect detection.

The next contribution [2] presents a method of camera-based calibration for optical see-through headsets used in augmented reality applications, also for consumer level systems. The proposed fast automatic offline calibration method is based on standard camera calibration and computer vision methods to estimate the projection parameters of the display model for a generic position of the camera. They are then refined using planar homography, and the validation of the proposed method has been made using a developed MATLAB application.

The analysis of infrared images for pedestrian detection at night is considered in the paper [3], where a method based on an attention-guided encoder–decoder convolutional neural network is proposed to extract discriminative multi-scale features from low-resolution and noisy infrared images. The authors have validated their method using two pedestrian video datasets—Keimyung University (KMU) and Computer Vision Center (CVC)-09—leading to noticeable improvement of precision in comparison to some other popular methods. The presented approach may also be useful for collision avoidance in autonomous vehicles as well as some types of mobile robots.

Another application of neural networks has been investigated in the paper [4], where the problem of semantic segmentation of aerial imagery is analyzed. The proposed application of Generative Adversarial Networks (GAN) architecture is based on two networks with the use of intermediate semantic labels. The verification of the proposed method has been conducted using Vaihingen and Potsdam ISPRS datasets.

Since the semantic scene analysis is also useful in real-time robotics, an interesting fast method for semantic association of the object’s components has been proposed in the paper [5]. The Authors have proposed an approach based on the component association graph and a descriptor representing the geometrical arrangement of the components and have verified it using a ShapeNet 3D model database.

Another application of machine vision is considered in the paper [6], where the problem of volume estimation of irregular shape pellets is discussed. The use of granulometric analysis of 2D images proposed by the authors has been verified by measurements in a real production line. The obtained results make it possible to apply a continuous monitoring of production of pellets.

Merino et al. [7] have investigated the combination of histogram based descriptors for recognition of industrial parts. Since many industrial parts are texture-less, considering their different shapes, in view of lack of big datasets containing images of such elements, the application of handcrafted features with Support Vector Machine has been proposed, outperforming the results obtained using deep learning methods.

A prototype sorting machine for transparent plastic granulate based on machine vision and air separation technology has been presented in the penultimate paper [8]. The vision part of the system is built from an industrial camera and backlight illumination. Hence, k-Nearest Neighbors based classification has been used to determine defective transparent polycarbonate particles, making it possible to use only completely transparent material for further reuse.

Another contribution utilizing combination based approach [9] focuses on the quality assessment of 3D printed surfaces. In this paper, an effective combination of image quality metrics based on structural similarity has been proposed, significantly increasing the correlation with subjective aesthetic assessment made by human observers, in comparison to the use of elementary metrics.

As may be concluded from the above short description of each contribution, computer vision methods may be effectively applied in many tasks related to automation and robotics. Although a rapid development of deep learning methods makes it possible to increase the accuracy of many classification tasks, it requires the use of large image databases for training. Since in many automation and robotic issues, a development of such big datasets is troublesome, costly and time-consuming or even impossible in some cases, the use of handcrafted features is still justified, providing good results as shown in most of the published papers.

Some of the presented approaches, e.g., utilizing a combination of features or quality metrics, may also be adapted and applied to some alternative applications. Therefore, the Guest Editor hopes that the presented works may be inspiring for the readers, leading to further development of new methods and applications of machine vision and computer vision methods for industrial purposes.