Development of an Optimal Algorithm for Detecting Damaged and Diseased Potato Tubers Moving along a Conveyor Belt Using Computer Vision Systems

Round 1

Reviewer 1 Report

The paper is proposing an algorithm for detecting damaged potato tubers on a conveyor, which is distinguished by its speed of operation and recognition accuracy. Specially, distinctive features of the proposed algorithm are a combination of Viola-Jones methods with several high-performance classification methods. The proposed algorithm can be used in a picking robot in stalled on the conveyor belt of a vegetable store.

In this paper, it is also possible to determine, with up to 10% error, the mass of potatoes that would pass on a conveyor.

The paper is very organized with enhanced results. But the paper needs minor English grammar check.

Author Response

Response to Reviewer 1 Comments

Point 1:

The paper needs minor English grammar check.

Response 1:

Dear Reviewer,

Thank you very much for your very valuable comments. We checked the article using the Grammarly program. We hope that this version is written more competently.

 

Thanks again for your attention, consideration, and time.

Reviewer 2 Report

The authors present a paper on the development of algorithms to detect potato damage and diseased in grading lines by image analysis. This work is difficult to evaluate if it meets the minimum conditions for publication since in the bibliographic review (Line 62) it is said that ‘Only a few works are devoted to the identification of diseased tubers’. They make an introduction based on applications of image analysis in other sectors, such as vehicles. However, image analysis has been widely used to classify fruits and vegetables, and specifically potatoes. Hundreds of articles have been published on these topics. Thus, just as examples we can cite (this is not a complete list, just a sample):

Przybyl, K., Boniecki, P., Koszela, K., Gierz, L., Lukomski, M. 2019. Computer vision and artificial neural network techniques for classification of damage in potatoes during the storage process. Czech Journal of Food Sciences. 37(2), pp. 135-140

Wang, C., Li, X., Wu, Z., Zhou, Z., Feng, Y. 2014. Machine vision detecting potato mechanical damage based on manifold learning algorithm.      Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering. 30(1), pp. 245-252

Noordam, J.C., Otten, G.W., Timmermans, A.J.M., van Zwol, B.H. 2000. High speed potato grading and quality inspection based on a color vision system. Proceedings of SPIE - The International Society for Optical Engineering. 3966, pp. 206-217

2018. Lu, R. Lu. 2018. Detection of Surface And Subsurface Defects of Apples Using Tructured Illumination Reflectance Imaging with Machine Learning Algorithms. Transactions of the ASABE Vol. 61(6): 1831-1842

Payman Moallem, Navid Razmjooy, and Mohsen Ashourian. 2013. Computer Vision-Based Potato Defect Detection Using Neural Networks and Support Vector Machine. International Journal of Robotics and Automation 28(2):1-9

1. Blasco1; N. Aleixos2; E. Molt. Machine Vision System for Automatic Quality Grading of Fruit. Biosystems Engineering (2003) 85 (4), 415–423

In particular, the latter authors work at IVIA, where since the 1980s they have been working on the applications of image analysis in fruit sorting and have a wide repertoire of works in which they have been addressing the various problems posed: illumination, exposure of the entire fruit surface, instrumentation, algorithms, etc.

Moreover, this subject has gone beyond the research barrier years ago and image analysis grading lines such as https://www.tomra.com/en-gb/sorting/food/why and many others are commercially available.

In fact, the authors comment in the introduction that there is no published work on this subject, but they do cite some in the other sections of the paper.

The narrative in the Introduction does not clearly explain the advances made in this manuscript. Therefore, it is difficult for readers to understand what is new and important. In the current form, the originality of the study cannot be assessed. In order to evaluate the scientific novelty and originality of the work, the authors should make a new study of the state of the art focusing the topic on potato, or even on fruits and vegetables, and reformulate the objectives. This is the only way to really evaluate the scientific quality of this work. In any case, there are some observations that can already be corrected.

The authors should give the source or origin of the information presented in Table 1.

Line 173, 177, 362, 438. The meaning of many of the abbreviations used is already indicated in the abstract, so it is not necessary to repeat them throughout the article.

The paper lacks a discussion of the results.

The results obtained are not clear

Author Response

Response to Reviewer 2 Comments

Point 1:

The authors present a paper on the development of algorithms to detect potato damage and diseased in grading lines by image analysis. This work is difficult to evaluate if it meets the minimum conditions for publication since in the bibliographic review (Line 62) it is said that ‘Only a few works are devoted to the identification of diseased tubers’. They make an introduction based on applications of image analysis in other sectors, such as vehicles. However, image analysis has been widely used to classify fruits and vegetables, and specifically potatoes. Hundreds of articles have been published on these topics. Thus, just as examples we can cite (this is not a complete list, just a sample):

Przybyl, K., Boniecki, P., Koszela, K., Gierz, L., Lukomski, M. 2019. Computer vision and artificial neural network techniques for classification of damage in potatoes during the storage process. Czech Journal of Food Sciences. 37(2), pp. 135-140

Wang, C., Li, X., Wu, Z., Zhou, Z., Feng, Y. 2014. Machine vision detecting potato mechanical damage based on manifold learning algorithm.      Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering. 30(1), pp. 245-252

Noordam, J.C., Otten, G.W., Timmermans, A.J.M., van Zwol, B.H. 2000. High speed potato grading and quality inspection based on a color vision system. Proceedings of SPIE - The International Society for Optical Engineering. 3966, pp. 206-217

1. Lu, R. Lu. 2018. Detection of Surface And Subsurface Defects of Apples Using Tructured Illumination Reflectance Imaging with Machine Learning Algorithms. Transactions of the ASABE Vol. 61(6): 1831-1842

Payman Moallem, Navid Razmjooy, and Mohsen Ashourian. 2013. Computer Vision-Based Potato Defect Detection Using Neural Networks and Support Vector Machine. International Journal of Robotics and Automation 28(2):1-9

1. Blasco1; N. Aleixos2; E. Molt. Machine Vision System for Automatic Quality Grading of Fruit. Biosystems Engineering (2003) 85 (4), 415–423

In particular, the latter authors work at IVIA, where since the 1980s they have been working on the applications of image analysis in fruit sorting and have a wide repertoire of works in which they have been addressing the various problems posed: illumination, exposure of the entire fruit surface, instrumentation, algorithms, etc.

Moreover, this subject has gone beyond the research barrier years ago and image analysis grading lines such as https://www.tomra.com/en-gb/sorting/food/why and many others are commercially available.

In fact, the authors comment in the introduction that there is no published work on this subject, but they do cite some in the other sections of the paper.

The narrative in the Introduction does not clearly explain the advances made in this manuscript. Therefore, it is difficult for readers to understand what is new and important. In the current form, the originality of the study cannot be assessed. In order to evaluate the scientific novelty and originality of the work, the authors should make a new study of the state of the art focusing the topic on potato, or even on fruits and vegetables, and reformulate the objectives. This is the only way to really evaluate the scientific quality of this work. In any case, there are some observations that can already be corrected.

Response 1:

Dear Reviewer,

Thank you very much for your very valuable comments. The selection of articles you proposed was very timely. We have carefully researched combinations of the SVM method with other methods widely represented in your selection, and we have chosen the best-proven SIFT-SVM combination. The publication on the creation of an expert system seemed to us especially interesting. It correlates with the HOG-BOVW-BPN method, and we will try to analyze it in more detail in the near future.

Now, on the merits of the comments.

1. Line 62 has been fixed. We added six more sources on the classification of potatoes and other crops to the list of references. We made the appropriate changes in the introduction. Along with the already available sources, at least 85% of references to literature sources are devoted to studying crops.
2. Data on the speed of processing images of potatoes on a conveyor belt were presented. Our method allows us to classify up to 100 tubers per second with an accuracy of 97%, which is significantly above the analogs we found. Most of these systems are designed for the piece processing of images. The article provides links to similar studies conducted over the past four years. The speed is achieved due to the Viola-Jones method used in the first stage, which, unlike other methods, works in real-time and due to the developed pulsed light system, which makes pictures of moving objects clear even in poor ambient light conditions.

Point 2:

The authors should give the source or origin of the information presented in Table 1.

Response 2:

In Table 1, we used The Manual on the Design conveyor transport belt conveyors (to SNIP 2.05.07-85), All-Union Design and Research Institute of Industrial Transport ("PromtransNIIproekt") Gosstroya USSR, Moscow Stroyizdat 1988 Available at: https://xn--c1ahwb.xn--p1ai/uploadedFiles/files/Metodika_rascheta_lentochnykh_konveyerov_k_SNiP__2.05.07-85.pdf

The document regarding conveyors is still relevant.

Point 3:

Line 173, 177, 362, 438. The meaning of many of the abbreviations used is already indicated in the abstract, so it is not necessary to repeat them throughout the article.

 

Response 3:

We have made the appropriate changes in lines 173, 177, 362, 438.

Point 4:

The paper lacks a discussion of the results.

The results obtained are not clear

Response 4:

We have added a "Discussion of results" chapter in which we tried to clarify the results of the research.

Discussion

At the modern stage of the development of computer vision technologies, the question of object classification makes no longer a problem. Convolutional neural networks, decision trees, etc., have been performing it much more accurately than a human would have done. However, we drew attention to several limitations of these methods associated with their application in real conditions of vegetable storage. So, for example, based on the conclusions of modern researchers, convolutional networks that are most promising for solving computer vision problems are not able to process the stream of images from a video camera of rapidly moving objects of small size and a significant number [32]. The algorithm, launched on inexpensive computer hardware, simply cannot keep up with the conveyor belt. And obsolete, even if properly processed, data will be of little use if damaged objects have already left the conveyor belt.

We proposed using the Viola-Jones algorithm at the first stage of processing the image from a video camera, which, unlike convolutional neural networks, works in a real-time mode [58-63]. This method was created for recognizing human faces and did not give good results when used to detect potato tubers; however, by selecting preprocessing filters, we achieved a probability of 97%, which corresponds to the results of a convolutional neural network (from 91 to 95% in works on convolutional networks for the last three years) [25-31].

At the second stage, we work with an image in which the sizes and coordinates of the tubers have already been determined. The classification task is reduced to determining whether the tuber is suitable for further use or if it should be recognized as damaged or diseased. Dividing the image into small fragments simplifies the task greatly. When processing small images, a convolutional neural network installed on a personal computer is capable of processing up to a hundred images per second. This corresponds to a complete classification of tubers that slightly fill the conveyor and move at low speed. The result of correct image classification by the convolutional network was up to 97%. On average, 50% more tubers are processed by the combination of HOG - BOVW - BPNN methods, but its classification result is about 2% less. These results significantly exceed the processing speed of computer vision systems installed on conveyors [9-17]. We plan to continue improving the second stage of the image processing in the direction of increasing the number of classified tubers per second and bringing it to several hundred, which will allow installing the computer vision system on conveyors with almost any load and moving at the maximum permissible speed. For this, we propose to create systems of parallel computation and split the objects selected at the first stage into several parallel processed threads.

The used methods allow us to find the percentage of damaged tubers and the number of tubers. Taking into account the area of each tuber in the image, we can determine the mass of potatoes passed along the conveyor with an error of up to 10%. This is consistent with the materials of the article by A. Kalantar et al., who determined the weight of agricultural products from its image [64].

           We believe that the proposed algorithm can be used in a picking robot installed on the conveyor belt of a vegetable store [65, 66].

 

Thanks again for your attention, consideration, and time.

Round 2

Reviewer 2 Report

I have carefully read the authors' replies and have reviewed the document again with the proposed modifications. Under these conditions, I believe that the manuscript can be published.