Real-Time Detection of Strawberry Powdery Mildew Disease Using a Mobile Machine Vision System

Abstract

Strawberry cropping system relies heavily on proper disease management to maintain high crop yield. Powdery mildew, caused by Sphaerotheca macularis (Wall. Ex Fries) is one of the major leaf diseases in strawberry which can cause significant yield losses up to 70%. Field scouts manually walk beside strawberry fields and visually observe the plants to monitor for powdery mildew disease infection each week during summer months which is a laborious and time-consuming endeavor. The objective of this research was to increase the efficiency of field scouting by automatically detecting powdery mildew disease in strawberry fields by using a real-time machine vision system. A global positioning system, two cameras, a custom image processing program, and a ruggedized laptop computer were utilized for development of the disease detection system. The custom image processing program was developed using color co-occurrence matrix-based texture analysis along with artificial neural network technique to process and classify continuously acquired image data simultaneously. Three commercial strawberry field sites in central Nova Scotia were used to evaluate the performance of the developed system. A total of 36 strawberry rows (~1.06 ha) were tested within three fields and powdery mildew detected points were measured manually followed by automatic detection system. The manually detected points were compared with automatically detected points to ensure the accuracy of the developed system. Results of regression and scatter plots revealed that the system was able to detect disease having mean absolute error values of 4.00, 3.42, and 2.83 per row and root mean square error values of 4.12, 3.71, and 3.00 per row in field site-I, field site-II, and field site-III, respectively. The slight deviation in performance was likely caused by high wind speeds (>8 km h⁻¹), leaf overlapping, leaf angle, and presence of spider mite disease during field testing.

1. Introduction

Powdery mildew (Sphaerotheca macularis) is a serious disease affecting strawberry production in both warm and dry climates [1] and reduces crop yields by causing decreased fruit set, inadequate ripening, poor flavor, fruit cracking and deformation, and reducing postharvest storage time [2]. Powdery mildew (PM) can be remarkably problematic when strawberry plants are grown in greenhouses or polythene tunnels, both of which are conducive to severe outbreaks of PM [3]. Generally, strawberries are commercially grown on raised beds covered with impermeable polythene mulch/tunnel in outdoor fields. The white patches of mycelia are the initial symptoms of PM disease, which appear on upper and/or lower strawberry leaf surfaces during disease progression [4]. The PM disease symptoms primarily appear on young leaves [5], scatter through petioles, runners, flowers, and fruits except roots [6]. The conidial germination of PM disease is optimally grown when the temperature ranges between 15 and 25 °C and relative humidity ranges from 75% to 98% [5]. When the PM disease becomes severe, the fruit may crack, causing exposure to secondary infections [7] and causing yield losses of up to 70% [8]. To date, visual observation is still the only method to measure the presence of PM disease in strawberry cultivation systems. According to Zhang et al. [9], the traditional visual inspection of diseases is a time-consuming and labor-intensive approach and is also quite impractical on large farms. Thus, an automated detection system is vital for strawberry growers to inspect PM disease occurrence in field.

Over the last decade, image processing-based machine vision technologies have been effectively used for fast and accurate plant disease monitoring and detection within fields [10,11]. Imaging in the open field is a challenge compared to laboratory conditions due to uncontrollable environmental factors such as illumination variation, leaf density and overlap, leaf angle, and wind speed [12]. Open fields can result in adverse operating environments, offering random variability due to terrain and weather, as well as imperfections of input images that can easily influence the accuracy and precision of machine vision systems [13]. Pajares et al. [14] provided guidelines for selection of appropriate agricultural machine vision systems based on lighting conditions in outdoor environments, irregular terrain, or different plant growth stages for optimum performance. Lee et al. [15] developed a real-time machine vision system, which was only capable of correctly detecting 47.6% of weeds with 24.2% oversprayed tomato plants in outdoor field conditions. Sabzi et al. [16] proposed a machine vision system based on hybrid artificial neural network-harmony search classifiers for automatic segmentation of plants under different illumination conditions and claimed it could be applied under all field applications with higher accuracies. However, reliable studies have not been conducted using mixed canopy conditions that are common in agricultural fields. Therefore, a real-time machine vision system that is able to detect plant disease under mixed canopy field conditions needs to be developed.

Several attempts applying a color co-occurrence matrix (CCM)-based image processing technique for crop disease detection have been reported by various researchers, with promising results under different cropping systems [10,17,18,19,20,21,22].

Table 1. Summary of disease detection accuracies using color co-occurrence matrix (CCM)-based textural analysis in different cropping systems.

Diseases Detection/Classification	Image Processing	Accuracy	References
Normal and greasy spot, melanose, and scab citrus leaf diseases	CCM and a back-propagation neural network	Over 90%	[17]
Normal and greasy spot, melanose, and scab citrus leaf diseases	CCM features extract from converted hue, saturation and value (HSV) color space	Over 95%	[18]
Early scorch, cottony mold, ashen mold, late scorch, and tiny whiteness diseases of tomato and other crops	CCM along with neural network classifier	93%	[19]
Early scorch, cottony mold, ashen mold, late scorch, and tiny whiteness diseases of tomato and other crops	CCM and neural network model	83–94%	[10]
Nursery stock	CCM and discriminant analysis	90.9%	[23]
Powdery mildew of strawberry	CCM along with support vector machine algorithm (SVM)	91.86%	[21]
Yellow spots, brown spots, scorch and late scorch diseases of banana, lemon, and bean	CCM texture analysis with SVM	94%	[20]
Scorch and spot classification of plant leaves	CCM and artificial neural network (ANN)	100%	[24]

summarizes the accuracies obtained from using a CCM-based textural analysis method for disease detection in crops.

Pydipati et al. [17] utilized a CCM-based textural analysis method to extract important features aimed at identifying diseased and normal citrus leaves. According to the authors, the CCM, along with a back-propagation neural network algorithm, achieved over 90% accurate detections in all cases used for analysis. In follow-up work, Pydipati et al. [18] used a CCM-based classification algorithm to detect healthy and greasy spot, melanose, and scab disease-affected citrus leaves. Results of their laboratory trials showed that the algorithm was able to achieve classification accuracies of over 95% for all classes. Other studies have successfully achieved 93% detection for tomato leaf disease [19], using a CCM-based texture analysis algorithm along with a neural network classifier, and 91% accuracy in classifying different types of nursery stock [23], using a CCM-based algorithm. CCM-based custom image processing is not a novel approach but to date limited research has been conducted on detection of strawberry powdery mildew, which causes severe yield crop losses around the world. Current detection of PM, specifically in strawberries, using machine vision-based automated techniques have largely been conducted under laboratory or controlled conditions [21,25]. Yang et al. [25] demonstrated the capacity to recognize strawberry PM disease from single harvested strawberry leaf using a convolutional neural network. Mahmud et al. [21] analyzed single leaf images to detect PM disease in strawberry under controlled environmental conditions during image acquisition. Real-time detection of weeds and disease poses significant challenges and only a few have been successfully evaluated in agricultural fields.

Based on the success of PM detection under controlled conditions in previous work, the goal of this study was to evaluate how a mobile machine vision system would perform in real-time detection of PM in field-based strawberry production systems.

2. Materials and Methods

2.1. Machine Vision System Development

A mobile machine vision system was developed by fabricating a custom platform (width × length × height: 2.184 m × 1.219 m × 0.737 m) using locally sourced bicycle components to minimize the cost (Figure 1).

The custom-based machine vision system was fabricated by modifying image acquisition hardware, as defined in Mahmud et al. [22], in order to effect real-time detection in the field. The system was manually propelled by pushing an integrated handle bar. An artificial cloud lighting condition (ACC) system was mounted on the mobile platform to minimize the variation of illumination during image acquisition in the field [22]. The ACC was made with a black cloth cover to avoid direct sunlight during image acquisition (Figure 1). The light illumination readings inside of the ACC chamber ranged from 800 to 900 lx during field trials. The system was designed to operate within single strawberry rows (1.219 m width) and slim bicycle wheels were used to minimize the damage on strawberry runners during the study. The system consisted of two μEye 1240 LE/C color cameras (IDS Imaging Development System Inc., Woburn, MA, USA), a HiPer^® Lite + RTK-GPS (Topcon Positioning Systems Inc., Livermore, CA, USA) for georeferencing and a ruggedized laptop computer (Toshiba Corporation, Minato, Tokyo, Japan). Each of the cameras acquired 24-bits blue–green–red (BGR) 640 × 256 pixels images covering a 0.6096 m × 0.2438 m (length × width) area of interest (AOI) of each section (half of row) in strawberry row (Figure 2).

The AOI was used to minimize the barrel effect caused by the wide-angle lens by extracting from the center of the full frame images of 1280 × 1024 pixels. The wide-angle field of view lenses (LM4NCL, Kowa Optimed Inc., Torrance, CA, USA) were set up to a fixed aperture (f/4.0) and infinity focus with a 3.5 mm focal length. The cameras were set up with a 30° inclination from the vertical Z-axis (downward), as shown in the Figure 3, for acquiring suitable images for PM detection in the field. The inclination of 30° was chosen based on testing different angles prior to the start of the study. Images acquired with a 30° inclination facing the driving direction made the PM symptoms more visible since the disease is located under the leaves initially and infected leaf edges may also roll upward. Examples of images captured using a two-camera set up (one is 90° vertically downward and another is 30° inclination with the vertically downward-axis) are shown in Figure 4.

Cameras were connected directly to the laptop computer using universal serial bus (USB) cables. Image acquisition by the cameras were conducted at a travelling speed rate of 25 frames per second (FPS) during the experiments. An image was extracted from every 15 frames to avoid overlapped areas between images. The cameras had a 0.3 m working depth from the camera sensor to leaf canopy. The system speed was manually controlled at 1.50 ± 0.40 km h⁻¹ and was maintained constant during the study by continuous monitoring of the graphical user interface (GUI) display. The system speed (in knots) was parsed from RTK-GPS string and converted into metric units (speed (ms⁻¹) = 0.51444 × speed (knots)). The 0.30 m working depth and image acquisition speed of 1.50 km h⁻¹ was previously determined to be optimum for strawberry powdery mildew detection [22]. Digital gain and exposure time were automatically controlled by autogain control and autoexposure shutter. The RTK-GPS antenna was mounted above the cameras to simultaneously record the coordinates. The RTK-GPS position of the images was continuously stored in laptop computer by using National Marine Electronics Association (NMEA-0183) standard code sentences. The program was designed to only store the powdery mildew image location, when it was detected, into a comma-separated values (CSV) file.

A real-time strawberry powdery mildew disease detection algorithm was developed in C# (sharp) using Visual Studio 2017 (Microsoft, Redmond, WA, USA). The algorithm was programmed to process images and differentiate powdery mildew affected leaves from healthy or other diseased leaves (that did not retain similar features to PM). The image processing steps started with a conversion of the acquired blue, green and red (BGR) image into a green ratio, and hue, saturation and intensity (HSI) image. According to Meng et al. [26], the HSI color space works better compared to RGB (red, green, and blue) and YUV (luminance, chrominance, and chroma) for image processing under field conditions. The hue channel from HSI color space represents the purity of color, such as pure blue (b), green (g), and red (r) in terms of degree, whereas saturation represents the measure, from 1 to 0, to which pure color is diluted by a neutral color [27]. The b, g, and r intensity levels of individual pixel of an image were utilized to calculate the hue (H), saturation (S), and intensity (I) components of that pixel by using the geometrical transformation relationships. These relationships were defined by the International Commission on Illumination (CIE) chromaticity diagram [27]. A color conversion was performed for this study from BGR to HSI color plane on input images. Each input image was used to create three two-dimensional arrays in the process of CCM, color co-occurrence matrix. The pixel intensity of each array was applied for CCM construction from each color plane. The H, S, and I images were converted using Equations (1)–(4) suggested by [27].

$${\theta }_h=co{s}^{-1}\{\frac{\frac{1}{2}\left[\left(r-g\right)+\left(r-b\right)\right]}{{[{\left(r-g\right)}^2+\left(r-g\right)\left(g-b\right)]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}}\}$$

(1)

where, ${\theta }_h$ is the angle, and hue (H) color plane was calculated based on angle (0–360°) of circle which was normalized in the range between 0 and 1. This normalized angle was then linearly transformed to 256 different intensity levels for calculating the H of particular pixel depending upon its r, g, and b components (Equation (2)).

$$H=\left\{\begin{array}{c}\frac{{\theta }_h}{360}\times 255 if b\le g\\ \frac{360-{\theta }_h}{360}\times 255 if b>g\end{array}\right\}$$

(2)

$$S=255\times \{1-\frac{3}{\left(r+g+b\right)}(Min\left(r, Min\left(g,b)\right)\right)\}$$

(3)

where, S is the saturation color plane.

$$I=\frac{r+g+b}{3}$$

(4)

The ratio used was (G × 255) (B + G + R)⁻¹ and a manually obtained threshold (>86) for conversion to g-ratio images. The g-ratio images were masked with the original images to remove the background. Another thresholding was conducted with the masked image along with the manually obtained threshold (red > 190 and red < 220, green > 220 and green < 245, and blue > 180 and blue < 200) to obtain the final image for extracting valuable features. Color co-occurrence matrices (CCMs) [23] were constructed from converted images, followed by textural features extraction. The CCMs were constructed with four converted images from one image followed by extracting a set of 10 features from the individual CCM. Since frequency in the CCMs is a function of the angular relationship and distance between neighboring pixels, in this research, an angular relationship ($o_a$) of 0° and a displacement vector ($d_v$) of 1 pixel were selected for CCM construction. The displacement vector of 1 pixel was selected, as it provided exceptional results when varied between 1 and 5 [28]. The features data were normalized and resulting normalization of each CCM value varied from 0 to 1. The CCMs were normalized by dividing the individual entity in the CCM matrix by the total number of pairs in each matrix using the relationship presented in Equation (5) [29].

$$N\left(a,f\right)=\frac{n\left(a,f, 1, 0\right)}{{{\displaystyle \sum }}_{a=0}^{F-1}{{\displaystyle \sum }}_{f=0}^{F-1}n\left(a, f, 1, 0\right)}$$

(5)

where, N (a, f) is a normalized CCM, n (a, f) is a marginal probability function, a is the intensity level at a certain pixel, f is another matching intensity level with displacement vector, $d_v$ = 1 and an orientation angle $o_a=0˚$, and the denominator of the equation, sum of n (a, f), is the total number of pairs in the matrix with specific orientation and displacement vector.

A total of 40 textural features were extracted from an individual image after constructing the CCM but 23 features were specifically chosen to detect PM disease. The textural features were extracted by equations used from Shearer and Holmes [23] and presented in

Table 2. Textural features equations suggested by [23].

Features Name	Equation [v]
Contrast	${\displaystyle {\displaystyle \sum }_{\left|a-f\right|=0}^{F-1}}{\left(a-f\right)}^2{\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right)$
Homogeneity	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right)\frac{1}{1+\left|a-f\right|}$
Entropy	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right) In N\left(a, f\right)$
Dissimilarity	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right)\left|a-f\right|$
Angular second moment	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N{\left(a,f\right)}^2$
Inverse difference moment	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}\frac{N\left(a, f\right)}{1+{\left(a-f\right)}^2}$
Average	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}f N_x\left(a\right)$
Sum of squares	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\left(a-\mu \right)}^2{N}_x\left(a\right)$
Product moment	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right)\left(a-\mu \right)\left(f-\mu \right)$
Correlation	${\displaystyle {\displaystyle \sum }_{a=0}^{F-1}}{\displaystyle {\displaystyle \sum }_{f=0}^{F-1}}N\left(a, f\right)\frac{\left(a-{\mu }_a\right)\left(f-{\mu }_f\right)}{{\mathsf{\sigma }}_a{\mathsf{\sigma }}_f}$

[v] F is the total number of intensity levels, N (a, f) is the (a, f)th entry in a normalized CCM, µ is the mean, ${\mu }_a$ is the mean of row, ${\mu }_f$ is the mean of column, ${\mathsf{\sigma }}_a$ and ${\mathsf{\sigma }}_a$ are the standard deviation along the ath row and fth column of N (a, f), respectively, and N_x (a) was obtained by summation of CCM values in ath row.

.

The 23 textural features were chosen due to their optimal performance compared to extracted 50 features for powdery mildew disease detection reported in laboratory experiments by Mahmud et al. [22] using stepwise discriminant analysis. The selected features are presented in

Table 3. Selected textural features.

Converted Images	Extracted Textural Features [S]
	Con	Hom	Ent	Dis	A2M	IDM	Avg	SoS	PM	Cor
G-ratio	✔	✔		✔					✔	✔
Hue	✔	✔	✔	✔			✔			✔
Saturation	✔	✔		✔			✔		✔	✔
Intensity	✔	✔		✔			✔		✔	✔

[S] Con: contrast, Hom: homogeneity, Ent: entropy, Dis: dissimilarity, A2M: angular second moment, IDM: inverse difference moment, Avg: average, SoS: sum of square, PM: product moment, and Cor: correlation.

.

Dandawate and Kokare [30] reported that a combination of feature extraction using CCM and artificial neural networks (ANN) machine learning was suitable for detection of different plant diseases. Among the many supervised machine learning technologies available, using co-occurrence-based feature extraction with ANN generated better results when dealing with different leaf angles or positions, which is likely to occur under field conditions [31]. Therefore, the CCM extracted features were analyzed for PM by using the ANN-based machine learning classifier. ANN-based machine learning was also chosen due to its superior performance relative to support vector machines and k-nearest neighbor for powdery mildew detection reported in controlled experiments [32]. Peltarion Synapse (Peltarion Corp., Stockholm, Sweden) software was used to select the optimal ANN model architecture using our extracted features data. A total of 20 combinations of model architectures were tested and 4 optimal architectures were selected (

Table 4. Tested mathematical functions at an epoch size of 15,000 with normalized data.

Model Structures	Tanh Sigmoid		Logistic Sigmoid		Linear		Exponential
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
1W (23/23) and 1F (23/1) 23 inputs 1 output	0.040	0.058	0.073	0.082	0.265	0.343	1.639	1.973
1W (23/23)1W (23/23)1F (23/1) 23 inputs 1 outputs	0.008	0.010	0.054	0.068	0.146	0.211	0.683	0.769
1W (23/46) and 1F (46/1) 23 inputs 1 output	0.004	0.005	0.017	0.021	0.119	0.193	0.614	0.687
1W (23/46)1W (46/46)1F (46/1) 23 inputs 1 output	0.003	0.004	0.010	0.023	0.102	0.179	0.528	0.692

W = weight layer; F = function layer.

).

A back-propagation artificial neural network (BP-ANN) algorithm was applied to train the proposed network architectures. Four different transfer functions, including the tanh sigmoid, exponential, logistic sigmoid, and linear functions were used to translate the input signals into output signals ranging from 0 to 2 (i.e., 0, 1, and 2). The extracted textural features were selected as inputs for the input layer and corresponding healthy or disease labels (powdery mildew and other diseases) were established as an output in the output layer. All the settings of developed models were kept constant, the mathematical functions were changed, and finally mean absolute error (MAE) and root mean square error (RMSE) were recorded to find an optimal mathematical function for this study. A 1W (23/46) 1W (46/46) 1F (46/1) ANN model architecture with a tanh sigmoid transfer function was chosen having the epoch size of 15,000. Farooque et al. [33] also developed an optimal ANN model with an epoch size of 15,000 and reported that the model was more suitable in capturing nonlinearity of relationships between variables. A 6000 image dataset was analyzed (60% for training and 40% for validation) to select an optimal model architecture. Three categories of images, e.g., healthy, powdery mildew affected, and other disease affected, were collected from 10:00 a.m. to 4:00 p.m. Upon selection of an optimal ANN model architecture, the model was deployed from Peltarion Synapse software as a dynamic link library file (dll) file. The deployed dll file used by the real-time powdery mildew disease detection algorithm ensured real-time selection of one of the three categories, i.e., powdery mildew affected, healthy, or other disease affected leaves. The algorithm had a statement (is the image powdery mildew affected?) to save the georeferenced coordinates in a CSV file of leaves identified as PM affected. The detection results of georeferenced coordinates location were imported into ArcGIS 10.5 computer software (ESRI, Redlands, CA, USA) for prescription mapping. Overview of the real-time detection algorithm is presented in Figure 5.

2.2. Field Evaluation of Machine Vision System for Strawberry Powdery Mildew Detection

Testing of the real-time machine vision system on powdery mildew disease detection in commercial strawberry fields was carried out on the 6, 7, and 13 August 2018. The commercial strawberry operations located in Debert, Nova Scotia, provided fields under production for the evaluation of the real-time system. The tests were conducted on sunny days, with temperatures ranging from 20 to 32 °C, relative humidity (R_H) from 50% to 82%, and wind speeds from 3 to 14 km h⁻¹ [34].

Three strawberry field sites were selected in Debert, Nova Scotia, to evaluate the performance of machine vision-based powdery mildew disease detection system. A commercial 12-hectare strawberry farm was used to conduct this study. The field sizes were 1.3 ha each (area) located in Debert site I (field 1; 45.429318° N, 63.483843° W), Debert site II (field 2; 45.429611° N, 63.48114° W), and Debert site III (field 3; 45.429098° N, 63.480276° W). All fields were cultivated with an Albion strawberry variety. Albion is a day-neutral or everbearing strawberry variety that grows quickly to about 12 inches (0.30 m) in height, with a spread of 12–24 in (0.30–0.60 m). They are high yielding and everbearing, which means they usually provide flower and fruit continuously from late spring into the fall. The fields have been cultivated in strawberries over the past few years and each maintained commercial management practices including mowing, pruning, fertilizing, and application of herbicides, pesticides, fungicide, etc. A total of 12 randomly selected strawberry rows were tested in each field (Figure 6). The strawberry rows evaluated had dimensions of 1.22 m × 220 m (wide × length) for field site-I and 1.22 m × 180 m for field site-II and field site-III with a 0.31 m buffer between rows (Figure 6).

Manual detection of PM disease was conducted for each experimental field based on recommendations from two experienced field scouts who checked for symptoms including white patches of mycelium on the upper leaf surface, roll upward leaf edges, and reddish irregular spots on leaf. Manual detection was conducted by carefully confirming all the symptoms of strawberry PM, especially checking white patches with roll edges leaves throughout the experiments. The detected points were marked by inserting red flags. The number of detected points of powdery mildew affected plants in a single row was calculated manually recorded in a paper notebook (Hilroy, Mississauga, ON, Canada) after counting. The locations of the detected points were also recorded using a ProMark3 mobile mapper (Thales Navigation, Santa Clara, CA, USA).

The machine vision system was deployed over individual rows of strawberry plants for continuous image acquisition by the two cameras. A total of 36 rows were covered over 3 strawberry fields (12 rows each) in this study. The images were processed through texture analysis by CCM and followed by detection using the ANN classifier. A step-by-step real-time strawberry PM disease detection process of an image is presented in Figure 7. The average processing time for an image was 7648 μs (~0.0076 s). The powdery mildew detected leaf image locations (latitude and longitude) were saved automatically to a CSV file using a function in the custom software. Manually and automatically detected points were compared by manually counting points to evaluate the outcomes of the system. The detected points were mapped to create prescription map using the co-ordinates (longitude and latitude) collected from RTK-GPS and mobile mapper.

The software interface of the machine vision system was developed using Microsoft Visual Studio 2017 (Microsoft Corp, Redmond, WA, USA) (Figure 8).

Communication between the RTK-GPS and laptop computer was established using a serial link setting (COM Port 1, baud rate 9600 bps, stop bit none, and parity bit 1). The speed of the system was calculated from NMEA-0183 standard code system directly from the RTK-GPS data displayed in GUI (Figure 8). The latitude and longitude were also recorded from standard code from the RTK-GPS. The checkbox of camera selection was added to control the two cameras used for real-time image acquisition. Although the experiments used four processed images from one camera, two picture boxes (for one camera) were added in the real-time software due to a lack of space in the GUI display. The processed g-ratio and hue images were displayed from both cameras. The program was able to process the images to differentiate powdery mildew leaf images in the strawberry fields in real-time from both images taken by the two cameras. The system performance was evaluated by correlating the manual detection results with the automatic system detection results from both cameras in all the fields. The performance of the developed system is also evaluated by calculating recall, precision, and F-measure using relationship presented in Equations (6)–(8), respectively. Prescription maps were developed from the experimental data using ArcGIS 10.5 software.

$$Recall=\frac{T_p}{T_p+F_n}\times 100 (\%)$$

(6)

$$Precision=\frac{T_p}{T_p+F_p}\times 100 (\%)$$

(7)

$$F_{\mathsf{\epsilon }}=\frac{\left(1+\mathsf{\epsilon }\right)\times Recall\times Precision}{\mathsf{\epsilon }\times Precision+Recall}$$

(8)

where, $T_p$ is the correctly detected points of PM disease, $F_p$ is the number of healthy and other diseases points that are falsely classified as PM disease, and $F_n$ is the number of points of PM disease that are falsely detected as healthy or other disease. $F_{\mathsf{\sigma }}$ is the F-measure value representing accuracy using recall and precision relationship, and $\mathsf{\epsilon }$ is a non-negative real value, we set $\mathsf{\epsilon }$ = 0.80 in this study to weigh recall more than precision.

2.3. Statistical Analysis

Linear regression analysis was used to compare the results of manual measurements (PM detected points) and real-time detection using the mobile machine vision system separately in each field by using Minitab^® 18 statistical software (Minitab^® Inc., State College, PA, USA). The coefficient of determination (R²), mean absolute error (MAE), and root mean square error (RMSE) were calculated. A paired t-test was also used to compare the mean of two measurements in Minitab version 18.

3. Results and Discussion

3.1. Laboratory Evaluation Results for ANN-Based PM Detection

Table 5. Laboratory-based evaluation of artificial neural network (ANN) classifier for strawberry powdery mildew (PM) detection.

		Predicted				Performance (PM Detection)
Validations	Response	Healthy	PM	Other Diseases	Total	Recall (%)	Precision (%)	F-Measure (%)
Internal	Healthy	790	3	7	800	98.00	98.49	98.27
	PM	6	784	10	800
	Other diseases	4	9	787	800
Fivefold Cross	Healthy	394	2	4	400	97.00	97.98	97.54
	PM	4	388	8	400
	Other diseases	3	6	391	400
External I a	Healthy	578	23	65	666	87.44	85.27	86.22
	PM	26	569	71	666
	Other diseases	33	60	573	666
External II b	Healthy	587	24	55	666	86.94	86.81	86.87
	PM	16	579	71	666
	Other diseases	19	64	583	666
External III c	Healthy	586	29	51	666	86.49	86.75	86.63
	PM	32	576	58	666
	Other diseases	27	59	580	666

External-I ^a: training with Field I + Field II and validated with Field III; External-II ^b: training with Field I + Field III and validated with Field II; External-III ^c: training with Field II + Field III and validated with Field I.

shows ANN-based machine learning classifier performances based on internal, external, and cross validations with different image datasets. The model detected healthy, powdery mildew, and other diseases leaf images with higher accuracy in all cases of internal validations. The recall was calculated by the ratio of total correctly detected PM and total numbers of PM being detected (e.g., in case of internal validation: recall for PM detection = 784/(6 + 784 + 10) and for calculation of precision for PM detection = 784/(3 + 784 + 9)). The highest recall (98.00%), precision (98.49%), and F-measure (98.27%) were reported during internal validation and the lowest recall of 86.49% was calculated in external-III validation, and the lowest precision and F-measure of 85.27% and 86.22%, respectively, were measured with external-I validation for strawberry PM detection. Laboratory evaluation suggested that the ANN classifier performed better with healthy and other diseases image classification resulting in fewer numbers of misclassifications, whereas a comparatively higher misclassification rate was obtained with powdery mildew images.

3.2. Strawberry Field Real-Time Powdery Mildew Detection Results

The machine vision-based powdery mildew disease detection system was tested at three strawberry field sites. The corresponding results are presented in Figure 9, Figure 10, and Figure 11 for field site-I, field site-II, and field site-III, respectively.

Linear regression of manually detected PM vs. real-time PM detection using the machine vision system suggests a highly significant correlation for field site-I (R² = 0.93; p < 0.001; N = 12) (Figure 9). The system also showed a significant correlation with the other two fields (field site-II: R² = 0.88; p < 0.001; N = 12, field site-III: R² = 0.92; p < 0.001; N = 12) (Figure 10 and Figure 11). The MAE and RMSE were also calculated from all field tests. The results showed that higher MAE and RMSE were obtained from field site-I test (4.00/row and 4.12/row). Lower MAE and RMSE were recorded in validation with field site-III, with values of 2.83/row and 3.00/row, respectively. The MAE calculated values from three fields were lower than RMSE values because MAE does not give undue importance to large errors [35]. The co-efficient of determination (R²) was higher in field site-I and the MAE and RMSE values were also higher compared to the other two fields. This result was due to the larger number of PM-affected plants and longer length of rows in field site-I compared to the other field sites. The lowest R² value was 0.88, generated from field site-II, due to overestimates by the machine vision system accounting white patch symptoms (i.e., spider mites) as powdery mildew disease. The underestimates were likely due to the higher density of plant leaves, resulting in powdery mildew-affected leaves being hidden under leaves. The variations in plant leaf density may have also contributed to the machine vision system accuracy. Palleja and Landers [36] noticed that the plant density could account for errors in image processing, while others have encountered issues with predicting harvest ripeness of blueberries using an image processing-based blue-ratio algorithm [37].

A direct comparison of manually and automatically detected powdery mildew disease in strawberry fields are presented in Figure 12.

The wind speed has direct effects upon real-time machine vision system performance under field conditions. Zhang and Chaisattapagon [38] developed a machine vision system for weed identification which was successful during laboratory experiments, but they pointed out that it was not practical for real-time weed detection, especially when a wind effect was considered. The wind speed can cause blurring boundaries in the images which becomes quite complicated for image analysis [39]. The more pronounced fluctuations for powdery mildew detections in our study were observed in the field site-II testing of strawberry rows (12–24 rows) under high wind speeds (7–14 km h⁻¹). Comparatively, lower fluctuations were reported in field site-I (1–12 rows) results where the wind speed was 4–10 km h⁻¹, followed by field site-III (25–36 rows) with a wind speed of 3–8 km h⁻¹. Since the wind speed was comparatively higher in field site-II, the movement of the branches and the leaves during real-time testing was a bit faster. Plant leaf movement during frame to frame image capture may have also resulted in a false match, which was similar to observations reported by Shrestha and Steward [40]. The number of affected plants were higher in field site-I compared to other two field sites due to longer row (220 m each) compared to other fields (180 m each) leading to misclassification rates of some strawberry rows being higher. Some of the misclassifications may be associated with spider mites that leave a white spider net over the leaves that the image processing algorithm identifies as PM.

The results of A paired-sample t-test suggested that a statistically significant difference existed between the two measurements (manual and automatic) for powdery mildew detection. At all field sites, manual identification of PM was always greater than the machine vision system implemented in the strawberry fields. The difference in mean detection of PM between manual and machine vision (automatic) at field site-I was 4.00 (95% CI, 3.34 to 4.66; p < 0.001), for field site-II, it was 3.42 (95% CI, 2.46 to 4.37; p < 0.001), and for field site-III, it was 2.83 (95% CI, 2.18 to 3.48; p < 0.001) (

Table 6. Pair-wise t-test for manually and automatically powdery mildew disease detection.

Fields	Detections	Mean (Points)	S.D. (Points)	Mean Diff. (Points)	p-Value
Field site-I	Manual	22.33	3.96	4.00	<0.001 a
	Automatic	18.33	3.65
Field site-II	Manual	18.67	4.12	3.42	<0.001 a
	Automatic	15.25	3.39
Field site-III	Manual	14.83	3.38	2.83	<0.001 a
	Automatic	12.00	2.92

^a Significant at a probability level of p = 0.05.

).

The real-time field evaluation results demonstrated that the machine vision image processing system could achieve a high recall (82.09%), precision (87.65%), and F-measure values (85.09%), specifically at field site-I (

Table 7. Real-time field evaluation results of developed machine vision system.

	Accuracy Parameters [s]
	${\mathrm{T}}_{\mathrm{p}}\left(\mathbf{Points}\right)$	${\mathrm{F}}_{\mathrm{p}}\left(\mathbf{Points}\right)$	${\mathrm{F}}_{\mathrm{n}}\left(\mathbf{Points}\right)$	Recall (%)	Precision (%)	F-measure (%)
Field I	220	31	48	82.09	87.65	85.09
Field II	183	27	41	81.70	87.14	84.64
Field III	144	26	34	80.90	84.71	82.97

[s] ${\mathrm{T}}_{\mathrm{p}}$ means true positive; ${\mathrm{F}}_{\mathrm{p}}$ means false positive; ${\mathrm{F}}_{\mathrm{n}}$ means false negative.

). The accuracy parameters at the other two field sites were comparable, despite having differences in field row lengths and wind conditions. Two types of statistical errors were noticed during the field evaluations (e.g., type-I error and type-II error). The type-I error, also called a false positive, and the type-II error, known as a false negative, are likely to occur under conditions when variables, such as climate, lighting, and environmental conditions, cannot be well controlled. In type-I error, machine vision system detected PM diseases when there were healthy or other diseased leaves present. The type-II errors occurred when the machine vision system confirmed no PM disease on the leaf but the manual detection identified it as being present. In the long run, reducing type-II error is important in order to minimize the spread of PM from one plant to another. In this experiment, the type-II error was comparatively higher than type-I error at field site-I and field site-II because the machine vision system mistook for PM disease detection and accounted PM as spider mites in some spots.

Although our real-time machine vision imaging system provided good insights for field-based detection of PM in strawberry fields, a number of challenges were also identified. Previous studies also identified challenges on their experiments in field conditions [41,42,43]. Jeon et al. [42] was able to detect 72.6% of crop correctly using ANN with a real-time crop and weed segmentation algorithm. Other researchers attempted to overcome the issues arising under field conditions by developing an environmentally adaptive segmentation algorithm, but it was only able to correctly identify 45% to 67% of tomato cotyledons [43]. This study detected over 80% of powdery mildew disease correctly in the three commercial strawberry fields tested due to errors caused by uncontrolled environment. A high frame per second (FPS) camera would have reduced effects of windy conditions in the field experiments but managing for other sources of error for real-time detection is still a major area of study. Faster processing also needs to be considered while using high-resolution images.

3.3. Prescription Map of Field Evaluations

Strawberry powdery mildew disease detection results from the three commercial strawberry field sites were mapped in ArcGIS 10.5 software (Figure 13, Figure 14, and Figure 15). The map consists of real-time machine vision system detected points that are marked with red bullets and manually detected points, represented by a star. The overlaid markers were georeferenced and compared to evaluate the accuracy of the machine vision detection system relative to the manual detection. Subjective analysis of prescription maps indicated that the system detected powdery mildew disease with an 82.09%, 81.70%, and 80.90% accuracy for field site-I, field site-II, and field site-III, respectively. The accuracies were calculated using the ratio of total correctly detected points with the machine vision system relative to the total manual detected points. Figure 13 identified that the presence of powdery mildew was greater in the western and eastern parts of field site-I compared to the central portion of the field. Correspondingly, Figure 14 identified that the western and central western parts of field site-II were more severely affected than other areas, and Figure 15 indicated that field site-III was severely impacted by powdery mildew in the eastern parts of the site. The map revealed that field site III (Figure 15) strawberry plants were less affected by powdery mildew disease compared to field site I (Figure 13) and field site II (Figure 14). The goal of generating the prescription maps is to provide the true scenario of the developed detection system’s effectiveness for PM monitoring and to help in understanding the distribution of disease throughout the fields. Another reason is to provide decision support for the spraying machine, which will be considered in our future study.

Overall, a total of 17.91% powdery mildew affected points were missed in field site-I compared to 18.30% points and 19.10% points in field site-II and field site-III, respectively. In contrast, the real-time machine vision system inaccurately detected powdery mildew points by 10.37%, 10.76%, and 12.75% for field site-I, field, site-II and field site-III, respectively. Despite our system can only detect over 80% of PM disease in the field evaluations as stated earlier, the developed vision system can be used onsite since it saves time and costs for PM disease monitoring with reliable observation. An over 80% of accuracy is sufficient to be claimed as useful for the strawberry growers because till date, there is no such technology available, however, improvement of the system is expected in the future considering various environmental factors. Additionally, manual monitoring is a time-consuming and costly approach since the diagnosis requires very careful observation. A higher risk of human error is also associated with manual monitoring due to the scout’s fatigue for continuously working in the field. On the other hand, this custom-based system is a feasible and reliable option for minimizing PM disease outbreak through monitoring symptoms rapidly and providing warnings about disease risk. Prescription maps have been used by many researchers as a means of visually reporting the detection and site-specific management results of their machine vision systems, particularly for applications of variable rate spraying technologies in different cropping systems [44,45,46], crop disease mapping [47], weed mapping [48,49], site-specific management [50,51,52], and spot applications [53,54]. Miller et al. [46] applied granular fertilizer in citrus and observed that GPS map-based control were similar to those found for real-time control. Prescription maps also incorporate well with real-time machine vision systems in different cropping systems to detect weeds, bare spots, and fruits [53,55,56]. Hence, the generation of prescription map based on disease outbreaks in the field could be used to support site-specific management for variable rate fungicide spraying by strawberry growers. In this study, it is aimed to detect the suspected region of the strawberry PM disease symptom-based on a mobile machine vision system, and it was possible that automatically detected PM symptom area was visible by the naked eyes of field scouts. It can improve the detection performance by reducing the time and cost as compared to the manual monitoring and also by minimizing scout’s fatigue in the field diagnosis.

4. Conclusions

Field-based evaluation of strawberry fields via a novel real-time mobile machine vision powdery mildew detection system, using image texture analysis, was successful. The real-time vision system detected PM disease with F-measure values averaging 84.23% across three commercial strawberry field sites. A custom CCM-based image processing algorithm in conjunction with ANN was used to detect the PM disease in strawberry fields. Two RGB cameras were used to continuously capture images from the fields. Acquired images were converted to green ratio, hue, saturation, and intensity images before generating CCM from each. The CCM was generated from each converted image and then, textural features were extracted. A total of 23 texture features, along with ANN classifier, were utilized in the machine vision system development. Manually measured powdery mildew detected points were compared with the real-time PM detected points using the machine vision system. Results of linear regression plots showed significant correlations between manually and automatically detected PM points. The MAE and RMSE of the detection results were 4.00, 3.42, and 2.83 per row and 4.12, 3.71, and 3.00 per row for field site-I, field site-II, and field site-III, respectively. Real-time field evaluation results of this study reported that the developed machine vision system achieved a high recall of 82.09%, precision of 87.65%, and F-measure of 85.09% during testing at field site-I compared to other two field sites. The automatic PM disease detection system was fairly accurate; however, some misclassifications were also reported in places. Environmental variability, especially spatial variability of wind speeds, effects of leaf overlap or differences in density, and presence of spider mites, resulted in slightly higher misclassification rates than desired. The advancement of a commercial real-time machine vision aided PM detection system needs to consider how to address these variables in future studies. Despite the challenges identified in our study, the relatively low cost of developing this system and the ease of implementation demonstrated the real potential for providing a real-time decision support system for strawberry growers.