Modified Barnacles Mating Optimization with Deep Learning Based Weed Detection Model for Smart Agriculture

Abstract

Weed control is a significant means to enhance crop production. Weeds are accountable for 45% of the agriculture sector’s crop losses, which primarily occur because of competition with crops. Accurate and rapid weed detection in agricultural fields was a difficult task because of the presence of a wide range of weed species at various densities and growth phases. Presently, several smart agriculture tasks, such as weed detection, plant disease detection, species identification, water and soil conservation, and crop yield prediction, can be realized by using technology. In this article, we propose a Modified Barnacles Mating Optimization with Deep Learning based weed detection (MBMODL-WD) technique. The MBMODL-WD technique aims to automatically identify the weeds in the agricultural field. Primarily, the presented MBMODL-WD technique uses the Gabor filtering (GF) technique for the noise removal process. For automated weed detection, the presented MBMODL-WD technique uses the DenseNet-121 model as feature extraction with the MBMO algorithm as hyperparameter optimization. The design of the MBMO algorithm involves the integration of self-population-based initialization with the standard BMO algorithm. At last, the Elman Neural Network (ENN) method was applied for the weed classification process. To demonstrate the enhanced performance of the MBMODL-WD approach, a series of simulation analyses were performed. A comprehensive set of simulations highlighted the enhanced performance of the presented MBMODL-WD methodology over other DL models with a maximum accuracy of 98.99%.

1. Introduction

Agriculture is confronting enormous difficulties, which include threats from diseases, weeds, and pests, varying climate, severe scarcity of water resources and arable lands, etc. [1]. Many efforts were exerted to weed control over the years by farmers and researchers to solve the difficulties brought by weeds [2]. Weeds appear everywhere arbitrarily in the domain and compete with crops for sunlight, water, and nutrients, which causes harmful effects on crop quality and yields if not controlled properly. Many studies have illustrated a strong correlation between weed competition and crop yield loss [3]. There are different weed varieties that are detrimental to crop productivity and should be identified in the initial stage of growth. The weed’s growth in the crop would affect fundamental resources such as minerals, sunlight, water, fresh air, soil, etc., which are the fundamental necessities of the crop [4]. It has been found that 35% of crops are ruined because of the growth of various weeds in the agricultural field. This study aimed to learn various techniques and tools employed by the researchers to classify and detect weeds, which are essential for assessing the growth of weeds [5]. Numerous computer-related approaches, such as wireless sensor networks (WSN), artificial intelligence (AI), and some other approaches, are used that enhance agriculture.

Weed detection in plants is now becoming a great challenge since there is not much effort put into weed detection. Techniques to realize field weed identification through computer vision (CV) technology primarily use conventional deep learning (DL) and image processing [6]. If weed identification has been carried out by conventional image-processing technology, then feature extraction, such as shape, color, and texture of the image, and integration with conventional ML classifiers, such as the Support Vector Machine (SVM) or random forest (RF) algorithm, for weed detection are essential [7]. Such techniques must devise features manually and highly depend on pre-processing methods, quality of feature extraction, and image acquisition algorithms. Classical algorithms for detecting agricultural weeds had a focus on direct identification of the weed; yet, there were substantial variances in weed species [8]. With the advances in computational power and growth in data volume, DL methodologies are capable of extracting multidimensional and multiscale spatial semantic feature information of weeds by using the Convolutional Neural Network (CNN) because of its enhanced data expression abilities for images, evading demerits of conventional extracted approaches [9,10]. Hence, CNN models have gained higher attention in the research community.

This manuscript introduces a Modified Barnacles Mating Optimization with Deep Learning based weed detection (MBMODL-WD) technique. The presented MBMODL-WD technique initially pre-processes the input images via the Gabor filtering (GF) technique to eradicate the noise. For automated weed detection, the presented MBMODL-WD technique uses the DenseNet-121 model as feature extractor. Moreover, the MBMO algorithm is used for the hyperparameter optimization process. At last, the Elman Neural Network (ENN) method was applied for the classification of images into plants and weeds. To demonstrate the enhanced performance of the MBMODL-WD approach, a series of simulation analyses were performed.

2. Literature Review

Sodjinou et al. [11] introduce a segmentation approach relevant to the compilation of K-means and semantic segmentation methods for segmenting weeds and crops in color images. The two distinct databases of agronomic images are employed for segmenting methods. Everything except the plants has been eliminated from the images with the use of the threshold method. Then, utilizing U-net and the K-means subtractive approach for the segmentation of weeds and crops, semantic segmentation has been implemented. A U-net for the segmentation of weeds and wheat on images was given in [12]. An image classifier task has been employed for choosing the backbone network for the encoding part. The abovementioned task on a similar dataset will be exploited for pretraining and choosing the decoding network. The training process implemented TL. Sa et al. [13] formulates a weed segmentation and mapping structure that processes multispectral images received from drones by utilizing deep neural networks (DNNs). Many researchers are increasingly making an effort in weed or crop semantic segmentation just by considering single images for classification and processing.

A weed species and density evaluation approach rely upon an image semantic segmentation NN that has been modeled in [14]. To train the network, an amalgamation of fine-tuning and pretraining training approaches has been employed. The pretraining datasets are the images that just have a single weed species in a single image. The weeds will be labeled mechanically by an image segmentation approach related to the minimum error threshold and Excess Green (ExG). The fine-tuning data are real imageries comprising many crops and weeds and are manually labeled. In [15], an AI-oriented method has been modeled to find weeds’ unintentional growth on agricultural fields and rise the control rate, concerning technological advances. This presented technique will train the dataset by utilizing pretrained DL techniques and has been optimized through metaheuristic optimization approaches through the selection of the species-related activations that can be sent to Softmax, which was in the final layer of DL techniques.

In [16], a technique related to DL has been modeled for weed segmentation from the images. This approach may segment weeds from crops and soil in the images. This semantic segmentation approach has been advanced using a simplified U-net. An image augmentation method has been devised because of the difficulty of image labeling for the semantic segmentation of weeds. The semantic segmentation network is trained by a two-stage training technique made up of fine-tuning and pretraining. Abdalla et al. [17] intended to assess three TL approaches with the use of a VGG16-oriented encoder net for segmenting the oilseed rape images in a field including high-density weeds. Three TL methods utilizing a VGG16-based encoder method have been modeled, and their performances will be compared with a VGG19-related encoder net.

3. The Proposed Model

In this article, we have introduced a new MBMODL-WD technique for the automated identification of weeds using computer vision techniques and improved crop productivity. It encompasses a series of processes: GF-based image pre-processing, DenseNet-121 feature extractor, MBMO-based hyperparameter optimizer, and ENN-based classification. Figure 1 demonstrates the block diagram of the MBMODL-WD system.

3.1. Image Pre-Processing

Primarily, the presented MBMODL-WD technique removes noise using the GF technique. Gabor filter has been employed in the presented system for enhancing the ridges and relaxing the valleys through the enforcement of short-term Fourier transformation including Gaussian window in the spatial domain [18]. It helps to gain deviations in characteristics and textures in the fingerprint images for distinct scales and orientations. Such statistical features generated image features that can be significantly accentuated by utilizing the frequency information and orientations in fingerprint images by fine-tuning a Gabor filter. A set of Gabor filters was employed on image $I\left(x, y\right)$ in distinct frequencies having distinct orientations, utilizing the Gabor function $g(x,$ y), as defined by Equation (1).

$$g\left(x, y\right)= \exp \left(-\frac{{x^{\prime }}^2+{\mathsf{\gamma }}^2{y^{\prime }}^2}{2{\sigma }^2}\right)\cos \left(2\pi \frac{X^{\prime }}{l}+\varphi \right)$$

(1)

Here ${𝜒}^{\prime }=xcox\theta +ys\dot{m}\theta$ and $y^{\prime }=ycox\theta -xs\dot{m}\theta$.

This Gabor transform will be applied in the Gaussian envelope $\sigma$, in addition to the $\chi$ and $y$ directions.

3.2. Feature Extraction

For automated weed detection, the DenseNet-121 model produces feature vectors. A CNN accomplishes higher performance in the domain of image classification [19]. However, training a CNN from scratch is not easier because classifier accuracy based on hyperparameter tuning, namely learning rate, initial weight, number of epochs, optimizers, and dropout, needs a wide-ranging amount of labeled training datasets and higher computation power. This problem can be leveraged by means of the TL method. In the TL model, the training time can be decreased by the weight attained from the pretrained mechanism that is applied as an initial weight for training the novel problem. This process of reusing the pretrained weight leads to lower generalization errors. The DenseNet framework is popular since the DenseNet module motivates feature reuse, attenuates the gradient vanishing problems, decreases the parameter count, and enhances feature propagation. In DCNN, all the layers are interconnected to other layers as a feed forward pattern. Every layer in DenseNet accepts the feature map of each previous layer as other inputs and passes on its own feature map to every succeeding layer. Therefore, the $n$-$\mathrm{th}$ layer has $n$ inputs of each previous layer.

In general, a CNN changes the size of feature maps via the down-sampling of layers. However, DenseNet facilitates feature concatenation and down-sampling by separating the network into multiple densely connected dense blocks. The feature map size in the block is unchanged. Inside the dense block, the size of the feature maps is similar which assists in performing concatenation while outside the dense block, while pooling and convolution operations were performed to down-sample. At the end of every dense layer, a transition block or layer was added. The transition layer comprises a batch normalization layer, a 1 × 1 convolutional, and 2 × 2 average pooling layers. The transition layer changes the size of the feature map. Therefore, the DenseNet comprises 1 classification, 117 Conv, and 3 transitions, making the size of the layer 121 [19].

Here, the MBMO algorithm as a hyperparameter optimizer is applied. Barnacles are certain kinds of arthropods which constitute an infraclass Cirripedia based on crabs and lobsters [20]. A barnacle is a marine animal that lives in shallow and tidal waters. They can be found all around the seawater and are raised on hard surfaces in seawater. After they hatch eggs, barnacle larvae were disseminated in the water to find and stick to hard surfaces. Indeed, hard surfaces cover the bodies of barnacles and enhance shell plates. They must seek a balance between managing ever-longer penises and accomplishing more mates in a turbulent flow. A novel optimization technique, named the barnacles mating optimizer (BMO) technique, based on these behaviors has been introduced. The balance behavior can be devised depending upon the Hardy–Weinberg equilibrium as follows:

The initial population of barnacles for the solution can be determined by:

$$X=\left\{\begin{array}{ll}{\chi }_1^1\hfill & {\chi }_1^N\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill \\ {\chi }_n^1\hfill & {\chi }_n^N\hfill \end{array}\right\}$$

(2)

where $n$ defines candidate number, and $N$ denotes count of decision parameters according to lower and higher bounds:

$$l_b=\left[l_b^1, \dots , l_b^i\right]$$

(3)

$$u_b=\left[u_b^1, \dots , u_b^i\right]$$

(4)

From the expression, $u_b$ and $l_b$ indicate the upper and lower bounds of variable $i$. By assessing the objective function for each candidate, better outcomes to worst outcomes are stored and arranged at the initial iteration.

The presented method involves exploration and exploitation. The offspring generation can be performed by the sperm cast method as an exploration term:

$$b_D=rand\left(n\right)$$

(5)

$$b_M=rand\left(n\right)$$

(6)

where $b_D$ and $b_M$ denote the mated parents.

Depend upon the Hardy–Weinberg theory, the BMO method considers the parents’ genotype frequencies or inheritance features in the offspring generation to model the reproduction process:

$$X_i^{N_{new}}=p{X}_{b_D}^N+q{X}_{b_M}^N$$

(7)

Now, $X_{b_M}^N$ and $X_{b_D}^N$ characterize the variables of Mum and Dad candidates, correspondingly, and p defines a pseudo-random value disseminated between 0 and 1, $=\left(1-p\right)$.

When the candidate choice for mating exceling $pl$ amount is taken into account initially, then the exploration term can take place:

$$X_i^{N_{new}}=rand\times X_{b_M}^n$$

(8)

In Equation (8), $rand$ defines a random integer between zero and one. The recently produced offspring for exploration can be created by Mum’s candidate.

The offspring will be added and analyzed to the parents to extend the solution matrices from the candidate dimension. Hence, to choose fifty percent top solutions, a technique was utilized for arranging individual dimension, and the inappropriate solution was removed. The MBMO algorithm involves the integration of self-population-based initialization with the standard BMO algorithm. Similar to other metaheuristic models, BMO has a population-related optimization method that begins with random initialization. This implies that it needs a control variable to determine the population size. Nevertheless, it is worth noting that the selection of population size to resolve case problems becomes difficult. The self-adaptive population will regulate the population size at the iteration. Now, the initial population size in the initial iteration can be accomplished by a self-adaptive population:

$$PopSize=10\times d$$

(9)

where $d$ signifies the problem dimension and it can be defined as follows:

$$PopSiz{e}_{new}= \max \left(d, round\left(PopSize+r\times PopSize\right)\right)$$

(10)

where $r$ defines a random number between $-0.5$ and 0.5.

Algorithm 1: Pseudocode of BMO

Initializing the population of barnacles $X_i$ Compute the fitness of all the barnacles Arrange for locating an optimum outcome at the top of populations (T = the optimum solution) while (I < Maximal iterations)                Fixed the value of $pl$                if selective of Dad and Mum = $pl$                               for all the variables                                              Off spring generation:                               end for                else if selective of Dad and Mum > $pl$                               for all the variables                                              Off spring generation:                end for end if                Bring the present barnacle back once it moves outside the boundaries                Compute the fitness of all the barnacles                Arranging and upgrading T if there is an optimum solution                l = l + 1 end while Return T

3.3. Weed Detection and Classification

For the weed recognition process, the MBMODL-WD technique exploited the ENN model. The ENN model is a common example of a dynamic recurrent network, and its architecture comprises an input layer with specific context nodes, an output, and hidden layers [21]. The key advantage of ENN is that the context node might be employed for remembering the prior hidden node activation, which makes it appropriate in the fields of dynamic system identification and prediction control. Consider the external input of the network as $u$. The output is $y$, and the output of the hidden layer is $\chi$. Hence:

$$x\left(k\right)=f\left[w_k^1{x}_c\left(k\right)+w_k^{2}u\left(k-1\right)\right]$$

(11)

$$x_c\left(k\right)=x\left(k-1\right)$$

(12)

$$y\left(k\right)=g\left[w_k^{3}x\left(k\right)\right]$$

(13)

where $w_k^1{w}_k^2,$ and $w_k^3$ indicate the weight connection matrices from context to implicit layers, the input to hidden layers, and hidden to output layers, correspondingly. Now, $f$ and $g$ represent the transfer function of implicit and output layers.

$$x_c\left(k\right)=x\left(k-1\right)=f\left[x_{k-1}^1{x}_c\left(k-1\right)+x_{k-1}^{2}u\left(k-2\right)\right]$$

(14)

Then,

$$x_c\left(k-1\right)=x\left(k-2\right)$$

(15)

where $x_c\left(k\right)$ depends on $w_{k-1}^1,w_{k-2}^2$ at distinct moments; hence, $x_c\left(k\right)$ represents the dynamic recursive method. Consequently, the BP model applied to Elman regression NN training is the dynamic BP learning mechanism.

4. Experimental Validation

The proposed model is simulated using the Python 3.6.5 tool. The proposed model is experimented on PC i5-8600k, GeForce 1050Ti 4 GB, 16 GB RAM, 250 GB SSD, and 1 TB HDD. The parameter settings are given as follows: learning rate: 0.01, dropout: 0.5, batch size: 5, epoch count: 50, and activation: ReLU.

The weed detection results of the MBMODL-WD model are tested using a dataset, compassing 3000 images. The dataset holds 287 crop images and 2713 weed images, as defined in

Table 1. Dataset details.

Class	No. of Images
Crop	287
Weed	2713
Total Number of Images	3000

. Figure 2 illustrates some sample images.

The confusion matrices of the MBMODL-WD model on the weed detection process are demonstrated in Figure 3. The outcomes displaying the MBMODL-WD model properly recognized the crop and weed images under all aspects.

Table 2. Weed detection outcome of MBMODL-WD system under 60:40 of TR/TS databases.

Class	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{b}\mathit{a}\mathit{l}}$	$\mathit{P}\mathit{r}\mathit{e}{\mathit{c}}_{\mathit{n}}$	$\mathit{R}\mathit{e}\mathit{c}{\mathit{a}}_{\mathit{l}}$	${\mathit{F}}_{\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}$	MCC	${\mathit{G}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
Training Phase (60%)
Crop	90.62	88.41	90.62	89.51	88.48	94.64
Weed	98.84	99.08	98.84	98.96	88.48	94.64
Average	94.73	93.75	94.73	94.23	88.48	94.64
Testing Phase (40%)
Crop	85.83	96.46	85.83	90.83	90.01	92.47
Weed	99.63	98.34	99.63	98.98	90.01	92.47
Average	92.73	97.40	92.73	94.91	90.01	92.47

and Figure 4 report the weed detection results of the MBMODL-WD method on 60% of TR and 40% of TS databases. The simulation values revealed the MBMODL-WD method properly classified the crop and weed images. On 60% of TR database, the MBMODL-WD model reached an average $acc{u}_{bal}$ of 94.73%, $pre{c}_n$ of 93.75%, $rec{a}_l$ of 94.73%, $F_{score}$ of 94.23%, MCC of 88.48%, and $G_{mean}$ of 94.64%. Concurrently, on 40% of TS database, the MBMODL-WD approach gained an average $acc{u}_{bal}$ of 92.73%, $pre{c}_n$ of 97.40%, $rec{a}_l$ of 92.73%, $F_{score}$ of 94.91%, MCC of 90.01%, and $G_{mean}$ of 92.47%.

Table 3. Weed detection outcome of MBMODL-WD system under 70:30 of TR/TS databases.

Class	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{b}\mathit{a}\mathit{l}}$	$\mathit{P}\mathit{r}\mathit{e}{\mathit{c}}_{\mathit{n}}$	$\mathit{R}\mathit{e}\mathit{c}{\mathit{a}}_{\mathit{l}}$	${\mathit{F}}_{\mathit{s}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}$	MCC	${\mathit{G}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
Training Phase (70%)
Crop	97.01	96.06	97.01	96.53	96.17	98.29
Weed	99.58	99.68	99.58	99.63	96.17	98.29
Average	98.30	97.87	98.30	98.08	96.17	98.29
Testing Phase (30%)
Crop	98.84	92.39	98.84	95.51	95.08	98.99
Weed	99.14	99.88	99.14	99.51	95.08	98.99
Average	98.99	96.13	98.99	97.51	95.08	98.99

and Figure 5 depict the weed detection results of the MBMODL-WD system on 70% of TR and 30% of TS databases. The simulation outcomes stated that the MBMODL-WD system properly classified the crop and weed images. On 70% of TR database, the MBMODL-WD system reached an average $acc{u}_{bal}$ of 98.30%, $pre{c}_n$ of 97.87%, $rec{a}_l$ of 98.30%, $F_{score}$ of 98.08%, MCC of 96.17%, and $G_{mean}$ of 98.29%. Simultaneously, on 30% of TS database, the MBMODL-WD algorithm attained an average $acc{u}_{bal}$ of 98.99%, $pre{c}_n$ of 96.13%, $rec{a}_l$ of 98.99%, $F_{score}$ of 97.51%, MCC of 95.08%, and $G_{mean}$ of 98.99%.

The TACC and VACC of the MBMODL-WD approach are investigated on weed detection performance in Figure 6. The figure states that the MBMODL-WD methodology revealed an improved performance with higher values of TACC and VACC. It is evident that the MBMODL-WD system has attained superior TACC outcomes.

The TLS and VLS of the MBMODL-WD methodology are tested on weed detection performance in Figure 7. The figure points out that the MBMODL-WD approach has exposed optimum performance with decreased values of TLS and VLS. It is noticeable that the MBMODL-WD method has resulted in lesser VLS outcomes.

A noticeable precision–recall study of the MBMODL-WD system under test database is represented in Figure 8. The figure pointed out that the MBMODL-WD algorithm has enhanced values of precision–recall values in various classes.

A comprehensive ROC investigation of the MBMODL-WD methodology under test database is depicted in Figure 9. The results denoted the MBMODL-WD algorithm has displayed its capability in categorizing in various classes.

Table 4. Comparative analysis of MBMODL-WD system with other algorithms.

Methods	$\mathit{A}\mathit{c}\mathit{c}{\mathit{u}}_{\mathit{b}\mathit{a}\mathit{l}}$	$\mathit{P}\mathit{r}\mathit{e}{\mathit{c}}_{\mathit{n}}$	$\mathit{R}\mathit{e}\mathit{c}{\mathit{a}}_{\mathit{l}}$	MCC	${\mathit{G}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
MBMODL-WD	98.99	96.13	98.99	95.08	98.99
RF Model	95.53	95.24	93.93	88.63	89.87
KNN Model	62.81	62.43	61.85	35.83	93.29
SVM Model	94.39	91.32	91.24	84.00	90.34
ResNet-101 Model	93.52	93.15	94.47	92.90	92.80
VGG-16 Model	93.29	93.96	92.76	92.93	89.07
SVM-Pixel-based	85.84	85.41	85.79	86.81	93.62
HLBODL-WDSA	98.96	95.84	95.18	93.34	95.16

reports a detailed weed detection result of the MBMODL-WD model and existing models [22]. Figure 10 examines the $pre{c}_n$ and $rec{a}_l$ results of the MBMODL-WD method and recent methods. The results demonstrated the MBMODL-WD method reached enhanced values of $pre{c}_n$ and $rec{a}_l$. For instance, based on $pre{c}_n$, the MBMODL-WD model has shown a higher $pre{c}_n$ value of 96.13%. In contrast, the RF, KNN, SVM, ResNet-101, VGG-16, SVM-Pixel-based, and HLBODL-WDSA models have shown reduced $pre{c}_n$ of 95.24%, 62.43%, 91.32%, 93.15%, 93.96%, 85.41%, and 95.84%, respectively. In addition, in terms of $rec{a}_l$, the MBMODL-WD approach has exposed a maximally $rec{a}_l$ value of 98.99%. In contrast, the RF, KNN, SVM, ResNet-101, VGG-16, SVM-Pixel-based, and HLBODL-WDSA models have shown lesser $rec{a}_l$ values of 93.93%, 61.85%, 91.24%, 94.47%, 92.76%, 85.79%, and 95.18%, correspondingly.

Figure 11 observes the MCC and $G_{mean}$ results of the MBMODL-WD algorithm and recent approaches. The outcomes exhibited that the MBMODL-WD system gained improved values of MCC and $G_{mean}$. For instance, with respect to MCC, the MBMODL-WD model outperformed a maximal MCC value of 95.08%. In contrast, the RF, KNN, SVM, ResNet-101, VGG-16, SVM-Pixel-based, and HLBODL-WDSA systems outperformed lower MCC values of 88.63%, 35.83%, 84%, 92.90%, 92.93%, 86.81%, and 93.34%, correspondingly.

Moreover, in terms of $G_{mean}$, the MBMODL-WD system has depicted a superior $G_{mean}$ value of 98.99%. In contrast, the RF, KNN, SVM, ResNet-101, VGG-16, SVM-Pixel-based, and HLBODL-WDSA algorithms have demonstrated minimal $G_{mean}$ values of 89.87%, 93.29%, 90.34%, 92.80%, 89.07%, 93.62%, and 95.15%, correspondingly. These results reassured the superior outcomes of the MBMODL-WD model.

5. Conclusions

In this article, we have modeled a new MBMODL-WD technique for the automated identification of weeds using computer vision techniques and improving crop productivity. Primarily, the presented MBMODL-WD technique removes noise using the GF technique. For automated weed detection, the DenseNet-121 model produces feature vectors with the MBMO algorithm as hyperparameter optimizer. At last, the ENN method was applied for the classification of images into plants and weeds. To demonstrate the enhanced performance of the MBMODL-WD approach, a series of simulation analyses were performed. A comprehensive set of simulations highlighted the enhanced performance of the presented MBMODL-WD methodology over other DL models, with a maximum accuracy of 98.99%. In the future, the performance of the proposed model can be improved using hybrid DL classification models. In addition, the computational complexity of the proposed model needs to be examined in the future.