Evolving CNN with Paddy Field Algorithm for Geographical Landmark Recognition

Abstract

Convolutional Neural Networks (CNNs) operate within a wide variety of hyperparameters, the optimization of which can greatly improve the performance of CNNs when performing the task at hand. However, these hyperparameters can be very difficult to optimize, either manually or by brute force. Neural architecture search or NAS methods have been developed to address this problem and are used to find the best architectures for the deep learning paradigm. In this article, a CNN has been evolved with a well-known nature-inspired metaheuristic paddy field algorithm (PFA). It can be seen that PFA can evolve the neural architecture using the Google Landmarks Dataset V2, which is one of the toughest datasets available in the literature. The CNN’s performance, when evaluated based on the accuracy benchmark, increases from an accuracy of 0.53 to 0.76, which is an improvement of more than 40%. The evolved architecture also shows some major improvements in hyperparameters that are normally considered to be the best suited for the task.

1. Introduction

Currently, convolutional neural networks (CNNs) are used as one of the fundamental techniques for image-processing tasks in artificial intelligence (AI) [1]. Convolutional neural networks (CNNs) are the most widely used architecture for image recognition, retrieval, and processing tasks. CNNs are used for convolving images into newer images so that we can get a better feel for the information stored in a particular digital image. The convolution operation was introduced by the then-postdoctoral researcher Yann LeCun in 1980 [2]; today, it has become one of the most commonly used operations in computing. Developers and researchers have to use this operation while dealing with computer vision tasks. Today, CNNs have become part and parcel of the method of dealing with image processing.

CNNs are part of the deep learning paradigm in AI; deep learning includes many architectures known as artificial neural networks (ANNs). ANNs are similar to the neural networks that exist in our bodies [3]. These architectures process data in the same way that the neural networks of our body process sensory information. In an ANN, many hyperparameters exist that must be fine-tuned so that we can achieve the best possible results. To get the best output, we perform a neural architecture search (NAS) to establish the best architecture for a CNN [4].

Search and optimization have existed throughout human history and algorithms known as metaheuristics have been developed [5]. Metaheuristics are generalized algorithms that are used to compute the best solutions to problems, even those that are NP-Hard [6]. Hundreds of metaheuristics have now been developed, one of which is the paddy field optimization algorithm [7]. The paddy field algorithm was based on the theory of the spread of seeds of a paddy crop and how they find the best place to grow.

For NAS, we created a PFANET, i.e., a paddy field algorithm network. We evolved the network and established the best architecture by finding the best-suited hyperparameters for a geographical landmark recognition task. The geographical landmarks dataset was derived from the Google Landmarks Dataset V2 and was augmented to allow the further improvement of results [8]. The contribution of this article is as below:

(i)

We proposed a paddy field algorithm-based approach to evolve an optimized CNN architecture.

(ii)

We validated the proposed approach to landmark recognition and its application.

Section 1 of this article introduces the motivation behind the work. Section 2 delves deeper into CNNs and deep learning. Section 3 explores other NAS works. Section 4 elucidates the paddy field algorithm, while Section 5 discusses the experimentation and results and Section 6 offers our conclusions.

2. Convolutional Neural Networks

ANNs are a network of many activation functions that work together to achieve data processing. Networks are trained in such a manner that the appropriate function is activated from a function in the previous layer [9]. The architecture includes many layers and performance is usually better with an increase in the number of layers [10]. This has made ANNs one of the best choices for machine learning (ML).

However, not all types of data can be handled by all types of ANNs. The digital images need to be handled by a type of ANN called a convolutional neural network (CNN). A digital image contains 2 dimensions of pixelated information, where each pixel itself contains one color out of millions of colors [11]. An 8-bit RGB encoding scheme contains around 16 M colors and each pixel contains one of these colors [12]. To handle this volume of data in tensor form, CNNs were developed.

A typical CNN takes parameters in the tensor form of:

$$x\in {ℝ}^{\left(C*W*H*n\right)}$$

(1)

where $x$ designates the input in the network, $C$ designates the color scheme in vector form, $W$ defines the width of an image in pixels, $H$ defines the height of the image, and n is the number of images provided to the network [13].

When the network is trained, the activation functions that are mostly used are the Softmax activation functions:

$$\sigma {\left(z\right)}_i=\frac{e^{z_i}}{{\mathsf{\Sigma }}_{j=1}^k{e}^{z_j}}$$

(2)

where $\sigma$ is the softmax activation function, $z_i$ is the input vector, $e^{z_i}$ is the standard exponential function for the input vector, $k$ is the number of classes in the multi-class classifier and $e^{z_i}$ is the standard exponential function for output vector [14].

The network trains itself in the backpropagation phase, wherein the information about correctness is fed back into the network and appropriate measures have to be taken. These measures are known as in-built optimizers; currently, many optimizers exist, like gradient descent, stochastic gradient descent, Adagrad, AdaDelta, and Adam [15,16,17,18,19,20]. We have used AdaDelta optimization to improve the network. The equation for AdaDelta optimization is:

$$\begin{array}{c}\eta {’}_t=\frac{\eta }{\sqrt{S_{d{w}_t}+\epsilon }} where S_{d{w}_t}=\beta S_{d{w}_{t-1}}+\left(1-\beta \right){\left(\frac{\partial L}{\partial w_{t-1}}\right)}^2\\ \epsilon is a small+ve number to avoid divisibility by 0\end{array}$$

(3)

where $\eta$ is the learning rate. This changes with respect to $S_{d{w}_t}$, which is the varied gradient descent, and it changes according to the equation shown on the right. Each layer $L$ and weight $w$ affect the optimization.

For evaluating the performance of an AI model, we may use one of the many parameters based on TP—true positive, TN—true negative, FP—false positive, and FN—false negative. True and false refer to the true and false classification of a class, while positive and negative represent two binary classes that can be extrapolated [21].

Some of the available performance metrics are accuracy, precision, recall, and F1 score. These can be explained as follows:

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FP}+\mathrm{FN}+\mathrm{TN}}$$

(4)

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(5)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(6)

$$\mathrm{F}1\mathrm{Score}=\frac{2*\left(\mathrm{Recall}*\mathrm{Precision}\right)}{\mathrm{Recall}+\mathrm{Precision}}$$

(7)

As one would guess, accuracy is the most intuitive parameter for the evaluation of an AI model. Each parameter has its benefits when assessing performance; however, we have used accuracy in this study to evaluate the model, since it is the most acceptable metric for the evaluation of any AI model created, and to question it is highly improbable.

Optimization gives us a hint that more layers can give us more performance. However, it has been seen previously that after a certain number of layers, the performance tends to remain the same or even worsen. For this purpose, various architectures of CNNs were developed, which include ResNets, LeNets, GoogLeNets, VGGNets, Inception Nets, DELF, and many more [22].

In all these architectures, it was seen that as the sizes and variety of data increased, the techniques failed miserably due to their heavy resource consumption [23]. This demanded a newer way of looking at the problem and, today, we can employ evolutionary computing to evolve the architectures of ANNs in an automated fashion. This is known as a neural architecture search (NAS) [24].

3. Neural Architecture Search

NP-Hard problems have existed since the beginning of computing. An NP-Hard problem refers to a problem in the computing arena that is not known to have been solved in polynomial time. No definite amount of looping can guarantee an answer to these problems. For finding a solution to such problems, soft computing is required [25]. Soft computing tends to find the most optimal approximate solution to a problem instead of an exact solution [26]. The best instances of soft computing are evolutionary algorithms, which can be used to find highly optimal solutions to NP-Hard problems.

Artificial neural architectures include millions of parameters that can be varied to produce the best-fitting individuals [27]. However, in the process of evaluating so many parameters in an ANN, we can run into serious resource exhaustion issues. Limits to available time, memory, processing capacity, etc., necessitate a different approach toward the best-fit ANN search.

Here is where the concept of NAS comes into play. The neural architecture search (NAS) amalgamates evolutionary computing and ANNs to produce better-fitting artificial neural networks [28]. These ANNs are evolved on limited data and, when tested on complete data, they prove to be highly proficient. Today, NAS has been tried alongside many techniques like genetic algorithms (GA) and variants of particle swarm optimization (PSO) [29,30].

4. Paddy Field Algorithm

Metaheuristic optimization concerns the use of metaheuristic techniques to solve optimization problems. From engineering to economics, optimization happens almost everywhere. Because money, resources, and time are all finite, making the best use of what is available is critical.

Under varied and difficult constraints, most optimizations are highly nonlinear and multimodal. For a single neural network, different hyperparameters might frequently be at odds. Even for a single goal, there are situations when optimal solutions do not exist. Finding an ideal or even sub-optimal solution is not an easy undertaking, in general.

The paddy field algorithm (PFA) is a metaheuristic that is used to search for the best individuals out of a generated population [31]. It was introduced by Premaratne et al. in 2009, inspired by the biological process of pollination in the seeds of a rice crop [32]. As shown in Figure 1, the paddy field algorithm decides the best solution to a problem, just as seeds spread themselves over a growing area to find the most suitable places to root.

When one paddy crop grows in an area, the future generations of that crop are decided based on fitness, due to pollination. Some seeds distribute in the neighboring areas and the viability of those seeds is at maximum, with maximum fitness. A seed with lesser fitness will produce inferior seeds when it grows. This process repeats until the most optimal population of paddy seeds is established [33]. To avoid the possibility of getting caught in a local optimum, seeds are dispersed widely to search for better spaces. This ensures that most of the search space is searched by the Algorithm 1.

Algorithm 1. Paddy Field Algorithm

Step 1: Initialize a random population of seeds Step 2: Calculate the fitness of each seed as: $$s=q_{max}\left[\frac{y-y_t}{y_{max}-y_t}\right]$$ (8) where the maximum fitness value is expressed as max y, y indicates the fitness value of seeds and $y_t$ indicates the threshold. Step 3: Sort the seeds, in order of fitness. Step 4: Produce seeds from the best individuals, where the best fits produce more seeds. Step 5: Pollinate seeds in the neighboring space: $$S_v=U*s$$ (9) where $0\le U\le 1$. Given a circle of radius a, for two plants Xj and Xk, they will be neighbors to each other if they meet the criteria in Equation (10); thus, determining the neighbor number vj of each plant and the maximum neighbor number of the plant in the same generation, which is expressed as vmax is Equation (11): $$u\left(X_j,X_k\right)=\left|X_j-X_k\right|-a<0$$ (10) $$U_j=e^{\left[\frac{v_j}{v_{max}}-1\right]}$$ (11)

Step 6: Disperse the plants: According to Gaussian distribution, the next generation of seeds produced by each plant is scattered within the parameter space; the positions of the seeds are expressed as: $$X_{seed}^{i+1}=N\left(x^i,\sigma \right)$$ (12) where σ is the coefficient of dispersion, which can determine the dispersion degree of the produced seeds.

Step 7: Reach termination if one of the termination conditions is met. Otherwise, repeat from Step 2. The termination condition that we used was a time boundary of 8 h. $S_{max}=S_1*S_2\dots S_n : E\left(n\right)<480 \min$ E(n) is the event after n iterations.

Step 8: Produce output as Smax, when the termination condition is met.

We can also perform a pattern search in the given algorithm if we have enough computing resources.

5. Dataset Pre-Processing

For this study, we used the Google Landmarks dataset, available at https://www.kaggle.com/c/landmark-recognition-2021 (accessed on 22 February 2022) for performing the network architecture search (NAS). This is a dataset comprising around 1.59 M images, including the ones shown in Figure 2. However, training on such a huge dataset would have taken an extremely long time. Therefore, for the evolution of our network, we used only 600 images out of the dataset. It was also noted that one landmark class had a much smaller number of images; hence, the data was augmented 15 times to about 9000 images. The data was augmented in terms of translation, rotation, scaling. cropping. flipping, etc. Augmentation paved the way for an improved generalization of the neural network.

Each picture was checked to make sure that it was in an RGB color encoding scheme, to ensure proper dimensions in terms of colors [34]. After this process, each picture was checked to make sure that it was the same size (64 × 64). An image of inappropriate size was resized to the given dimensions.

Then, the entire dataset was processed so that the input variable had the tensor of the shape:

x = (9000,64,64,3)

(13)

The output tensor had the shape:

y = (9000,24)

(14)

This meant that the input tensor had 110,592,000 parameters in total. The output had to be one out of 24 classes, as depicted by the output tensor. The dataset was converted into the input tensor variable, while the output tensor was assigned one out of 0–23 numbers for each of the 9000 images. After this, the x tensor and y tensor were used for training. The TensorFlow library provided by Google was imported for creating tensors. However, the overall algorithm was self-designed.

6. Experimentation and Results

We ran the experiment on an Asus VivoBook S15 laptop with 8 GB RAM, a 4 GB NVIDIA graphics card, and an Intel CORE i7 processor. A self-designed code was run in a loop indefinitely for 8 h. In the code, a 3-layer CNN was designed that was then evolved with the paddy field algorithm, and the results were observed.

As shown in Figure 3, the designed CNN consisted of three convolutional layers that were then followed by three max-pooling layers, all of which were then followed by a fully connected layer, with 100 neurons that were varied afterward and 200 neurons that were kept constant.

In Equations (4)–(7), T and F stand for True and False, respectively, while Cn stands for nth class. Accuracy is one of the performance parameters which decide the goodness of a CNN. The complete list of hyperparameters that were evolved can be described as:

• Kernel Frame Size: the 3 × 3 kernel frames are considered highly optimal for CNNs. However, varying the kernel frame size, we saw that a size of 7 × 7 was the kernel with the best fit of seed. This performed the best with other arrangements of hyperparameters [35]. The kernel frame sizes chosen were between 1 × 1 and 11 × 11. These were in the form of square matrices.
• Number of Kernels: the number of kernels was varied to establish the best number that could be checked to fall between 22 and 42. It has been suggested that 32 or 64 kernels seem to work well but, for us, 42 was the variant giving the best seed [36]. Other numbers could have worked even better, had we increased the search space.
• Learning Rate: the learning rate refers to the speed with which the network trains itself. With a slower learning rate, a network can achieve better accuracy, but this increases its chances of running into a local minimum. The network also takes more time to run. A fast learning rate will quicken the rate of learning but run into the problem of deviation from the global minima. A learning rate of 0.01 is considered optimal in the usual cases; indeed, 0.099 was the best parameter found [37]. The learning rate varied between 0.001 and 0.99.
• Batch Size: the batch size refers to the number of images given to the network for training in one go. A batch size of 32 is considered good; in this case, it was found that 32 is optimal and seemed to perform well [38]. A little variation was found to be good in the batch size, which included 33 and 34 even though it varied between 22 and 42.
• Neurons: Neurons were also varied to check for the best types of connections. The initial 100 neurons were altered to fall between 90 and 110; the results showed that many variations within this range seemed to do well, although the eventual best fit was 102 [39].

When the CNN was run, the backpropagation ran in the following form:

$$W_x^{new}=W_x^{old}-a\frac{d\left(Error\right)}{d{W}_x}$$

(15)

where $W_x^{new}$ are the newly trained weights, $W_x^{old}$ are the old weights, a is the learning rate, and $a\frac{d\left(Error\right)}{d{W}_x}$ is the change in an error with respect to the weights. It was run with an AdaDelta optimizer, using the Keras in-built libraries for the optimal changes of weights [40,41,42].

In the code, the CNN hyperparameters were replaced with variables; these variables were initially assigned default CNN hyperparameter values. The accuracy was measured first, then these variables were varied by means of the paddy field algorithm in the search space formed, and the accuracies were consistently recorded. The best seed found, based on accuracy, was [7, 42, 0.0099, 32, 102]. This was in the format of [kernel frame length, number of kernels, learning rate, batch size, neurons] [43,44,45,46].

Figure 4 describes how the maximum accuracy of the evolved networks developed over the elapsed time. Figure 5 shows the accuracy of checked seeds as time progresses. This gives an idea of how the accuracy changes according to the number of seeds that were checked. Figure 6 shows how the accuracy changes from start to finish, as time progresses, when a default CNN is compared to an evolved CNN.

Table 1. Accuracies in a default CNN and the network evolved with PFA are compared over best fits, shown as the steps of time and epochs elapsed.

			Accuracy
S. No.	Epochs	Time (in mins.)	Default CNN (Approx. in %)	PFANET (Best Fit) (Approx. in %)
1	13	5	8	12
2	25	10	16	24
3	38	15	24	35
4	50	20	31	45
5	63	25	37	55
6	75	30	43	63
7	88	35	48	70
8	100	40	53	76

gives an insight into how well the self-designed PFANET fared, when compared to a regular CNN with default hyperparameters. Each network has been compared on the basis of the approximate accuracy gained by each network over time. As is apparent, the best-fit PFANET variant gains more accuracy in shorter time frames, compared to a regular CNN. The key observations that should be made are:

• The best kernel frame length is 7, while the length that was normally used was a 3. 5 kernel frame length, which also performed better than 3 in some combinations. One striking observation was that an unusual kernel frame length of 4, which is not expected to be good because of its being an even kernel frame, also performed better in many regards.
• The number of kernels after optimization was 42, within the range of 22–42, meaning that the maximum number of kernels improved the model’s accuracy. However, more research would be required to validate how many kernels are optimum.
• The learning rate that is usually used is 0.01; there was a positive observation that the best learning rate came out to be 0.0099, which is almost 0.01.
• The number of optimum epochs is 100, but that figure was chosen manually since the network did not then require much processing power at once.
• The best batch size that was seen was the regularly used size of 32.
• The neurons did not seem to vary a great deal; the best number was 102 when initialized with 100.
• The code was run for 8 h and as many as 18 seeds were checked, over a wide variety of combinations.
• In 18 seeds only, the accuracy improved considerably, from 53%, with the default CNN, to 76%.
• The experiment showed that the paddy field search algorithm is a very viable evolutionary metaheuristic for searching best-fit hyperparameters.

Figure 7 shows the confusion matrix developed for the trained CNN using the evolved hyperparameters.

Table 2. The best-fitted seed after the evolution of the CNN architecture with PFA.

Kernel Frame Length	Number of Kernels	Learning Rate	Batch Size	Neurons
7	42	0.0099	32	102

shows the best-evolved hyperparameters. The study demonstrated that there can be a high level of change in the performance of a CNN when evolved with an evolutionary metaheuristic, such as the paddy field algorithm.

7. Conclusions

We have demonstrated how to evolve a convolutional neural network using the paddy field algorithm (PFA). We concluded that there is a wide hyperparametric space in which to search for the best combination in a CNN; evolutionary metaheuristics help us to find the best possible combinations in this wide search space. It was found that high performance was recorded using the hyperparameters found by the paddy field algorithm; the accuracy improved a great deal once the CNN was trained. We also concluded that the hyperparameters we employed, which are very different from the hyperparameters that are regularly used, performed much better. We drew a comparison between a default CNN and our evolved CNN; the evolved CNN or PFANET seemed to perform much better. However, more research is required to establish if a particular hyperparameter can work with all combinations. Moreover, the evolution of a network can take too much time for some applications; therefore, we need to improve these algorithms to evolve the ANNs quickly and with stable hyperparameters. Data augmentation helps in the overall improvement of a network.