LMFRNet: A Lightweight Convolutional Neural Network Model for Image Analysis

Abstract

Convolutional neural networks (CNNs) have transformed the landscape of image analysis and are widely applied across various fields. With their widespread adoption in fields like medical diagnosis and autonomous driving, CNNs have demonstrated powerful capabilities. Despite their success, existing models face challenges in deploying and operating in resource-constrained environments, limiting their practicality in real-world scenarios. We introduce LMFRNet, a lightweight CNN model. Its innovation resides in a multi-feature block design, effectively reducing both model complexity and computational load. Achieving an exceptional accuracy of 94.6% on the CIFAR-10 dataset, this model showcases remarkable performance while demonstrating parsimonious resource utilization. We further validate the performance of the model on the CIFAR-100, MNIST, and Fashion-MNIST datasets, demonstrating its robustness and generalizability across diverse datasets. Furthermore, we conducted extensive experiments to investigate the influence of critical hyperparameters. These experiments provided valuable insights for effective model training.

1. Introduction

Convolutional neural networks (CNNs) are a type of deep learning neural network. They have revolutionized various fields, including image classification [1], object detection [2], and image segmentation [3]. CNNs leverage convolutional layers to extract features from input images, and fully connected layers to classify images. This architectural design enables CNNs to capture both local and global information in images, facilitating the learning of relationships between image content and semantics. In various domains, CNNs have been widely applied, demonstrating their versatility and effectiveness [4]. These applications encompass diverse areas such as lesion detection [5,6] and diagnostic support within the medical field, autonomous driving [7], and face recognition [8], among others.

Since the introduction of AlexNet [9], CNNs have undergone rapid development. AlexNet demonstrated the potential of large-scale deep convolutional neural networks for image recognition tasks. It employed an eight-layer convolutional network structure and achieved breakthrough performance on the ImageNet dataset. Subsequently, VGGNet [10] simplified network design by stacking small convolutional filters to achieve a balance between depth and efficiency. The Inception [11] module enabled CNNs to learn more complex feature representations by combining filters of different sizes. The ResNet [12] architecture effectively addressed the vanishing gradient problem, enabling the construction of models with over a hundred layers. DenseNet [13] promoted efficient feature reuse by connecting every layer to every other layer.

Deep learning models have achieved significant success in the field of image analysis, demonstrating outstanding accuracy in tackling challenging image recognition tasks. However, improving accuracy often implies increased demands on computational resources.

Real-world tasks often demand optimal accuracy within constrained resources, influenced by factors such as the specific platform requirements and diverse application scenarios [14,15,16,17]. Merely pursuing accuracy might constrain their practical applicability on resource-limited mobile devices. Therefore, designing a model that balances parameter quantity and accuracy is a challenging task.

As a result, some lightweight models have emerged, such as SqueezeNet, ShuffleNet-V2, MobileNetV2, and EfficientNet. SqueezeNet [18] decreases the model size by compressing convolutional filters and employing a small number of 1x1 convolutional kernels. ShuffleNet-V2 [19] reduces computational costs through group convolution and channel shuffling techniques, maintaining strong performance in resource-constrained environments. MobileNetV2 [20] reduces the number of parameters by using Depthwise Separable Convolution, making it suitable for deployment on mobile and embedded devices. EfficientNet [21] adopts compound scaling, balancing model depth, width, and resolution to enhance performance while retaining a relatively compact model size.

In this study, inspired by these classic models, we introduce LMFRNet, a lightweight network designed specifically for image classification tasks. The experimental results demonstrate that the proposed model achieves an exceptional accuracy of 94.6% on the CIFAR-10 dataset, despite having only 0.52 million parameters. It was compared to state-of-the-art (SOTA) models on the CIFAR-10 dataset, and the model achieved superior accuracy under equivalent parameter conditions. Additionally, we substantiate the model’s effectiveness through validation on the CIFAR-100, MNIST, and Fashion-MNIST datasets. Furthermore, the model does not use residual blocks, which also provides an idea for exploring scenarios without such blocks. This underscores the efficiency and feasibility of the model. In addition, we conducted extensive experiments to investigate the influence of critical hyperparameters, providing valuable insights for effective model training.

This study contributes to the following:

• First, we propose a lightweight model that attains an outstanding accuracy of 94.6% on the CIFAR-10 dataset, leveraging a mere 0.52 million parameters. Notably, in equivalent parameter setups, the model surpasses existing SOTA models, showcasing a remarkable balance between performance and parameter efficiency.
• Second, we conducted an extensive array of experiments to carefully adjust various common hyperparameters, including training epochs, optimizer selection, data augmentation strategies, and learning rates. By comparing the performance of different hyperparameter settings, we provide valuable experimental results to help practitioners better understand and apply these critical parameters, facilitating more effective model training.

2. Method

In the section, we deconstruct the fundamental CNN components that constitute the model’s underlying architecture. Building upon these core components, we now delve deeper into the intricate architecture of our proposed model, dissecting its components in detail. Figure 1 visually represents the model, showcasing its six stages, each playing a crucial role in progressively extracting pertinent features from the input image, ultimately yielding the desired classification output.

2.1. Basic CNN Components

Convolutional layers are the most important elements of CNNs [9,10]. They are responsible for extracting features from the input data. Convolutional layers work by applying a convolution operation to the input data. The convolution operation [22] is defined as follows:

$$y(i,j)=\sum _m\sum _{n}x(i+m,j+n)w(m,n).$$

In this equation, $y(i,j)$ represents the element in the output feature map at position $(i,j)$, $x(i+m,j+n)$ represents the element in the input data at position $(i+m,j+n)$, and $w(m,n)$ signifies the weight of the convolutional kernel at position $(m,n)$. The indices m and n iteratively cover all possible positions within the convolutional kernel, performing the crucial convolution operation across the input data. This fundamental operation plays a pivotal role in extracting meaningful features from the input, which is instrumental for CNNs in various tasks.

Pooling layers are used to reduce the spatial dimensions of the feature maps [9,12]. The purpose of this is to reduce the computational cost of the network and to make the network more robust to noise. Max pooling operates by selecting the maximum value within each region of the feature map, while average pooling computes the average value within these regions. Additionally, global average pooling (GAP) represents a unique pooling operation aimed at condensing the spatial dimensions of a feature map into a single vector [11,12]. This is achieved by computing the average value across each channel in the feature map.

Batch normalization [23] is a technique that is used to improve the stability and generalization ability of CNNs. It is defined as follows:

$$\begin{array}{c}\hfill \mathrm{BN}\left(x\right)=\gamma \frac{x-\mu }{\sqrt{{\sigma }^2+ϵ}}+\beta .\end{array}$$

Here, x represents input data, $\mu$ is the mean of the input data, ${\sigma }^2$ is the variance of the input data, $\gamma$ and $\beta$ are learnable scaling and shifting parameters, and $ϵ$ is a small constant introduced to prevent division by zero. Batch normalization works by normalizing the activations in the hidden layers. This helps to prevent the activations from becoming too large or too small, which can lead to instability and overfitting.

The ReLU activation function, commonly employed in CNNs [12,24,25], is defined as

$$\begin{array}{c}\hfill y\left(x\right)=max(0,x),\end{array}$$

where x is the input to the activation function, and y is the output. The ReLU function introduces non-linearity into the network, which is essential for capturing complex patterns in the data.

Filter concatenation is a technique that is used to combine the outputs of multiple convolutional layers into a single feature map [13]. This technique is often used in CNNs to increase the model’s ability to capture diverse feature representations, as each convolutional kernel tends to specialize in different input features and patterns.

Fully connected (FC) layers are used to perform the final classification or regression task [9,12,13]. Fully connected layers work by connecting all of the neurons in one layer to all of the neurons in the next layer. This allows the network to learn complex relationships between the input data and the output labels.

2.2. The Proposed Model

The model comprises six stages designed to progressively extract features from the input color image and ultimately produce the corresponding categorical output. The tensor representation of this color image is $(3,32,32)$, where 3 represents the number of color channels (red, green, and blue), and 32 and 32 denote the spatial dimensions of the image, representing the height and width, respectively. This tensor goes through six successive stages, ultimately producing the classification result. Figure 1 provides a detailed illustration of the proposed model. The six stages can be composed of different types of layers.

Stage 1 (see the top part of Figure 1b) employs a two-dimensional convolutional operation aimed at extracting spatial features from the input image. The convolutional kernels are of size $3\times 3$, with a stride of 1 and a padding of 1, similar to EfficientNet [21]. The computational cost of a $3\times 3$ convolutional kernel is relatively low, leading to an overall reduced computational burden. Based on this reference [23] and the experience of some classical models [9,10], after the convolution operation, batch normalization and ReLU activation operations are performed subsequently. The batch normalization layer is used to normalize input tensors, speeding up model convergence, enhancing training stability, and preventing gradient vanishing and exploding issues. The ReLU activation function converts the input tensor into non-linear values, boosting the model’s learning capability. The output of the initial stage is directly propagated to the subsequent stage.

Stages 2 through 5 constitute the pivotal phases responsible for fundamental feature extraction of the model. The structure of the stages has a similar structure (see Figure 1c). The difference lies in the number of multi-feature (MF) blocks contained within these stages. The number of MF blocks in Stages 2 through 5 are set to 3, 4, 8, and 3, respectively. Then, the obtained feature maps are successively passed through a $1\times 1$ convolution and an average pooling layer, similar to DenseNet [13]. The combined use of a $1\times 1$ convolution followed by an average pooling layer can help in further reducing the dimensionality of the feature space while preserving important features, aiding in computational efficiency and potentially improving model generalization.

MF blocks’ design is inspired by these models [11,13,26,27,28,29]. Inception series [11,26,27] inspired us to concatenate different convolutions from multiple branches to derive a new feature map. The idea of feature reuse in DenseNet [13], also influenced the design process of MF block. In DenseNet, feature maps from all preceding layers are utilized and fed directly into subsequent layers. It alleviates the difficulty of model training, enhances the utilization of highly informative features, and consequently improves performance. The structural design of FR-ResNet [29] and PeleeNet [28] also inspired the construction of the MF block.

The MF block employs a sequence of convolutional operations, including a $1\times 1$ convolution followed by three consecutive $3\times 3$ convolutions. Subsequently, the output of each $3\times 3$ convolution is concatenated with the input of this block, resulting in a novel feature map. Using $1\times 1$ convolutions can reduce the number of channels, achieving dimensionality reduction and decreasing the model’s parameters and computational load. Meanwhile, employing three consecutive $3\times 3$ convolutions can extract features at different scales, enhancing the model’s robustness and generalization capability. Connecting these three convolutions helps improve parameter utilization and reduces the number of model parameters. Moreover, the MF block does not utilize any residual structure, exploring scenarios independent of reliance on residual structures. Specifically, the structure of the MF block is illustrated in Figure 1d. It can be expressed as follows:

$$\begin{array}{cc}\hfill Y_1& =Con{v}_{3\times 3}\left(Con{v}_{1\times 1}\left(X\right)\right)\hfill \\ \hfill Y_2& =Con{v}_{3\times 3}\left(Y_1\right)\hfill \\ \hfill Y_3& =Con{v}_{3\times 3}\left(Y_2\right)\hfill \end{array}$$

Then, these three features combined with the input are directly concatenated, resulting in a novel feature:

$$\begin{array}{cc}\hfill Y& =Concat(X,Y_1,Y_2,Y_3).\hfill \end{array}$$

The final sixth stage of the network consists of GAP and a FC layer to generate the final classification result (see the bottom part of Figure 1b). This design, drawing inspiration from ResNet [12], leverages GAP to aggregate spatial information across the feature maps and condense them into a fixed-dimensional vector. Subsequently, the FC layer projects this vector onto the desired number of output classes, enabling the network to make its final classification decision. The detailed architecture of the proposed model is presented in

Table 1. The detailed architecture of the proposed model.

Stages	Layers	Patch Size	Stride	Output Size
Stage 1	Convolution layer	3 × 3	1	32 × 32 × 32
Stage 2	MF Block $\times 3$	-	-	104 × 32 × 32
	Convolution layer	1 × 1	1	104 × 32 × 32
	Average pooling layer	2 × 2	2	104 × 16 × 16
Stage 3	MF Block $\times 4$	-	-	200 × 16 × 16
	Convolution layer	1 × 1	1	200 × 16 × 16
	Average pooling layer	2 × 2	2	200 × 8 × 8
Stage 4	MF Block $\times 8$	-	-	392 × 8 × 8
	Convolution layer	1 × 1	1	392 × 8 × 8
	Average pooling layer	2 × 2	2	392 × 4 × 4
Stage 5	MF Block $\times 3$	-	-	464 × 4 × 4
	Convolution layer	1 × 1	1	464 × 4 × 4
Stage 6	Global average pooling	-	-	464
	Fully connected layer	-	-	10

.

3. Dataset

This section delineates the dataset employed for model evaluation, encompassing its provenance, scale, and dataset partitioning.

The CIFAR-10 dataset [30] holds significant importance within computer vision, encompassing a repository of 60,000 images. It serves as a rigorous evaluation platform for computer vision algorithms, with a primary focus on tasks. Figure 2 presents visual examples drawn from each of the ten object classes in the CIFAR-10 dataset. Additionally,

Table 2. Distribution of categories in CIFAR-10 dataset.

Category	Training Set	Test Set	Total
Airplanes	5000	1000	6000
Automobiles	5000	1000	6000
Birds	5000	1000	6000
Cats	5000	1000	6000
Deer	5000	1000	6000
Dogs	5000	1000	6000
Frogs	5000	1000	6000
Horses	5000	1000	6000
Ships	5000	1000	6000
Trucks	5000	1000	6000

provides a detailed breakdown of the dataset’s category distribution, offering a comprehensive overview of the dataset’s composition. Apart from evaluating the model’s performance on the CIFAR-10 dataset, we conducted evaluations across the CIFAR-100 [30], MNIST [31], and Fashion-MNIST [32] datasets. This broader assessment enabled us to gauge the model’s generalizability across diverse image domains.

4. Experiments

This section encompasses an in-depth exploration of the experimental setup, the environment details, hyperparameter influences, comparative analysis of proposed and plain blocks, and a comprehensive performance evaluation against SOTA models.

4.1. Experimental Environment

The experiments were conducted on a Linux operating system (version 5.4.0-132-generic) with an x86_64 CPU architecture. The setup comprises an Intel Xeon Platinum 8255C with 12 virtual CPUs and an NVIDIA GeForce RTX 3080 GPU. The CPU, manufactured by Intel, boasts significant computational capacity. The Python programming language (version 3.8.10) and the PyTorch deep learning framework (version 2.0.0+cu118) were used in this study, along with the TorchVision library (version 0.15.1+cu118).

4.2. Experimental Results

We conducted extensive experiments to investigate the influence of critical hyperparameters, including training epochs, learning rate, choice of optimizer, and data augmentation strategies. Through this rigorous experimentation, we aim to provide valuable empirical insights that can empower practitioners to make informed decisions regarding these crucial parameters, fostering more effective model training practices.

4.2.1. The Influence of Epochs

The model’s training outcomes are influenced by various hyperparameters, among which determining the appropriate number of “epochs” or training iterations is pivotal. Each epoch signifies one full pass of the model through the entire training dataset. During training, the model continuously adjusts its weights and biases to minimize the loss function. Therefore, the number of epochs directly affects how well the model learns from the data.

Based on the trends presented in Figure 3, the following observations can be made:

• In the first approximately 25 rounds of training, the loss rapidly decreases, and the accuracy significantly improves.
• From the 25th round to roughly the 175th round, the loss continues to decrease, and the accuracy steadily increases.
• From the 175th round to approximately the 200th round, both the loss and accuracy gradually converge.

Setting the epoch size too small can lead to insufficient training, as the model may not have enough time to learn all of the patterns in the training data. This can result in poor performance on the test data. Setting the epoch size too large can lead to overfitting, as the model may learn the training data too well, including noise and irrelevant features. This can also result in poor performance on the test data.

Therefore, it is important to carefully consider the factors that affect epoch size selection, such as the size of the training dataset, the complexity of the model, and the risk of overfitting. In general, for smaller datasets, a smaller epoch size is recommended to avoid overfitting. For larger datasets, a larger epoch size may be necessary to fully learn the training data.

4.2.2. The Influence of Learning Rate

The learning rate directly influences the speed at which model parameters are updated, thereby having a profound impact on the model’s performance and convergence. To accurately assess the role of the learning rate, we employed the method of controlling variables. We ensured that all other hyperparameters, such as the training epoch, data transformation, optimizer, and batch size, were kept constant throughout our experiments. This allowed us to isolate the impact of the learning rate on model performance.

We conducted experiments using various learning rate values, specifically including 0.1, 0.01, 0.001, and 0.0001. By comparing how model performance varied with epoch under these different learning rates, we gained a more intuitive understanding of the pivotal role played by the learning rate during training.

As illustrated in Figure 4, the learning rate has a significant impact on the model’s training process. Several observations can be made:

• When the learning rate is set to a higher value (e.g., 0.1 in Figure 4), the model’s learning speed noticeably accelerates, leading to a rapid increase in accuracy. However, if the learning rate is set excessively high, it may lead to training oscillations and non-convergence and cause the model to miss the global optimum.
• Conversely, when the learning rate is set to a lower value(e.g., 0.0001 in Figure 4), the model’s learning speed slows down, resulting in a more stable training process. However, this can significantly slow down the convergence speed. Additionally, a smaller learning rate may cause the model to become stuck in a local optimum.

Therefore, the choice of learning rate necessitates a delicate balance between achieving rapid convergence and maintaining stability, allowing for comprehensive model training while avoiding the pitfalls of local optima.

4.2.3. The Influence of Optimizer

The choice of optimizer is a critical factor that influences the performance of model training. In this study, we evaluated the performance of two popular optimizers: stochastic gradient descent (SGD) and Adam. We compared the relationship between the number of training epochs and accuracy under different learning rates for SGD [33] and Adam [34].

As shown in Figure 5, the training curve of Adam is smoother than that of SGD, mainly due to the momentum mechanism it uses. Momentum can help the model overcome oscillations during training, making the training curve more stable. In addition, Adam is relatively simple to use and less sensitive to hyperparameters, making it easier to achieve good results. However, for large or noisy training datasets, SGD is more likely to converge. Overall, Adam is more suitable for most tasks and datasets.

4.2.4. The Influence of Data Augmentation

Data augmentation strategies and transformations are pivotal during model training [35,36]. We examined the impact of various data augmentation methods, including random cropping, rotation, flipping, and more, on model performance. Our objective was to determine which data augmentation strategies contribute to enhanced model generalization and robustness.

This study investigated the following six data augmentation strategies:

• Baseline: Original data without any augmentation.
• Norm: Pixel values in training images are normalized to a standard distribution.
• HFlip: Randomly flip images horizontally.
• Crop: Randomly crop and pad images.
• Cutout: Randomly mask out a rectangular region in input images.
• All: Combines all of the above strategies.

The experimental results indicate that data augmentation significantly enhances the model’s generalization ability and robustness.

Data normalization contributes to the stability of training, as evidenced by the blue curve (Norm) in Figure 6a. This curve is slightly smoother and exhibits less fluctuation than the black curve (Baseline), which corresponds to using the original data without any processing.

Additionally, Figure 6a clearly shows that the data augmentation techniques (HFlip, Crop, and Cutout) have a positive impact on the model’s accuracy. Specifically, the accuracy reaches 0.84 after HFlip is applied, 0.85 after Crop is applied, and 0.83 after Cutout is applied. The most significant improvement in performance is observed when these data augmentation techniques are combined, resulting in an accuracy of 0.90. Analogously, Figure 6b demonstrates that data augmentation is also effective when using the Adam optimizer.

Data augmentation is a crucial technique for improving the performance of deep learning models. In practical applications, appropriate data augmentation strategies should be selected based on the characteristics of the data and the objectives of the task. By judiciously augmenting the data, we can effectively enhance the model’s generalization ability and robustness, thereby improving its performance in real-world applications.

4.3. Comparison between the MF Block and the Plain Block

In MF block (see Figure 7a), we set the 1 × 1 convolutional channel to 12, the first 3 × 3 convolutional channel is also 12, the second 3 × 3 convolutional channel is 6, and the third 3 × 3 convolutional channel is 6. In the MF block, these three convolutions are concatenated with the input, so with each passage through an MF block, the number of feature channels increases by 24 (12 + 6 + 6).

To elucidate the differences between the MF block and the plain block, we replaced the MF block in the model with the plain block (see Figure 7). Subsequently, we conducted a comparison between the two models and found that the model using the plain block was significantly more difficult to train (see Figure 8). The excessive number of convolutional layers primarily causes this difficulty, leading to issues such as gradient vanishing and exploding. These challenges exacerbate the problems during model training. In contrast, the MF block facilitated a smoother training process for the model and achieved superior results.

Furthermore, we observed that the model employing the plain block had a parameter count of 0.78 M, despite having the same number of layers as our original model, which had a parameter count of 0.52 M. This indicates that the model, which utilizes the MF block, is more parameter-efficient, resulting in a higher parameter utilization rate.

4.4. Comparative Analysis with Other Models

We extensively validated the model across four distinct datasets: CIFAR-10, CIFAR-100, MNIST, and Fashion-MNIST. By validating the model on these four datasets, we were able to comprehensively assess its performance. The experimental results indicate that the model performed well across these datasets, showcasing its strong image classification capability, generalization, and robustness.

4.4.1. Performance on the CIFAR-10 Dataset

We conducted a comparative analysis between our proposed model and some classical models. As presented in

Table 3. Model comparison of the CIFAR-10 dataset.

No.	Model	Accuracy (Top-1)	Parameters
1	Ours	94.60%	0.52 M
2	SqueezeNet [18]	92.83%	0.73 M
3	ShuffleNet [37]	92.31%	0.93 M
4	ShuffleNetV2 [19]	92.86%	1.26 M
5	MobileNetV2 [20]	94.19%	2.30 M
6	MobileNet [38]	92.76%	3.22 M
7	GoogLeNet [11]	95.02%	6.17 M
8	DenseNet121 [13]	95.04%	6.96 M
9	ResNet50 [12]	93.62%	23.52 M
10	VGG13 [10]	94.06%	28.33 M

, we observed a range of accuracy performances among the models, which varied from 92.31% to 95.04%. This variability is further explored in Figure 9. Notably, GoogLeNet and DenseNet121 demonstrated slightly higher levels of accuracy compared to the other models.

In contrast, our model surpassed these accuracy ranges by achieving an impressive accuracy of 94.60% while utilizing a significantly smaller parameter count of only 0.52 M. Remarkably, when compared to DenseNet121, our model required less than one-thirteenth of its parameters while experiencing only a marginal decrease in accuracy of 0.44%. This remarkable achievement is a result of our approach, which places equal emphasis on both high performance and parameter frugality. By prioritizing efficiency and resource optimization, our model proves to be an excellent choice for tasks with limited computational resources. The results unequivocally demonstrate the model’s effectiveness in striking a favorable balance between accuracy and efficiency.

Note that our model’s superior performance does not rely on additional datasets. This further reinforces the significance of our approach, as it showcases the model’s ability to excel without the need for extra data, making it even more valuable in practical applications.

4.4.2. Performance on the CIFAR-100 Dataset

To further evaluate the model’s performance, we conducted experiments on the CIFAR-100 dataset. The model achieved an accuracy of 75.82%, as depicted in

Table 4. Model comparison on CIFAR-100 dataset.

No.	Model	Accuracy (Top-1)	Parameters
1	Ours	75.82%	0.56 M
2	SqueezeNet [18]	69.41%	0.78 M
3	ShuffleNet [37]	70.06%	1.00 M
4	ShuffleNetV2 [19]	69.51%	1.30 M
5	MobileNetV2 [20]	68.08%	2.36 M
6	MobileNet [38]	65.98%	3.30 M
7	GoogLeNet [11]	78.03%	6.20 M
8	DenseNet121 [13]	77.01%	7.00 M
9	ResNet50 [12]	77.39%	23.70 M
10	VGG13 [10]	72.00%	28.70 M

. To accommodate the expanded range of 100 categories in CIFAR-100, the final fully connected layer of the model was correspondingly adjusted to 100 output neurons. This modification resulted in a total parameter count of 0.56 M for the model. CIFAR-100 comprises more categories, necessitating models to possess stronger discriminative and generalization capabilities. The experimental results indicate that the model maintains outstanding performance despite having significantly fewer parameters. Through experiments on these datasets, we further validated the effectiveness and broad applicability of the proposed model.

4.4.3. Performance on the MNIST Dataset

The MNIST dataset is commonly used to evaluate models in relatively simple image classification tasks. Despite its simplicity, MNIST provides a benchmark for measuring a model’s ability to solve basic image classification problems. The model achieved an accuracy of 99.71% on MNIST with just 30 training rounds, demonstrating its capability in handling relatively straightforward image classification tasks. Note that most models can easily achieve over 99% accuracy on the MNIST dataset.

4.4.4. Performance on the Fashion-MNIST Dataset

Fashion-MNIST, characterized by more intricate image content compared to MNIST, closely simulates real-world applications. the model achieved an accuracy of 94.11% on the Fashion-MNIST dataset, and a comparison with SOTA models is presented in

Table 5. Model comparison on Fashion-MNIST dataset.

No.	Model	Accuracy (Top-1)	Parameters
1	ViT [39]	90.98%	0.212 M
2	Ours	94.11%	0.520 M
3	MCNN15 [40]	94.04%	2.596 M
4	MobileNet [38]	93.96%	2.236 M
5	EfficientNet [21]	93.64%	4.019 M
6	ResNet18 [12]	93.20%	11.175 M
7	AlexNet [9]	92.74%	58.302 M

. These experiments enabled a deeper insight into the model’s proficiency in processing diverse and complex real-world images.

5. Conclusions

Lightweight models are significant for tasks such as image analysis, real-time image analysis, and defect detection on resource-constrained devices. In this paper, we propose a lightweight model with only 0.52 million parameters that achieves a 94.6% accuracy on the CIFAR-10 dataset. Our lightweight model was compared to SOTA models on the CIFAR-10 dataset. The model achieved superior accuracy under equivalent parameter conditions. Additionally, we conducted experiments on CIFAR-100, MNIST, and Fashion-MNIST datasets to validate the performance of the model. The MF block in LMFRNet effectively realizes feature reuse, thereby reducing the computational complexity of the model and improving its accuracy. Moreover, the model does not use residual blocks, which also provides an idea for exploring scenarios without residual blocks. Furthermore, to assist practitioners in understanding the hyperparameters used in our experiments, we conducted a comprehensive series of experiments to elucidate the roles of these hyperparameters and how to correctly configure them in practical applications.

In summary, the model maintains performance while having a very small parameter count, making it particularly amenable to resource-constrained domains or scenarios requiring real-time analysis. Future research directions based on our approach include:

• Extending our versatile classification model to a wider range of resource-constrained scenarios, such as Internet of Things (IoT) applications and embedded systems with limited computational resources.
• Investigating the use of the model as a backbone network for real-time object detection and other vision tasks, such as in autonomous vehicles, real-time crowd counting, and augmented reality environments.

To facilitate widespread utilization and advancement of the proposed model, we have made it open source (https://github.com/wgqhandsome/LMFRNet, accessed on 24 December 2023).