Mask-Space Optimized Transformer for Semantic Segmentation of Lithium Battery Surface Defect Images

Abstract

The segmentation of surface defects in lithium batteries is crucial for enhancing the overall quality of the production process. However, the severe foreground–background imbalance in surface images of lithium batteries, along with the irregular shapes and random distribution of foreground regions, poses significant challenges for defect segmentation. Based on these observations, this paper focuses on the separation of foreground and background in surface defect images of lithium batteries and proposes a novel Mask Space Optimization Transformer (MSOFormer) for semantic segmentation of these images. Specifically, the Mask Boundary Loss (MBL) module in our model provides more efficient supervision during training to enhance the accuracy of the mask computation within the mask attention mechanism, thereby improving the model’s performance in separating foreground and background. Additionally, the Dynamic Spatial Query (DSQ) module allocates spatial information of the image to each query, enhancing the model’s sensitivity to the positions of small foreground targets in various scenes. The Efficient Pixel Decoder (EPD) ensures deformable receptive fields for irregularly shaped foregrounds while further improving the model’s performance and efficiency. Experimental results demonstrate that our method outperforms other state-of-the-art methods in terms of mean Intersection over Union (mIoU). Specifically, our approach achieves an mIoU of 84.18% on the lithium battery surface defect test set and 85.53% and 87.05% mIoUs on two publicly available defect test sets with similar defect characteristics to lithium batteries.

1. Introduction

Due to the uncontrollability of industrial equipment and the production environment, product surface defects will inevitably appear, which will affect the manufacturing efficiency and economic benefits. Surface defect detection (SDD) has become an indispensable part of modern production lines by identifying defect areas in industrial product images through a pixel level prediction method. Especially in the fields of industrial manufacturing [1], product quality testing [2], urban transportation [3] and military aerospace [4], surface defect detection plays a vital role. With the increasing demand for high-precision automatic detection, image semantic segmentation technology based on deep learning is gradually replacing the traditional manual detection methods, to overcome the shortcomings of manual detection in accuracy and efficiency, and has become the mainstream detection method. At present, most traditional deep learning neural networks are designed for general image prediction tasks, and their performance in defect detection scenarios is still limited. Therefore, it is particularly necessary to optimize the design of deep learning network architecture for defect detection scenarios.

In recent years, Convolutional Neural Networks (CNNs) [5] have been widely applied to various computer vision tasks, including classification, segmentation, and detection, due to their ability to extract more discriminative and robust features within deep networks [6]. For semantic segmentation tasks, Fully Convolutional Networks (FCN) [7] have been proposed to achieve end-to-end dense pixel prediction directly through convolutional and upsampling layers, achieving breakthroughs in natural scene analysis. Due to the simplicity and outstanding performance of the FCN framework, many studies [8,9,10,11,12] have applied it to industrial defect image detection, making pioneering progress. However, the inherent locality of convolutional operations often limits their ability to explicitly model long-range interactions, which is crucial for pixel-level defect detection in complex scenarios, such as cluttered backgrounds and hard-to-identify pseudo-defects [13].

To effectively capture information about remote interactions, various attention mechanisms have been introduced into industrial defect detection algorithms [14,15], facilitating rapid advancements in semantic segmentation of industrial defect images. For example, Xiao et al. [16] combined the local feature bias of CNNs with the global feature bias of Transformers, enhancing the model’s capability for defect feature extraction. At the same time, Zhang et al. [17] proposed a multi-layer attention mechanism to capture spatial dependencies between multi-level features, thereby strengthening the expression of defect features. In terms of global feature representation, Zhang et al. [18] designed a context-aware attention module that integrates high-level and low-level image feature regions that often influence the result(s) within the backbone network to improve global feature representation of defects. On the other hand, Wang et al. [19] proposed a bidirectional stripe-refined attention mechanism that utilizes stripe pooling to capture vertical and horizontal global context dependencies, enhancing the representation of edge features.

Although attention-based methods demonstrate strong remote-dependency modeling capabilities and effectively learn global-spatial representations of industrial surface defect images, they still face a performance bottleneck. This bottleneck primarily arises regions often result features within global spatial information and the foreground-background imbalance in surface defect images. Specifically, the imbalance in area between the foreground and background regions often results in missed detections of defect regions and false detections of pseudo-defects. Numerous studies [20,21] have noted that the foreground proportion in surface defect images is significantly smaller than the background proportion, and relevant research has explored foreground object segmentation accordingly. However, achieving complete foreground–background separation remains challenging due to varying foreground proportions across different defect categories in industrial surface defect images.

Inspired by the mask classification task Mask2Former [22], we observed that the segmentation task can be designed to predict a set of binary masks, with each mask associated with a class prediction. This method naturally separates the foreground and background, clearly indicating their proportions. Meanwhile, the mask cross-attention mechanism selectively focuses on regions with high attention scores, ignoring regions with lower scores. In this paper, we introduce mask classification into the analysis of industrial lithium battery surface defect images and propose a novel Mask Spatial Optimization Transformer (MSOFormer) for the semantic segmentation of these images. MSOFormer aims to explore a new paradigm for the semantic segmentation of surface defect images through foreground–background separation. Specifically, MSOFormer introduces a mask classification pipeline to separate the foreground and background of surface defect images through predicted masks. Additionally, by studying the impact of the computational region accuracy in the mask attention mechanism on model performance, we propose a Mask Boundary Loss (MBL) module to provide supervision information to the computational regions. This not only eliminates redundant background features but also ensures the effective utilization of background information around the foreground regions. Furthermore, the proposed Dynamic Spatial Query (DSQ) module enhances the model’s sensitivity to the spatial positions of small foreground objects, while the Efficient Multi-Scale Pixel Decoder (EPD) improves the model’s ability to extract irregular foreground objects through deformable receptive fields. These components further improve the model’s foreground–background separation performance. The main contributions of this work are summarized as follows.

• We introduce mask classification into the analysis of lithium battery surface defect images and propose a novel Mask Boundary Loss (MBL) module to aid the mask attention mechanism in learning more precise computational regions, thereby further improving the model’s foreground segmentation accuracy.
• Considering the randomness in the generation of small defect locations in lithium battery surface defect images, the Dynamic Spatial Query (DSQ) module integrates position encoding of image features into the query, aiming to enhance the model’s sensitivity to small foreground objects.
• The Efficient Pixel Decoder (EPD) module achieves deformable receptive fields for irregular foreground objects through the fully convolutional FPN (Feature Pyramid Network) architecture, enhancing both model performance and operational efficiency.
• Experimental results demonstrate that the proposed MSOFormer surpasses existing methods, achieving state-of-the-art performance on the lithium battery surface defect dataset, the MT dataset, and the NEU-Seg dataset.

The structure of this paper is arranged as follows: Section 2 introduces the related work; Section 3 describes the proposed MSOFormer model for semantic segmentation of lithium battery surface defect images; Section 4 analyzes the performance of the proposed method through experiments; and Section 5 concludes the paper.

2. Related Work

2.1. General Semantic Segmentation

Semantic segmentation involves classifying each pixel in an image into specific semantic categories, thereby achieving a pixel-level understanding and segmentation of the image content. After the introduction of an FCN [7], many semantic segmentation algorithms adopted its design principles to enable dense pixel-level predictions at arbitrary resolutions, which laid the foundation for subsequent encoder–decoder structures. For instance, UNet [23] proposed a U-shaped architecture based on an FCN, which captures both contextual and positional information, thereby achieving higher performance and robustness. In the meantime, PspNet [24] addressed the limitations of a FCN in scene analysis, such as insufficient contextual information and a small receptive field, by introducing a Pyramid Pooling Module to enhance the detection accuracy of objects with similar colors and shapes. On the other hand, the Deeplab series [25,26,27,28] employed atrous convolution to effectively expand the receptive field without increasing the number of parameters, while the Atrous Spatial Pyramid Pooling (ASPP) structure further strengthened the network’s robustness in multi-scale, multi-class segmentation tasks. In addition, models like PSANet [29], CCNet [30], SegFormer [31], and SegNeXt [32] utilized various attention mechanisms to further exploit global spatial modeling capabilities and capture long-range dependencies, resulting in outstanding semantic segmentation performance.

2.2. Semantic Segmentation of a Surface Defect Image

Deep learning methods have made significant progress in the semantic segmentation of industrial surface defect images. For instance, Yang et al. [33], Liu et al. [34], and Du et al. [35] applied semantic segmentation techniques to surface defect detection with promising results. Considering that surface defect images contain richer semantic information than natural scene images, various studies have attempted to design deep networks specifically suited to surface defect images. In this case, Yu et al. [36] designed an adaptive network depth selection mechanism to distinguish similar defects by aggregating features at different depths. Building on this, Li et al. [37] combined strip pooling and a spatial pyramid pooling module (SPPM) to enhance the model’s ability to perceive defect location and shape information. In the meantime, Zhou et al. [38] developed a hybrid backbone network based on CNN inverse-residual blocks and Swin Transformer blocks, introducing an efficient channel attention-enhanced feature fusion module to merge the features output by these two modules. In addition, Yao et al. [39] developed a vision transformer with dual attention mechanisms to meet the needs of surface defect detection under more complex semantic conditions. On the other hand, Wang et al. [13] combined the advantages of convolutional attention blocks and multi-pool self-attention to achieve better local and global information perception. Zhang et al. [40] used a wavelet transform to guide the down-sampling layers in a Swin Transformer, improving the network’s sensitivity to surface defect textures and its ability to detect high-frequency information. Although current methods have achieved significant advances by enhancing remote spatial representation, their performance has encountered a bottleneck. Based on the characteristics of lithium battery surface defects, this paper focuses on foreground–background separation in surface defect images and proposes a more effective mask space optimization method to overcome the performance limitations in defect image semantic segmentation.

2.3. Mask Classification

Mask classification has been extensively utilized in image segmentation methodologies. For example, Mask R-CNN [41] filters and classifies regions of interest produced by the Region Proposal Network, subsequently generating masks for these classified areas to achieve pixel-level segmentation. More recently, MaskFormer [42] has sought to redefine the segmentation task as predicting a series of binary masks in conjunction with their respective class predictions to compute the final output, where each mask is linked with a class prediction. Building on this concept, Mask2Former [22] introduced a mask attention mechanism that effectively distinguishes between foreground and background regions. Its multi-scale deformable attention mechanism further refines mask features at various scales. However, the computational complexity of the multi-scale deformable attention mechanisms makes Mask2Former unable to meet the efficiency requirements of industrial application scenarios. In contrast, PEM [43] redesigned the mask attention mechanism through a prototype selection strategy, thereby reducing the model’s computational load. Our method employs Mask2Former as a baseline model and introduces a mask space optimization approach for the semantic segmentation of lithium battery surface defects.

3. Methods

MSOFormer follows the standard structure of mask classification models in its design. Initially, four hierarchical features ($F_1$ to $F_4$) are extracted from the network image. The pixel decoder, EPD, based on the FPN architecture, further refines these features and sends three lower-resolution features in a cyclical manner to the consecutive transformer decoder layers, while the higher-resolution feature is utilized as the mask feature ${Mask}_F$. The decoder learns multi-scale features and trainable queries $Query$ to train the query embeddings ${Query}_{embed}$. In the semantic segmentation process, the query embeddings ${Query}_{embed}$ generate the class prediction ${Mask}_{class}$ and the corresponding mask embeddings ${Mask}_{embed}$ through an MLP. The binary mask prediction ${Mask}_{pred}$ is obtained via the Einstein summation convention between the mask features ${Mask}_F$ and the mask embeddings ${Mask}_{embed}$. Finally, by using the Einstein summation convention to combine the mask predictions ${Mask}_{pred}$ with the class predictions ${Mask}_{class}$, the final semantic segmentation results are produced. The specific network structure of MSOFormer is shown in Figure 1.

3.1. Architecture of an Efficient Multi-Scale Pixel Decoder

The pixel decoder plays a fundamental role in extracting multi-scale features for precise object segmentation. Mask2Former implements this by employing the FPN enhanced with multi-scale deformable attention. Compared to the pixel decoder of MaskFormer, it offers several advantages: dynamic weight calculation based on the input, feature refinement through multi-scale information, and enhanced local information representation using deformable receptive fields. Although the performance has been improved, considering the small amount of defect data produced by actual industrial production lines, using deformable attention on the FPN will bring additional computational overhead and computing resources, resulting in an inefficient pixel decoder that is unsuitable for application on actual production lines. To balance performance and computational efficiency, this paper employs a fully convolutional FPN architecture for the pixel decoder. This approach maintains the benefits of a multi-scale deformable attention FPN while using a cost-effective and simple convolutional design. Firstly, to reintroduce dynamic weights and multi-scale information, this study refers to the CoordAtt [44] design and proposes a Self-Adaptive Module (SAM) based on spatial channel weights and context. This module globally rebalances the importance of scene representation at each scale, suppressing channels and spatial information with less data and enhancing those with more data. Additionally, to achieve deformable receptive field capabilities, deformable convolutions are used to dynamically adjust the receptive field to focus on relevant regions of the image, thereby improving the model’s representation of irregular defects. This method maintains computational efficiency while delivering competitive performance.

Specifically, given the input features of the $i$-th stage $F_i\in R^{H_i\times W_i\times C_i}$, with $i\in \left\{1, 2, 3, 4\right\}$, they are first projected into a low-dimensional space $C$ to obtain $F_i^C\in R^{H_i\times W_i\times C}$. Then, global pooling is applied along the horizontal and vertical directions at each scale, resulting in the global pooling visual feature projection $F_i^G\in R^{(H_i+W_i)\times C}$ for subsequent contextual information fusion:

$$F_i^G=\mathrm{M}\mathrm{l}\mathrm{p}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left({\displaystyle \frac{1}{W_i}}\sum _{0<j\le W_i}{F}_i^C\left(H_i,j\right)+\underset{0\le j\le W_i}{\max }\left\{F_i^C\left(H_i,j\right)\right\},{\displaystyle \frac{1}{H_i}}\sum _{0<j\le H_i}{F}_i^C\left(j,W_i\right)+\underset{0\le j\le H_i}{\max }\left\{F_i^C\left(j,W_i\right)\right\}\right)\right),$$

(1)

where MLP refers to a two-layer network composed of $1\times 1$ convolutions. Next, a pair of feature maps with directional perception and channel information, namely $F_i^H\in R^{H_i\times C}$ and $F_i^W\in R^{W_i\times C}$, are separated along the horizontal and vertical directions of $F_i^G$. These feature maps are processed through the sigmoid function $\mathsf{\sigma }$ to obtain the correlation for each spatial direction and channel. Finally, the output of the SAM module for each scale $F_i^{SAM}$ is obtained as follows:

$$F_i^{SAM}=F_i^C+F_i^C·\mathsf{\sigma }\left(\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{j}\left(F_i^H\right)\right)·\mathsf{\sigma }\left(\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{j}\left(F_i^W\right)\right),$$

(2)

where $\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{j}$ represents linear projection. After obtaining the contextual features for each scale, they are aggregated to construct a feature pyramid network. This work follows previous efficient segmentation works [45,46] and utilizes deformable convolutions [47] to integrate features across different scales as follows:

$${EPD}_i=\left\{\begin{array}{cc}\hfill \mathrm{DefConv}\left(F_i^{SAM}\right),& i=4\hfill \\ \hfill \mathrm{DefConv}\left(F_i^{SAM}+\mathrm{U}\mathrm{p}\left({EPD}_{i+1}\right)\right),& i=1, 2, 3\hfill \end{array}\right.,$$

(3)

where $\mathrm{D}\mathrm{e}\mathrm{f}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}$ refers to deformable convolutions, and $\mathrm{U}\mathrm{p}$ represents the upsampling operation. As shown in Figure 1, the three low-resolution features ${EPD}_i\left(i\in \left\{2, 3, 4\right\}\right)$ output by the pixel decoder are used as multi-scale visual features in the transformer decoder to further enhance the quality of the defect area masks. The high-resolution feature ${EPD}_1$ is processed through a depth-wise separable convolution layer to obtain the refined mask features ${Mask}_F\in R^{C\times H\times W}$ as described by the following:

$${Mask}_F=\mathrm{D}\mathrm{w}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\left({EPD}_1\right).$$

(4)

3.2. Architecture of a Dynamic Spatial Query Module

Natural images are typically captured by photographers with a specific purpose, usually placing the subject in the visual center of the image. In contrast, on automated industrial production lines, the positions of industrial cameras relative to industrial components are generally fixed, while the locations of defects on the components are random. In industrial surface defect detection, small-scale surface defects such as dirt and foreign objects, are randomly distributed across the surface area, and lithium batteries are no exception. Metal scrap randomly appears in the Tab area, while nonmetal impurities randomly occur in the Sealant area. Compared to natural images, the distinct positional differences of defect targets on lithium battery surfaces increase the difficulty of extracting foreground objects in semantic segmentation. To address this, we developed a Dynamic Spatial Query (DSQ) module to enhance MSOFormer’s sensitivity to the spatial positions of small-scale defect targets in lithium battery defect detection. This is achieved by incorporating spatial location information of the image into the learnable queries, as illustrated in Figure 2. The DSQ module enables information interaction between the second, third, and fourth layers of the image features output by the pixel decoder and the query block in the transformer decoder. This design facilitates the detection of targets of varying sizes.

The computation of the standard mask attention function with residual connections is given by Equations (5) and (6):

$${Query}_{cross}=W_A·f_V\left({Feature}_i\right)+{Query}_{l-1},\hspace{1em}\hspace{1em}l=0, 1, 2, \dots ,t-1, i=4-l\%3,$$

(5)

$$W_A=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(M_{l-1}+f_Q\left({Query}_{l-1}+{POS}_{Query}\right)·f_K{\left({Feature}_i+{POS}_{{Feature}_i}\right)}^{\mathrm{T}}\right),$$

(6)

where $l$ denotes the number of layers in the transformer decoder; $f_Q$, $f_K$, and $f_V$ are linear transformations of the given query $Query\in R^{N\times C}$ and image features ${Feature}_i\in R^{C\times H_i\times W_i}$ output by the pixel decoder, where $i=2,3,4$; ${POS}_{Query}$ represents the learnable positional embedding with the same dimension as $Query$; ${POS}_{{Feature}_i}$ denotes the cosine positional encoding with the same dimension as ${Feature}_i$; and $M_{l-1}$ is the attention mask obtained from the previous layer of the transformer decoder.

To further incorporate additional image positional features into the learnable query, we first compute the mixed intermediate features that include the image features. This is achieved through the dot product of the query features and key image features, as shown in Equation (6). The mixed intermediate features are represented as $W_A\in R^{N\times (H_i·W_i)}$, where $N$ is the number of queries and $H_i$ and $W_i$ are the height and width of the key image features, respectively. $W_A$ records the patch information of interest for each query. To inject cosine positional encoding into the learnable query, we multiply the positional encoding of the input image features by the attention weights $W_A$, obtaining the positional encoding of the patches of interest for each query, which matches the shape of the queries that record the information of interest. Therefore, the final query can be represented as in Equation (7):

$${Query}_{DSQ}=W_A·{POS}_{{Feature}_i}+{Query}_l.$$

(7)

The final query obtained by the Dynamic Spatial Query (DSQ) module is ${Query}_{DSQ}$. As shown in Equation (7), spatial location information is explicitly embedded in the learning process. This enhances the segmentation quality in defect detection scenarios, as MSOFormer’s learned queries are more sensitive to mask features at different spatial locations. Furthermore, $W_A$ is already part of Equation (5), and ${POS}_{{Feature}_i}$ is an existing variable. Therefore, the number of parameters in the DSQ module does not increase, and the convergence speed of mask attention is not affected.

3.3. Architecture of a Mask Boundary Loss Module

Due to the small pixel proportion of defect regions, there is an imbalance between foreground and background in lithium battery defect images, which leads to a performance bottleneck in semantic segmentation of defect images. In the mask attention mechanism of Mask2Former, the foreground and background are separated by recording the mask regions of interest from the previous layer decoder, enhancing the representation of local target regions. This approach not only effectively avoids unnecessary computations in the background regions and reduces the training cycle of the model, but also gradually refines the foreground regions through multi-layer attention decoders. This allows the model to focus more on the foreground areas, thereby improving accuracy [22]. Therefore, the mask regions of interest passed between decoders determine the computational areas of the mask attention mechanism, and the accuracy of these areas is crucial for improving model performance. Considering this and the binary nature of mask regions, this paper attempts to use edge loss to supervise the computational areas, forcing the model to generate more accurate computational regions and more accurately distinguish between foreground and background areas.

We found that the performance upper limit of Mask2Former is positively correlated with the overlap between the sum of the foreground regions of interest for the $N$ mask queries in the mask attention mechanism and the expanded GT mask area. The proof of this correlation is detailed in Section 5.1.

According to Equation (6), the mask attention calculation area of a certain layer in the transformer decoder is derived from the mask $M_{l-1}$ of the previous layer. The content at the feature position $(x, y)$ in each layer mask $M_l$ is calculated using Equation (8):

$$M_l\left(x, y\right)=\left\{\begin{array}{cc}\hfill 0,& \mathrm{if}{Mask}_{attn}\left(x, y\right)=1\hfill \\ \hfill -\infty ,& \mathrm{otherwise}\hfill \end{array}\right.,$$

(8)

where the content at the feature position $(x, y)$ of the mask calculation area ${Mask}_{attn}\in R^{N\times H\times W}$ is obtained from the predicted mask ${Mask}_{pred}$ after a sigmoid activation function operation and taking a fractional threshold of 0.5. The predicted mask ${Mask}_{pred}$ is obtained by the mask embedding ${Mask}_{embed}\in R^{N\times C}$ output by the transformer decoder and the mask feature ${Mask}_F\in R^{C\times H\times W}$ output by the pixel decoder by the Einstein summation convention calculation. The specific formulas are shown in Equations (9) and (10):

$${Mask}_{pred}=\mathrm{e}\mathrm{i}\mathrm{n}\mathrm{s}\mathrm{u}\mathrm{m}\left({Mask}_{embed}·{Mask}_F\right),$$

(9)

$${Mask}_{attn}\left(x, y\right)=\left\{\begin{array}{cc}\hfill 1,& \mathrm{if} \sigma \left({Mask}_{pred}\left(x, y\right)\right)>0.5\hfill \\ \hfill 0,& \mathrm{otherwise}\hfill \end{array}\right..$$

(10)

Additionally, when Mask2Former performs surface defect segmentation tasks, the $N$ mask queries in ${Mask}_{attn}$ can be roughly divided into two categories. The first category, ${Mask}_1\in R^{N_1\times H\times W}$, primarily covers the defect areas of the surface images. These mask queries are the majority and are used to separate defect and normal regions, forcing the model to focus on learning information related to the defect areas. The second category, ${Mask}_2\in R^{N_2\times H\times W}$, mainly covers the normal areas of the images, extracting more potentially valuable information from the foreground regions. Figure 3 illustrates the two types of regions of interest.

These two types of mask information satisfy Equation (11):

$${Mask}_{attn}=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left({Mask}_1, {Mask}_2\right).$$

(11)

Based on this, the first category of the regions of interest is selected from these two categories. In the first category, the mask queries result in inconsistent mask queries due to varying attention scores, causing each mask query to include part of the foreground region. The overall foreground calculation region for mask queries, which is also the calculation region $CR$ of the mask attention mechanism, is obtained by Equation (12):

$$CR=\underset{i=0}{\overset{N_1-1}{\bigwedge }}{Mask}_1\left[i\right].$$

(12)

Since the foreground mask calculation regions are generally block-shaped, the accuracy of the edge determines the accuracy of the overall calculation region. Therefore, calculating the overlap of the edge regions between $CR$ and the ground truth mask area is superior to calculating the Intersection over Union of the overall mask regions. The edge loss of the mask region is given by Equation (13):

$$l_{edge}=-\mathrm{G}\mathrm{E}\left(gt, \alpha \right)·\log \left(\mathsf{\sigma }\left(\mathrm{G}\mathrm{E}\left(CR, \alpha \right)\right)\right)-\left(1-\mathrm{G}\mathrm{E}\left(gt, \alpha \right)\right)·\log \left(1-\mathsf{\sigma }\left(\mathrm{G}\mathrm{E}\left(CR, \alpha \right)\right)\right),$$

(13)

where $\mathrm{G}\mathrm{E}$ is the boundary generation function [48], and $\alpha$ is the dilation coefficient for boundary generation, providing a more accurate reference value for the calculation region of the mask attention mechanism. Therefore, the target loss function of the MSOFormer model is given by Equation (14):

$$l=l_{cls}+l_{mask}+l_{edge}={\lambda }_{cls}{l}_{cls}+{{\lambda }_{ce}{l}_{ce}+{\lambda }_{dice}{l}_{dice}+\lambda }_{edge}{l}_{degd},$$

(14)

where the hyperparameters ${\lambda }_{cls}$, ${\lambda }_{ce}$, ${\lambda }_{dice}$, and ${\lambda }_{edge}$ are used to balance the weights of the classification loss, mask loss, and edge loss.

4. Experiments

4.1. Dataset Description

This chapter introduces the construction of the lithium battery surface defect dataset, as well as the open-source MT and NEU-Seg datasets, which share similar defect characteristics. In the subsequent chapters, comprehensive experiments on these three datasets will be conducted to demonstrate the effectiveness and generalizability of MSOFormer.

4.1.1. The Lithium Battery Surface Defect Dataset

This dataset was collected from the production line at CATL’s headquarters from October to December 2022, using a Hikvision MV-CA032-10GC camera with an effective pixel count of 3.2 million. The dataset consists of images from 325 lithium battery sets from different production batches. Each battery set was photographed from four angles (positive and negative views of the anode and cathode), resulting in a total of 1300 images. After selection by professional technicians, 1272 images were retained for the dataset. The defect pixels in the lithium battery images vary in area and shape, with a resolution of 2048 × 1536 pixels. The area proportion of various defects ranges from approximately 0.05% to 5.12%, while the foreground area occupies only 1.24%. The dataset includes six main types of surface defects in lithium batteries: tab damage, nonmetal impurity, metal scrap, electrode fold, electrode damage, and electrode weld crack. Each type of defect is manually labeled by professionals. Figure 4 illustrates the regions on the lithium battery surface, and

Table 1. Introduction to the categories of the lithium battery surface defect dataset.

Defect Category	Introduction	Example
Free	No defect
Tab damage	The welding of the tab area is crooked or damaged
Nonmetal impurity	There are foreign objects such as carbon powder, hair, fiber flocs, free flakes, etc. on the sealant
Metal scrap	It often appears in the area around the lug, producing small debris with the color characteristics of the lug.
Electrode fold	The outer electrode is folded to the battery body, left and right edges, or is not welded to the tabs.
Electrode damage	The outer cathode electrode is damaged, generally in the welding area, and the damage occurs in the transverse direction and the width exceeds half of the electrode width.
Electrode weld crack	There is a distinct black color on the electrode, or a pattern that is obviously different from the color of the solder joint and the electrode, extending horizontally from the solder joint.

provides a brief introduction to each type of defect.

As shown in the Table 1, there is a significant imbalance in the data volume across different defect categories. Since models tend to learn features from data with higher proportions, this imbalance can lead to learning biases. This paper will discuss data balancing optimization methods in the dataset preprocessing section. The entire dataset is divided into training and test sets in a 4:1 ratio. The simulation production line test data in Section 4.5 consists of images from 63 lithium batteries sourced from a different production line, and this section will analyze the model’s performance in real-time production line settings.

4.1.2. The MT Dataset

The MT dataset [49] encompasses five distinct types of surface defects on magnetic tiles: unevenness, wear, cracks, pores, and fracture defects, in addition to a normal class. The proportion of foreground pixels is 2.54%. All defect shapes are irregular, with their generation areas being random. Each image within the dataset includes pixel-level annotation masks. The MT dataset comprises a total of 1344 images. Given that MT did not provide a specific division protocol, the experiment was randomly divided into training and testing sets at a ratio of 4:1.

4.1.3. The NEU-Seg Dataset

The NEU-Seg dataset [11] encompasses three prevalent surface defects in hot-rolled strip steel: inclusions, patches, and scratches. The dataset comprises 11.76% foreground pixels, with all defects exhibiting irregular shapes and random generation areas. Each defect category is represented by 300 images, and the dataset is partitioned into training and testing sets at a 4:1 ratio.

4.2. Experimental Settings

4.2.1. Implementation Details

In the CentOS 9 environment, all models involved in the experiments were implemented using the PyTorch 1.13.1 [50] and MM-Segmentation [51] frameworks and executed on an NVIDIA RTX 4090 GPU (NVIDIA, Santa Clara, CA, USA) with 24 GB of memory. During the training phase, MSOFormer followed the same training strategy as Mask2Former. All models applied random scale jittering (between 0.5 and 2.0), random horizontal flipping, random cropping, and random color jittering for data augmentation within the pipeline, and comparison experiments were conducted on the same training and testing datasets. MSOFormer is compatible with any backbone architecture. For fair comparison, the experiments used a standard convolutional-based 50-layer ResNet [52] and a Transformer-based Swin-Transformer (Swin-T) [53] as the visual backbones. Unless otherwise specified, all backbone networks were pre-trained on ImageNet-1K [54]. Additionally, a batch size of eight was used in the experiments, and all models were trained for 80,000 iterations. MSOFormer employed the AdamW [55] optimizer with a Poly learning rate schedule, an initial learning rate of 10⁻⁴, and a weight decay of 0.05.

4.2.2. Evaluation Metrics

During the testing phase, four evaluation metrics are used to assess the performance of the proposed model across three datasets: Intersection over Union (IoU), Precision, Recall, and F1 score. IoU measures the model’s ability to differentiate between foreground and background. Precision and Recall evaluate the model’s accuracy in predicting specific categories, with Precision focusing on the false positive rate and Recall on the false negative rate. The F1 score provides a combined measure of Precision and Recall. The definitions of these four metrics are provided in Equations (15)–(18):

$$mIoU={\displaystyle \frac{1}{n_{cls}}}·\sum _i{\displaystyle \frac{n_{ii}}{{\sum }_j{n}_{ij}+{\sum }_j{n}_{ji}-n_{ii}}},$$

(15)

$$mPrecision={\displaystyle \frac{1}{n_{cls}}}·\sum _i{\displaystyle \frac{n_{ii}}{{\sum }_j{n}_{ji}}},$$

(16)

$$mRecall={\displaystyle \frac{1}{n_{cls}}}·\sum _i{\displaystyle \frac{n_{ii}}{{\sum }_j{n}_{ij}}},$$

(17)

$$F1={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}},$$

(18)

where $n_{cls}$ denotes the number of classes and $n_{ij}$ denotes the number of pixels of class $i$ predicted as class $j$.

In the simulation production line testing phase, since the test data lack GT labels for pixel-level evaluation metrics, defect recognition accuracy (Accuracy), false positive rate (False), and false negative rate (Leak) are used to assess the model’s production line performance. The recognition accuracy for a specific defect category is the percentage of correctly detected instances of that category out of the total number of detections for that category; the false positive rate is the percentage of instances of that category incorrectly detected as other categories, relative to the actual number of instances of that category; and the false negative rate is the percentage of instances of other categories incorrectly detected as that category, relative to the actual number of instances of other categories. The specific formulas are given in Equations (19)–(21):

$$Accuracy={\displaystyle \frac{TP}{TP+FP+FN}},$$

(19)

$$False={\displaystyle \frac{FN}{TP+FN}},$$

(20)

$$Leak={\displaystyle \frac{FP}{TN+FP}},$$

(21)

where TP is the number of samples correctly classified as the target category, TN is the number of samples correctly classified as other categories, FP is the number of samples incorrectly classified as the target category, and FN is the number of samples incorrectly classified as other categories.

4.3. Dataset Preprocessing

The issues of insufficient and imbalanced training samples in industrial surface defect detection have long been critical problems that need to be addressed in this field [56]. There is a significant disparity in the number of labeled samples for each category. Traditional data augmentation methods involve randomly applying several pre-defined augmentation techniques to each image, and then randomly selecting a fixed number of samples from all samples of each category as training samples. Although this approach increases the sample size, the imbalance in the number of training samples significantly restricts the average and overall accuracy of the model. To enable the model to learn more discriminative features and enhance classification robustness, this paper employs an adaptive data augmentation-based balancing strategy.

Currently, most models during training only consider positive samples in the images, while ignoring unlabeled negative samples. This can lead to misclassification of some negative samples as positive ones, affecting the accuracy of the final predicted images. In the proposed strategy, negative samples are uniformly labeled as class 0. Before data augmentation, the number of samples for each class must be iterated and the proportion of samples for each class recorded. Since each lithium battery image may contain one or more types of defects, it is not feasible to fully balance the sample proportions for each category by merely increasing the number of samples in a small dataset; instead, an approximation is used. Experiments show that the sample proportions are better balanced according to the data augmentation methods listed in the table below.

Table 2. Introduction to the quantity of the categories of the lithium battery defect dataset.

Category	Baseline		Augmentation		Adaptive
	Quantity	Proportion	Quantity	Proportion	Quantity	Proportion
Free	45	3.54%	450	3.54%	576	14.74%
Tab damage	39	3.07%	390	3.07%	429	10.98%
Nonmetal impurity	1175	92.37%	11,750	92.37%	3347	85.67%
Metal scrap	139	10.93%	1390	10.93%	832	21.3%
Electrode fold	403	31.68%	4030	31.68%	759	19.43%
Electrode damage	48	3.77%	480	3.77%	547	14%
Electrode weld crack	114	8.96%	1140	8.96%	674	17.25%
Total num	1272	1	12,720	1	3907	1
Method	mIoU	m-F1	mIoU	m-F1	mIoU	m-F1
Deeplabv3plus	73.54	79.56	74.23	80.78	76.53	85.21
SegFormer	74.77	81.02	75.34	81.54	79.04	87.04
Mask2Former	78.5	85.7	79.57	86.24	83.01	90.02

demonstrates that the adaptive data augmentation-based balancing strategy is effective in addressing the issues of insufficient and imbalanced training samples.

4.4. Comparison with Other Methods

To validate the effectiveness of the proposed method, this paper compares MSOFormer with other existing methods on a lithium battery surface defect segmentation dataset and two standard industrial surface defect segmentation datasets. The comparison methods include classical semantic segmentation approaches such as FCN [7], DeepLabv3+ [28], UperNet [57], KNet [58], SegFormer [31], and SegNeXt [32], as well as advanced methods in the industrial defect segmentation field such as McNet [59] and NadiNet [60]. Additionally, several mask-based semantic segmentation methods are considered, including MaskFormer [42], Mask2Former [22], and PEM [43].

4.4.1. Segmentation Results on the Lithium Battery Surface Defect Dataset

The quantitative comparison between the lithium battery defect test set and other methods is shown in

Table 3. Quantitative comparison with state-of-the-art methods on the lithium battery surface defect test set (%). (a) Background; (b) tab damage; (c) nonmetal impurity; (d) metal scrap; (e) electrode fold; (f) electrode damage; and (g) electrode weld crack.

Method	Backbone	Class							mIoU	m-Pre	m-Re	Flops (GB)	Param (M)
		(a)	(b)	(c)	(d)	(e)	(f)	(g)
FCN	Resnet50	99.84	96.53	32.22	63.55	85.09	63.45	64.17	72.12	85.91	79.60	57.99	47.13
Deeplabv3plus	Resnet50	99.84	97.03	38.92	71.04	82.8	65.35	80.74	76.53	88.83	83.22	59.79	41.22
UperNet	swint	99.85	97.12	44.06	65.06	85.3	70.17	79.91	77.35	86.77	85.7	236	58.95
KNet	swint	99.86	97.43	47.83	76.91	85.18	72.63	83.51	80.48	89.89	86.72	249	72.16
SegFormer	-	99.87	96.75	42.25	72.3	87.74	72.47	81.73	79.04	90.66	84.61	41.94	44.6
SegNeXt	-	99.89	98.56	42.34	72.88	89.44	72.1	79.97	79.31	89.03	86.26	32.49	27.56
McNet	Resnet50	99.87	97.88	37.47	76.92	88.45	68.7	72.26	77.36	89.7	83.23	54.35	46.77
NadiNet	-	99.87	97.6	43.75	75.53	87.25	71.46	84.24	79.96	90.92	85.35	61.92	41.27
MaskFormer	swint	99.87	97.68	49.74	82.61	87.41	73.13	85.64	82.3	90.25	88.66	54.89	46.46
Mask2Former	swint	99.88	98.54	52.16	84.16	87.21	72.6	86.56	83.01	91.09	88.98	69.44	47.4
PEM	swint	99.88	98.59	50.78	86.42	88.13	76.7	86.12	83.8	91.2	89.69	40.65	47.95
MSOFormer	Resnet50	99.87	98.47	51.09	86.3	85.32	74.4	87.34	83.26	91.48	88.71	33.74	46.4
	swint	99.89	98.56	52.86	86.75	88.28	76.79	86.12	84.18	92.15	89.56	34.69	46.83

. The following analysis can be carried out:

• The MSOFormer model demonstrates superior performance compared to other advanced models, regardless of whether the ResNet or Swin backbone network is used.
• In the incremental comparison of MSOFormer relative to other methods, the increase in mIoU is the most significant among the three evaluation metrics. For instance, compared to the KNet model, which performs best in pixel classification, MSOFormer achieved increases of 3.7%, 2.26%, and 2.84% in mIoU, mPrecision, and mRecall, respectively, showing consistent results across other methods. IoU reflects the model’s ability to differentiate between foreground and background, indicating that the mask classification mechanism of MSOFormer effectively addresses the issue of foreground–background imbalance in lithium battery defect images.
• For defect mPrecision and mRecall, MSOFormer shows more substantial improvements over Mask2Former (baseline) and achieves the best Intersection over Union (IoU) performance in the categories of nonmetal impurity, metal scrap, electrode fold, and electrode damage. In categories prone to false defects, such as nonmetal impurity and metal scrap, MSOFormer and its variants also demonstrate significant progress compared to other methods, highlighting the considerable advantage of mask classification in learning imbalanced image information. Additionally, under the same backbone network conditions, the MSOFormer model achieves higher performance with fewer parameters and computational resources, thanks to the design of the EPD network. For the detection of small target categories such as nonmetal impurity and metal scrap, the MSOFormer model, based on a multi-scale learning strategy, incorporates spatial location awareness information according to defect characteristics, enhancing the representation of small-size foreground targets and further alleviating the issue of random generation of small target positions.

Qualitative comparisons of the lithium battery surface defect image test set are shown in Figure 5. This paper presents six representative local visualization examples to demonstrate the superiority of the proposed method, with some key areas highlighted using dashed boxes. In the results of the fifth row, a subtle and shallow crack on the left side of the electrode tab is nearly imperceptible to the naked eye; only MSOFormer can extract inter-class differences and correctly classify it, demonstrating the effectiveness of the method. In the last row of results, the final solder point on the right side of the electrode tab clearly does not have a weld crack defect, and its texture and color significantly differ from the surrounding area, showing substantial intra-class variability. Pixel-based classification networks, such as FCN, KNet, and NadiNet, incorrectly identified this area as a solder crack defect, whereas MSOFormer made an accurate prediction. This indicates that the MSOFormer model effectively learns the interaction between foreground and background, highlighting the importance of foreground–background balance in semantic segmentation of lithium battery defect images. Additionally, results from other rows show that MSOFormer more accurately identifies surface defects on lithium batteries. Overall, the proposed MSOFormer model produces more accurate segmentation maps, particularly for complex and irregular defect patterns, offering finer boundary details and superior performance compared to baseline models.

4.4.2. Segmentation Results on the MT Dataset and the NEU-Seg Dataset

Table 4. Quantitative comparison with state-of-the-art methods on the MT test set (%). (a) Background; (b) Blowhole; (c) Break; (d) Crack; (e) Fray; and (f) Uneven.

Method	Backbone	Class						mIoU	m-Pre	m-Re
		(a)	(b)	(c)	(d)	(e)	(f)
FCN	Resnet50	99.54	51.77	81.05	60.97	89.68	73.73	76.12	87.06	83.98
Deeplabv3plus	Resnet50	99.29	61.25	65.19	71.99	42.63	77.63	69.66	90.00	76.02
UperNet	swint	99.68	62.74	87.22	67.55	93.41	81.57	82.03	89.51	89.73
KNet	swint	99.68	63.99	88.32	71.03	93.14	81.33	82.91	89.26	91.22
SegFormer	-	99.67	55.95	87.6	69.7	92.46	81.01	81.06	88.52	89.12
SegNeXt	-	99.66	66.29	89.5	75.55	94.65	78.38	84.0	91.62	90.23
McNet	Resnet50	99.54	64.91	84.09	72.59	92.9	71.51	80.92	90.49	87.54
NadiNet	-	99.59	64.53	89.14	71.27	92.42	74.51	81.91	92.44	86.93
MaskFormer	swint	99.56	70.98	87.59	74.84	93.66	72.71	83.22	90.21	90.76
Mask2Former	swint	99.69	71.83	88.98	73.58	93.76	81.99	84.97	89.55	93.85
PEM	swint	99.68	73.25	89.53	75.71	93.32	80.85	85.39	91.92	91.79
MSOFormer	Resnet50	99.69	65.15	88.8	75.46	94.93	80.71	84.12	90.1	92.06
	swint	99.7	74.14	90.06	73.95	93.3	82.04	85.53	91.52	92.4

and

Table 5. Quantitative comparison with state-of-the-art methods on the NEU-Seg test set (%). (a) Background; (b) Inclusion; (c) Patches; and (d) Scratches.

Method	Backbone	Class				mIoU	m-Pre	m-Re
		(a)	(b)	(c)	(d)
FCN	Resnet50	97.5	73.39	85.44	79.88	84.05	90.66	91.56
Deeplabv3plus	Resnet50	97.56	72.86	85.7	81.92	84.51	91.12	91.62
UperNet	swint	97.84	77.25	86.7	83.64	86.36	92.10	92.94
KNet	swint	97.79	77.48	86.58	83.15	86.25	91.6	93.33
SegFormer	-	97.81	76.82	86.63	83.1	86.09	92.15	92.57
SegNeXt	-	97.84	77.91	86.65	83.29	86.42	91.84	93.29
McNet	Resnet50	97.65	76.11	85.71	81.29	85.19	91.39	92.23
NadiNet	-	97.62	75.0	86.48	82.14	85.31	89.71	94.29
MaskFormer	swint	97.83	78.54	86.31	83.62	86.57	91.91	93.42
Mask2Former	swint	97.87	77.83	86.68	83.45	86.46	92.43	92.75
PEM	swint	97.85	78.45	86.44	84.16	86.73	91.89	93.62
MSOFormer	Resnet50	97.79	77.06	86.3	83.64	86.2	91.93	92.92
	swint	97.97	79.08	87.27	83.86	87.05	93.21	92.71

report the quantitative comparisons on the MT and NEU-Seg test sets, respectively. Both MT and NEU-Seg are open-source industrial surface defect datasets, characterized by imbalances between foreground and background and irregular defect shapes. This study evaluates the generalization ability of MSOFormer using these two open-source datasets. The following conclusions can be drawn:

• MSOFormer consistently outperforms other methods, showing significant advantages in mIoU, mPrecision, and mRecall, which demonstrates the superiority and generalization capability of the proposed MSOFormer model.
• A further analysis of the IoU scores for each category reveals that MSOFormer achieves the best performance in categories such as Blowhole, Break, Crack, and Uneven in the MT dataset, as well as Inclusion and Patches in the NEU-Seg dataset. In some categories, its performance is only slightly behind other mask classification methods. This indicates that the proposed method is more capable of modeling complex and irregular targets and highlights the importance of mask classification approaches in the semantic segmentation of industrial surface defect images.
• Compared to the baseline, with the same backbone, MSOFormer shows improvements in mIoU and mPrecision over the baseline model Mask2Former by 0.56% and 1.97% on the MT dataset, and 0.59% and 0.78% on the NEU-Seg dataset. These results preliminarily demonstrate the advantages of the DSQ and MBL modules, providing a more accurate and robust segmentation model and exploring a new paradigm for foreground–background separation in industrial surface defect image semantic segmentation. More details of the ablation experiments are analyzed in Section 5.3.

4.5. Simulation Production Line Test

To evaluate the operational efficiency of MSOFormer in a production pipeline and its detection rate for lithium battery defects, professionals utilized an additional 63 sets of lithium battery packs to simulate the production line and test the model. The model was run on an RTX3060 GPU using the TensorRT acceleration framework.

Due to the absence of standard ground truth masks on the production line, it was impossible to calculate pixel-level evaluation metrics such as IoU. Therefore, detection rate, false detection rate, and missed detection rate were used to assess the models’ performances. The models’ operational efficiencies were measured using an input resolution of 512 × 512 pixels, and the speed was evaluated in frames per second (fps).

As shown in

Table 6. Ablation study of the effectiveness of each modules on the lithium battery surface defect dataset. (a) Free; (b) tab damage; (c) nonmetal impurity; (d) metal scrap; (e) electrode fold; (f) electrode damage; (g) electrode weld crack.

Method	Class							Macc	Mfal	Mleak	Fps
	(a)	(b)	(c)	(d)	(e)	(f)	(g)
FCN	45.97	88.89	38.03	45.83	95.12	85.71	100.0	71.37	0.52	20.50	222.92
Deeplabv3plus	63.53	80.0	74.58	38.89	86.81	66.67	89.47	71.42	3.1	10.88	88.88
UperNet	74.29	100.0	87.03	60.71	89.89	75.0	100.0	83.85	3.42	6.01	59.55
KNet	79.69	100.0	91.74	88.0	91.67	100.0	100.0	93.01	3.71	2.24	26.84
SegFormer	77.46	88.89	83.54	68.0	96.39	75.0	100.0	84.18	1.79	6.68	97.22
SegNeXt	73.42	100.0	80.43	58.62	97.56	85.71	100.0	85.11	0.68	7.69	86.9
McNet	47.27	100.0	46.03	45.83	96.34	100.0	100.0	76.5	2.68	18.22	194.12
NadiNet	73.61	100.0	83.75	74.07	96.39	100.0	100.0	89.69	2.85	4.69	75.9
MaskFormer	76.67	100.0	92.37	95.83	95.18	85.71	100.0	92.25	6.31	0.54	70.95
Mask2Former	90.16	100.0	94.92	95.83	93.98	85.71	100.0	94.37	1.44	1.14	32.13
PEM	86.89	100.0	94.54	88.46	95.18	100	100	95.01	2.39	0.90	55.55
MSOFormer	93.33	100.0	96.2	100	95.24	100	100	97.82	1.36	0.48	54.21

, MSOFormer demonstrated superior performance on the TensorRT acceleration framework, achieving higher detection rates, lower false detection rates, and lower missed detection rates, while also achieving higher FPS compared to the baseline model.

5. Discussion

5.1. Certification of MBL Module

This section analyzes the mask attention mechanism of the Mask2Former model, focusing on the correlation $\gamma$ between the foreground regions of interest for $N$ mask queries and the true mask regions, and its correlation with the model performance limit $\delta$. Here, $\gamma$ represents the IoU between the regions of interest and the true regions, while $\delta$ denotes the IoU between the model’s predictions and the true masks. The Pearson correlation coefficient between $\gamma$ and $\delta$ was calculated and followed by a quadratic polynomial fitting. The Pearson correlation coefficient measures the linear relationship between two variables and is calculated based on the covariance matrix of the data. The formula for the Pearson correlation coefficient between $\gamma$ and $\delta$ is as follows:

$$\mathrm{C}\mathrm{o}\mathrm{r}\mathrm{r}\left(\gamma , \delta \right)={\displaystyle \frac{\mathrm{c}\mathrm{o}\mathrm{v}\left(\gamma , \delta \right)}{\sqrt{\mathrm{v}\mathrm{a}\mathrm{r}\left(\gamma \right)\times \mathrm{v}\mathrm{a}\mathrm{r}\left(\delta \right)}}},$$

(22)

where $\mathrm{c}\mathrm{o}\mathrm{v}\left(\gamma , \delta \right)$ is the covariance of $\gamma$ and $\delta$, and $\mathrm{v}\mathrm{a}\mathrm{r}\left(\gamma \right)$ and $\mathrm{v}\mathrm{a}\mathrm{r}\left(\delta \right)$ are the variances of $\gamma$ and $\delta$, respectively. The computed Pearson correlation coefficient of 0.7205 indicates a significant positive correlation between $\gamma$ and $\delta$, with a strong linear relationship. The significance level, with a p-value of 1.161 × 10⁻¹⁷, is far below 0.05, confirming the statistical significance of this correlation.

The relevant data of $\gamma$ and $\delta$ were fitted linearly and quadratically, and the RMSE of the linear fitting and quadratic fitting were 5.852 and 5.751 respectively. Therefore, the relationship between $\gamma$ and $\delta$ can be more intuitively observed through quadratic polynomial fitting, as shown in the curve graph in Figure 6.

The fitted curve indicates that as $\gamma$ increases, $\delta$ generally shows an upward trend, further confirming the strong positive correlation between $\gamma$ and $\delta$. However, in higher ranges of $\gamma$, the growth rate of $\delta$ slows down and even begins to decline. This phenomenon suggests that the region of interest should include not only the GT mask area but also its surrounding area, meaning that a higher overlap between the region of interest and the expanded GT mask area results in better model performance. Therefore, the expansion coefficient $\alpha$ of the GT mask area becomes a critical factor in determining the effectiveness of the MBL guidance module. An ablation study of related parameters will be presented in Section 5.3.

5.2. Study of Hyperparameters

In the MSOFormer model, four hyperparameters directly influence network performance: the number of mask queries $N_q$, the number of mask query decoders $N_{td}$, the boundary generation expansion coefficient $\alpha$, and the MBL loss weight $\beta$. The following is an ablation study of these hyperparameters on the lithium battery defect dataset.

The number of mask queries $N_q$ and the number of mask query decoders $N_{td}$ both affect the quality of the mask query generation. In the experiments, $N_{td}$ was set to 3, 6, 9, and 12 (corresponding to 1, 2, 3, and 4 decoder groups), while $N_q$ was set to 20, 50, 100, and 200. As shown in the 3D curves in Figure 7, as $N_q$ increases from 20 to 100 and beyond, the mIoU initially increases and then stabilizes, while the model performance stabilizes when $N_{td}$ reaches 9 layers or more. Considering the impact of $N_q$ and $N_{td}$ on model running speed, the final hyperparameters are set to 50 and 9, respectively.

The MBL loss weight $\beta$ and the boundary generation expansion coefficient $\alpha$ both affect the precision of the MBL module in generating guidance for the region of interest. Therefore, these two hyperparameters were evaluated separately, with one parameter fixed and the other varied. First, $\beta$ was set to 20 to analyze the boundary generation expansion coefficient $\alpha$, which ranged from 0 to 4. The performance curve is shown in Figure 8a. The results indicate that mIoU increases and then decreases, with the highest mIoU when $\alpha$ is 1. Similarly, in the analysis of $\beta$, $\alpha$ was set to 1 and $\beta$ was varied from 0 to 40. Figure 8b shows that the model’s mIoU performs best when $\beta$ is 20. Thus, the hyperparameters $\alpha$ and $\beta$ are finally set to 1 and 20, respectively.

5.3. Effectiveness Analysis of the Modules

In the proposed MSOFormer model, the EPD focuses on addressing the irregular shapes of foreground objects, DSQ mitigates the random generation of small foreground objects in defect images, and the MBL mechanism aims to address the issue of foreground–background imbalance. To better evaluate the contribution of each module, we conducted ablation studies using eight different settings to analyze the effectiveness of EPD, DSQ, and MBL. Among these settings, the first corresponds to the baseline model Mask2Former, while the eighth represents the complete MSOFormer model. The comprehensive ablation study results on the lithium battery defect dataset are shown in

Table 7. Ablation study of the effectiveness of each of the modules on the lithium battery surface defect dataset.

Ablation Setting	EPD	DSQ	MBL	mIoU (%)	mF1 (%)	Flops (GB)	Params (M)
1				83.01	90.02	69.44	47.4
2	✓			83.63 (+0.62)	90.29 (+0.27)	34.48	46.83
3		✓		83.76 (+0.75)	90.36 (+0.34)	69.86	47.4
4			✓	83.72 (+0.71)	90.37 (+0.35)	69.44	47.4
5	✓	✓		83.94 (+0.93)	90.47 (+0.45)	34.69	46.83
6	✓		✓	84.05 (+1.04)	90.55 (+0.53)	34.48	46.83
7		✓	✓	83.99 (+0.98)	90.54 (+0.52)	69.86	47.4
8	✓	✓	✓	84.18 (+1.17)	90.78 (+0.76)	34.69	46.83

, documenting how the model continuously improves segmentation performance, with the improvements relative to the baseline shown in parentheses. The following conclusions can be drawn:

(1)

Adding EPD and DSQ to the baseline module resulted in improvements across all metrics. Compared to the baseline model Mask2Former, DSQ can more effectively integrate the semantic and visual representations of randomly dispersed objects through positional information.

(2)

The MBL module provides a more significant performance boost compared to the traditional mask attention mechanism, with improvements of 0.71% and 0.34% in mIoU and m-F1, respectively.

(3)

The EPD, DSQ, and MBL modules are complementary and work synergistically to enhance the final model’s performance. Thus, the proposed MSOFormer can achieve state-of-the-art performance on the lithium battery defect dataset by learning a balanced interdependence between foreground and background.

6. Conclusions

The segmentation of surface defects in lithium batteries is crucial for various industrial applications, particularly in the manufacturing process, as it significantly impacts product quality and reliability. To address issues of false positives and negatives caused by the severe foreground–background imbalance in defect images, this paper proposes a defect segmentation approach based on the mask classification model MSOFormer. By introducing the MBL module, the boundaries of the mask computation area in the mask attention mechanism are refined to achieve a more complete foreground region. Additionally, challenges such as the randomness of defect location generation and irregular shapes present further difficulties in defect segmentation. The DSQ module leverages the spatial attributes of defect images, assigning the spatial location data to each mask query. The EPD, through a fully convolutional FPN architecture design with relatively low computational complexity, captures the deformed receptive field of irregular defects, thereby enhancing the quality of the defect segmentation.

Comprehensive experiments conducted on the lithium battery surface defect dataset and two public datasets with similar defect characteristics demonstrate that the proposed MSOFormer achieves higher accuracy compared to other state-of-the-art defect detection methods. A series of ablation studies also confirm the effectiveness of each module in MSOFormer. Furthermore, the visualization results on the lithium battery surface defect test set indicate that MSOFormer and other mask classification tasks achieve lower false positive and false negative rates, further validating the reliability of the model’s final recognition results.

Given the high computational complexity of the mask attention mechanism in the MSOFormer model, which demands significant computational power for real-time production lines, future research will focus on optimizing algorithms for the mask attention mechanism to reduce the computational requirements for real-time production lines.