DMnet: A New Few-Shot Framework for Wind Turbine Surface Defect Detection

Abstract

In the field of wind turbine surface defect detection, most existing defect detection algorithms have a single solution with poor generalization to the dilemma of insufficient defect samples and have unsatisfactory precision for small and concealed defects. Inspired by meta-learning ideology, we devised a cross-task training strategy. By exploring the common properties between tasks, the hypothesis space shrinks so that the needed sample size that satisfies a reliable empirical risk minimizer is reduced. To improve the training efficiency, a depth metric-based classification method is specially designed to find a sample-matching feature space with a good similarity measure by cosine distance. Additionally, a real-time feedback session is innovatively added to the model training loop, which performs information enhancement and filtering according to the task relevance. With dynamic activation mapping, it alleviates the information loss during traditional pooling operations, thus helping to avoid the missed detection of small-scale targets. Experimental results show that the proposed method has significantly improved the defect recognition ability under few-shot training conditions.

1. Introduction

Blades and towers are the key components of wind turbines (WTs). The former carries the main load to obtain wind energy, while the latter plays a vital role in supporting and absorbing the vibration of the generator set. A survey has suggested that these two parts account for 42% of the total cost of a wind turbine [1]. However, WTs usually operate in remote fields. Due to the harsh environment and complex working conditions, the surfaces of wind turbine blades and towers may be prone to defects such as cracks, coating peeling, edge corrosion, blisters, and pits [2,3], which will reduce the energy conversion rate and have a negative impact on the power generation quality and the unit life. If the restoration work is not completed in time, it may become a major safety hazard, or even a serious power accident in the long run. Therefore, the fast and efficient detection of WT surface defects has become an urgent task.

Through a review of the literature for WT defect detection, it was found that the earlier traditional detection methods mainly rely on different types of signal sensors for diagnosis, such as ultrasonic [4], vibration [5], acoustic emission [6], and infrared thermography [7] techniques. However, the above methods with high cost have poor stability in harsh environments, resulting in low detection efficiency. Therefore, the mainstream research has gradually begun to focus on image-based visual detection.

With the wide application of UAV technology, the current research on WT surface defect detection based on UAV images is mainly divided into two categories: artificial feature methods and deep learning methods. Methods based on artificial features first extract features such as contour, texture, and color (ex. Haar-like [8], LBP [9], SIFT [10], and HOG [11]) of image samples through traditional image processing techniques and then train commonly used classifiers such as SVM (support vector machine) [12], Adaboost [13], and Random Forest [14] for defect recognition. Considering the distribution, severity, and development trend of cracks, Peng and Liu [15] proposed a crack analysis method combining Wiener filtering and the adaptive median filtering algorithm, which can effectively reduce the negative impact of motion blur. Chen and Shen [16] investigated a three-point slope deviation method to monitor the operating condition of WT blades in a visual inspection way, but they did not consider the impact of blade pitch, non-fault deformation, and bad weather on the system. Long Wang et al. [17] select Haar-like features to depict cracks on WT blades. Additionally, an extended cascade classifier is developed to perform crack detection on UAV inspection images, using a stretchable scan window to locate crack regions. However, pre-processing and post-processing of the signal is still complex and time-consuming, and detection is limited to specific damage types.

In spite of having a certain degree of expressiveness, the manually extracted features lack sufficient effective information due to the low signal-to-noise ratio, so that the precision cannot meet the requirements of industrial applications. Meanwhile, it is time-consuming to select appropriate features.

Applying the automatic feature extraction of CNN networks, deep learning methods [18,19,20,21,22] have received considerable attention recently; their accuracy and efficiency are superior to those based on artificial features. Using the VGG-11 model, Xu et al. [23] provide an automatic feature extractor for defect blades and employ the “alternating direction method of the multiplier” algorithm for model compression to reduce the requirements for hardware equipment. However, as only an 11-layer CNN was employed, Xu’s classification experiments yielded unfavorable results. Yang [24] used the ResNet50 algorithm to identify multiple types of leaf defects, achieving a recognition accuracy of more than 95%, while this result was obtained by an unbalanced dataset with only 10% of defect samples, which is an inadequate validation. Qiu et al. [25] designed a WT detection system YSODA with an improved YOLOv3 algorithm. They modified the YOLO architecture to support multi-scale feature pyramids in CNN and expanded the number of samples by image enhancement, to improve the performance on small-size defect detection. On the downside, the speed of the system is slower than before due to the increased network complexity. Shortly afterwards, Shihavuddin et al. [26] provide an automatic recommendation system by exploiting a faster RCNN algorithm. To adapt to high-resolution images and difficult-to-separate samples, an augmentation step called the “multi-scale pyramid and patching scheme” is proposed to achieve higher sensitivity.

Despite a proliferation of studies that have been published in this field, the following issues remain to be discussed:

• In practical applications, owing to the generally rare defect data, deep network training is prone to overfitting. Collecting large-scale annotation data from scratch, however, is time-consuming and expensive. Existing studies are limited to augmentation from a data perspective, which performs conventional deep learning by directly expanding the number of samples. However, few scholars have drawn on any systematic research into considerations in model composition or into training patterns. Knowing how to construct a robust detection model under a few-shot data scenario without sample expansion is a primary concern.
• The diversity of shooting time, angle, and distance of UAVs increases the difficulty of equipment image defect detection, which ensures the existing deep learning-based methods have low recognition accuracy for extremely small and concealed defects.

For the above situation, we developed a new few-shot training framework for wind turbine surface defect detection, and we present our phased design and field trials in this article. The contributions of this article are as follows:

(1)

Inspired by meta-learning, a cross-task training strategy is designed pertinently for WT surface-defect recognition. Using the MVTec dataset as raw material, a series of different but related tasks are constructed, to find common guidelines of defect identification in all things rather than learning the given data itself. Without expanding the amount of original data, we achieve high-precision defect recognition with only 20 training samples.

(2)

To alleviate the huge computational cost of traditional classifiers with supervised labels, we establish a non-parametric connection between samples by cosine distance in a high-dimensional vector space and expect the network to learn a general similarity metric that maintains identity across the entire data. It helps to quickly distinguish similarities and differences for unfamiliar data, thereby improving the training efficiency.

(3)

To tap the potential of identifying small defects and hidden defects, the depth feature map is additionally overlaid with an equivalent soft mask map to enhance task-relevant information and filter redundant information according to task relevance, helping to make real-time feedback corrections in model training.

(4)

In this article, class activation mapping (CAM) technology is innovatively integrated into each round of training. This study explores, for the first time, the dynamic interpretability of feature space in the training state, to understand and uncover the secrets of the “black box” in deep learning.

2. Methodology

2.1. Motivation

In supervised machine learning, the learning process is often approximated by fitting a function$f$ to a dataset $D$. According to the empirical risk minimization theorem [27,28], since the true optimal hypothesis $\widehat{h}$ is unknown, we give a hypothesis space $H$ and expect to find the best approximation hypothesis $h_I$ in $H$ that satisfies the empirical risk minimization through model training.

Therefore, for a dataset $D$ with $I$ training samples, the total error between the empirical risk and the actual expected risk is as follows:

$$\mathbb{E}\left[R\left(h_I\right)-R\left(\widehat{h}\right)\right]=\mathbb{E}\left[R\left(h^{\ast }\right)-R\left(\widehat{h}\right)\right]+\mathbb{E}\left[R\left(h_I\right)-R\left(h^{\ast }\right)\right]=e_{app}\left(H\right)+e_{est}\left(H,I\right)$$

(1)

The approximation error $e_{app}\left(H \right)$ represents how close the function in $H$ can approximate the true optimal hypothesis $\widehat{h}$, and estimation error $e_{est}\left(H,I\right)$ measures the effect of replacing expected risk minimization with empirical risk minimization. Obviously, the total error is affected by hypothesis space $H$ and sample size $I$.

As shown in Figure 1a,b, for common supervised learning, when there is sufficient training data with supervised information, $h_I$ can provide a good approximation. Conversely, in the case of few training samples, $e_{est}\left(H, I\right)$may exceed a reasonable range. $h_I$ is no longer reliable, and overfitting occurs [29].

In response to this problem, most studies have mostly focused on performing data-augmentation techniques. However, since the augmentation strategy is only applicable to a specific data set, it is not easy to migrate to other datasets. Moreover, the effect relies heavily on the quality of newly generated data. It is reasonable to believe that such research does not tackle the problem radically.

Interestingly, we found the following phenomenon in human learning: when people learn a series of different but related tasks, they can draw inferences from each other by distilling the cross-task knowledge and skills. For new tasks, this kind of generalization ability ensures that the model has rules to follow and gets started quickly.

Inspired by the above idea, meta-learning, also known as learning to learn [30], enables the machine to find a set of effective “learning paths” in the past abundant tasks, so that it can generalize well on unseen tasks. Particularly, the generalization achieves good performance only with few training samples.

As Blumer argues [31], the lower bound on the number of training samples required for fitting a function can be estimated. Specifically, if sample size I satisfies:

$$I\ge \frac{1}{-\ln \left(1-\epsilon \right)}\left(\ln \left(\left|H\right|\right)+\ln \left(\frac{1}{\delta }\right)\right)$$

(2)

Any hypothesis $h$ in $H$ that is consistent with the objective function $f$ on dataset $D$ will guarantee with probability$\left(1-\delta \right)$ that the error rate of predictions on future data is lower than $\epsilon$.

It is clear that the sample size required depends only on three variables: the expected error rate $\epsilon$, the guaranteed probability $\delta$, and the size of hypothesis space $H$, regardless of the objective function $f$ and the data’s distribution. Therefore, under the premise of ensuring that $H$ always contains $f$, the sample size can be reduced by shrinking $H$.

Meta-learning simultaneously learns on multiple tasks with common properties. For example, different image recognition tasks follow the translation invariance and rotation invariance of images. In Figure 1c, these common attributes prune the hypothesis space $H$, and narrow the search area of optimation parameters, thereby reducing the training complexity of the training and helping model adapt well to few-shot scenarios.

That is, meta-learning is practical for solving few-shot learning problems.

2.2. Introductory Definition

The idea of meta-learning is divided into two main phases: meta-train and meta-test [32] (Figure 2). In meta-train, we expect the model to acquire generalizability across tasks in the task distribution $P\left(T\right)$. To solve this problem in practice, a set of respective independent source tasks is usually sampled from $P\left(T\right)$, where each task $T \in P\left(T\right)$ has a respective task-related training and respective testing data, i.e., $D_T=\left\{D_s,D_q\right\}$, and $D_s$, $D_q$ are called the support set and query set, respectively. The purpose of training is to optimize the meta-parameter $\theta$ such that

$$\underset{\mathsf{\theta }}{\min }{\mathbb{E}}_{T\in P\left(T\right)}L\left(D_T;\mathsf{\theta }\right)$$

(3)

where $L\left(D_T;\mathsf{\theta }\right)$ denotes the loss over the data set $D_T$ with the model, and the optimal solution ${\theta }^{\ast }$ can be regarded as the learned cross-task knowledge (or meta-knowledge). In the meta-test, the model is tested on a new task $T_{new} \in P\left(T\right)$ that is disjointed from the data used for the source tasks in meta-train.

In contrast to traditional machine learning, the fast adaptation to the training set of the target task $T\_new$ benefits from regarding the meta-knowledge ${\theta }^{\ast }$ as prior knowledge. ${\theta }^{\ast }$ could be the estimation of the model’s initial parameters, hyperparametric optimization strategies, the neural network architecture design, etc.

2.3. DMnet: A New Few-Shot Training Framework

2.3.1. The Overall Looking

Inspired by meta-learning ideology, we construct a new few-shot training framework called DMnet for wind turbine defect detection. The basic flow chart is suggested in Figure 3. To absorb prior knowledge, the machine learns from a large number of tasks to obtain high-level generalization capability. In each task, the pre-processed image data undergoes feature extraction by CNN to get a deep feature map, which is finally input to the metric module for category determination. Further, the proposed dynamic activation mapping strategy monitors this process and provides real-time feedback and corrections. After that, the machine becomes a more powerful learner. For our target task, i.e., WT surface defect detection, defect recognition and location on unseen samples can be achieved with just only a small amount of supervised sample fine-tuning.

2.3.2. The Cross-Task Training

From Section 2.3.1, meta-learning requires the support of multiple different but related tasks, each of which has its own training and test sets. To realize the snap recognition of WT surface defects, it is necessary to construct multiple tasks with similar settings to this, which will participate in meta-training as training sets.

In the paper, the MVTec anomaly detection dataset [33], including 15 items such as toothbrushes, leather, pills, wood, etc., is selected as data support for meta-learning (Figure 4).

As depicted in Figure 5, we specifically design the following three-stage process: meta-training, fine-tuning, and meta-testing. Dataset $D$ is divided into three mutually exclusive meta-sets: meta-training set $D_{meta-train}$, meta-validation set $D_{meta-valid}$, and meta-testing set $D_{meta-test}$. MVTec is used to build the first two parts, in which the wood dataset builds $D_{meta-valid}$, while the other 14 items build $D_{meta-train}$.

In the first stage, we construct several tasks for meta-training and expect the model to learn the task-to-task generalization ability: meta-training set $D_{meta-train}$ is a dataset collection of $n_{tasks}$tasks, i.e., $D_{meta-train}=\left(D_{task1},D_{task2},\dots \right)$. Each task dataset $D_{task}=\left(D_{Support},D_{query}\right)$, where support set $D_{Support}$ and query set $D_{query}$correspond to the training process and testing process in a single task, respectively. Applying random sampling for each class, we get $n_{train\_s}$ sheets for support set and $n_{train\_q}$ for query set. Meta-validation set $D_{meta-valid}$ is built to identify defects of wood. Similarly, we have $n_{val\_s}$ sheets for support set and $n_{val\_q}$for query set, to observe the trend of model generalization performance on a new task over model training.

However, we observed that the defects in the MVTec dataset are relatively simple and obvious, while WT surface defects often have a complex background, which makes the defect identification more challenging. Inevitably, the difference between our target task and either task in the meta-training stage will be significantly greater than the within-group task difference. To alleviate the negative influence, the fine-tuning in the second stage performs supervised learning on the support set in meta-test set $D_{meta-test}$.

Finally, the target few-shot task is tested. The meta-test set $D_{meta-test}$ serves our target task—WT surface defect detection, with $n_{wind\_s}$ sheets as the support set and $n_{wind\_q}$ sheets as the query set.

It should be noted that $n_{val\_s}$, $n_{train\_s},$and $n_{wind\_s}$ must be small numbers in order to match the few-sample circumstance. In general, the size selection for both the support set and the query set in the meta-training stage requires comprehensive consideration of computing resources. The principle is to improve the information utilization rate as much as possible with limited resources.

2.3.3. Feature Extraction

By analyzing the characteristics of WT’s drone images, it can be found that the defects such as cracks and edge corrosion are in various irregular forms, which are difficult to describe uniformly by a kind of specific feature. This raises demand for the semantic analysis ability of the model. Moreover, factors such as the large variety of defect scales, the variable perspective of images and the complex background also increase the recognition difficulty. Therefore, we choose CNN for the feature extraction in this case.

Simulating the structure of the human brain, CNN-based deep learning techniques build multilayer neural networks to extract low-level and high-level features from the input data layer by layer [34]. Through this hierarchical method, it effectively establishes the mapping relationship from the underlying signal to the high-level semantics. While improving the target recognition rate, it avoids the complicated operation of manually designed features in the development of traditional defect recognition.

The convolutional layer and the pooling layer are the most important components of CNN. Combining these two elements can form a variety of CNN feature extractors. The shallow neurons express edge and angle information, while the role of the convolutional layer is to gradually extract more abstract structural information by increasing the local perceptual field. The convolution kernel is calculated as follows:

$$y_{p\prime }^L=f_{ac}\left({\displaystyle \sum }_{i\in M_{p^{\prime }}} y_p^{L-1}\ast k_{p{p}^{\prime }}^L+b_{p^{\prime }}^L\right)$$

(4)

where $M_{p\prime }$ denotes the input feature map; $y_p^{L-1}$ represents the convolution kernel to which the $p$th feature map of the previous layer is connected with the $p\prime$th feature map of the $L$-th layer; $\ast$ is the convolution operation; $b_{p\prime }^L$ is the bias; and $f_{ac}\left(·\right)$ is the activation function.

The pooling layer, also called the down-sampling layer, reduces the dimensionality of the feature map by compressing the image, which can improve model’s noise immunity and maintain some invariance of the features (rotation, translation, stretching, etc.).

2.3.4. The Metric Classification Module

During the meta-training, the support set in each task helps the model build a classification pattern, and the query set validates and tunes the strengths and weaknesses of the pattern. Inspired by deep metric learning, we expect the model to learn the essential associations of things.

Therefore, our meta-learner aims to obtain a great cross-task feature space for similarity measurement, which performs well on new tasks. We develop a metric module for classification based on cosine distance measurement. The higher the similarity score between a test sample and the given data, the more likely they belong to the same class.

In Figure 6, the process of classifying defects in a single task is as follows:

Define a task $t$ with support set $D_{support}^t$ and query set $D_{query}^t$.

First, the embedding model is constructed by a convolutional neural network (CNN). In this way, each sample in $D_{support}^t$can be converted into a vector representation, i.e., a point in high-dimensional space. Next, calculate the similarity of any two vectors in the feature space by the metric module. Here, we have the cosine distance for measurement, which is a parameter-free distance metric. The similarity output forms the final prediction probability of the query sample. The overall similarity of a query sample $x_q$ to the example set ${\widehat{x}}_k$ of category $k$:

$$\sin \left(x_q,{\widehat{x}}_k\right)=\frac{1}{N}{\displaystyle \sum }_{n=1}^N\cos \left(x_q,x_i^k\right)$$

(5)

where $x_i^k$ is the $n$-th sample in ${\widehat{x}}_k$ and cos represents the cosine function. Then, inscribe its prediction probability on all categories:

$$pred\left(x_q\right)=\frac{1}{{{\displaystyle \sum }}_{j=1}^K{e}^{\sin \left(x_q,{\widehat{x}}_j\right)}}[e^{\sin \left(x_q,{\widehat{x}}_1\right)},e^{\sin \left(x_q,{\widehat{x}}_2\right)},\dots ,e^{\sin \left(x_q,{\widehat{x}}_K\right)}]$$

(6)

The one with the highest score is the prediction category for $x_q$.

2.3.5. The Dynamic Activation Mapping Strategy

To address the issue of low recognition accuracy for small and concealed targets in WT surface defect detection, a dynamic activation mapping strategy is introduced in the embedding model at the single task level. Figure 7 describes how it implements:

Assume that for any image sample $z$ with category label $c$, a depth feature map $fea$ is obtained after feeding it into a CNN-based embedding model. In the beginning, a target layer $Laye{r}_{target}$ needs to be selected. In the context of this paper, we believe that the target layer should neither be too shallow nor too deep: if it is too shallow, excessive noise will interfere with the accurate representation of WT defective features, while if it is too deep, the features of small-scale defects will be drowned in the surrounding pixels due to the high level of feature map extraction. Collectively, target layer selection depends on the average percentage of defects in the original image. After performing pre-forward propagation and gradient back-propagation, record the target layer’s activation value as well as the target layer’s gradient information by backpropagation of the score on category $c$.

The channel weight vector on neurons is obtained,

$${\beta }_z^c=\frac{1}{w_{tar}\ast h_{tar}}{\displaystyle \sum }_i {\displaystyle \sum }_j-\frac{\partial scor{e}_c}{\partial T_{ij}^z}$$

(7)

where $w_{tar}$ and $h_{tar}$ represent the length and width of $Laye{r}_{target}$, $scor{e}_c$ represents sample score on category $c$, and $T_{ij}^k$represents pixel value in the $i$-th row and $j$-th column in $Laye{r}_{target}$.

In accordance with the activation value and channel weight vector, we implement linear feature fusion on the channel dimension for $Laye{r}_{target}$. Especially,

$$L_{map}^z={\displaystyle \sum }_l{\beta }_z^c\left(l\right)Activatio{n}_l^z$$

(8)

where $ctivatio{n}_l^z$, ${\beta }_z^c\left(l\right)$ represent the activation map and contribution on the $l$-th channel, respectively.

Only pixel points that have a positive influence on the category are taken into account, so the Relu function is applied to the heat map. Generate a rough activation heat map, which means the weight contribution distribution in the spatial dimension on $Laye{r}_{target}$.

$$Ca{m}_{coarse}=Relu\left(L_{map}^z\right)$$

(9)

Based on the min-max normalization criterion, a normalization operation is applied. The pixel value in the $i$-th row and $j$-th column of the new heat map $Ca{m}_{coarse}^{\prime }$ is

$$C_{ij}^k{}^{\prime }=\frac{C_{ij}^k-\min (Ca{m}_{coarse})}{\max (Ca{m}_{coarse})-\min (Ca{m}_{coarse})+eps}$$

(10)

where $C_{ij}^k$ represents pixel value in the $i$-th row and $j$-th column in $Ca{m}_{coarse}$. $\max (Ca{m}_{coarse})$ and $\min (Ca{m}_{coarse})$ are the maximum and minimum values of all elements in $Ca{m}_{coarse}$, respectively. $eps$ is a pretty small number to prevent overflow, which is taken as 10 × 10⁻⁶.

Then, adapt $Ca{m}_{coarse}^{\prime }$ to the same size as the depth feature map $fea$ to get the equivalent soft mask map:

$$Cov=avgpooling\left\{Ca{m}_{coarse}^{\prime }\right|r=\frac{siz{e}_{cam}}{siz{e}_{fea}}\}$$

(11)

where $siz{e}_{cam}$ and $siz{e}_{fea}$ are the length (or width) of $Ca{m}_{coarse}^{\prime }$ and $fea$, $r$ is the down-sampling rate, and $avgpooling$ means the average pooling operation.

Finally, overlap the equivalent soft mask map on $fea$ to achieve the enhancement of valid information and the filtering of redundant information on the basis of defect correlation. The enhanced feature map

$$fea\prime =Cov\otimes fea$$

(12)

where $\otimes$ refers to the element multiplication operation.

We replace $fea$ with $fea\prime$ as input to the subsequent classification module for defect recognition. The equivalent soft mask map can be continuously and dynamically updated according to the current parameters, thus assisting the model training.

As for the reason why this strategy can alleviate the accuracy degradation caused by small and concealed defects, we take the following two underlying factors into consideration: on the one hand, a dynamic activation mapping strategy, as a means of information filtering, not only alleviates the pressure of high convolutional computation but also enhances task-related information on the depth feature map based on defect correlation metrics, which increases the training efficiency and the exploitation of the model’s potential on the target task. On the other hand, the pooling operation performs forced-down sampling in order to make the image size smaller. In the base model, scholars are generally accustomed to choosing the last layer of features as the input to the classification module. When the defect accounts for a small proportion of the original image, important defect information is likely to be lost in pooling. The dynamic activation mapping pays attention to the intermediate layer information so that the lost information can be reused, thus helping to avoid the missed detection of small-scale targets.

2.3.6. Optimization Goal

In contrast to the base-level learning in ordinary supervised training, we proposed meta-level learning, which is dedicated to uncovering the commonalities that exist between multiple similar tasks.

Different tasks have their own adapted optimal function, while our strategy is done over the whole function space to get the common properties that all these functions follow. The objective function in this paper considers all training tasks $t\in {\mathcal{T}}_{train}$ and minimizes the sum of their loss functions on their respective test sets $D_{query}^t$, i.e.,

$$\min {\displaystyle \sum }_{t\in {\mathcal{T}}_{train}} {\displaystyle \sum }_{\left(x,y\right)\in D_{query}^t} L\left(y,f\left(x,D_{support}^t; \mathsf{\Theta }\right)\right)$$

(13)

where $\mathsf{\Theta }$ represents the meta parameter and $f\left(x,D_{support}^t; \mathsf{\Theta }\right)$ represent the predicted value for sample $x$.

In our approach, the meta parameter $\mathsf{\Theta }$ is not expressed in an explicit parametric way. Instead, the deep metric learning in 2.3.4 ensures that $\mathsf{\Theta }$ is implicitly incorporated into the model parameters.

3. Experiments and Discussion

3.1. Implementation Details

The experimental environment in the article is NVIDIA Tesla V100 32GB and the Linux operating system. With PyTorch, the code is implemented in python3.8, CUDA11.3. $n_{tasks}$ = 1000, $n_{train\_s}=10$, $n_{train\_q}=30$, $n_{val\_s}=10, n_{val\_q}=10, n_{val\_q}=80$, $epochs=100$, and $batchsize=128$. The uniform input size of the image is 224 × 224 × 3, and we used an Adam optimizer with a 1e-3 initial learning rate for optimization training.

3.2. The Wind Turbine Inspection Dataset

The data set used in this experiment comes from the UAV aerial wind turbine dataset provided by the Power Grid Company. To satisfy the few-shot condition, we have 20 samples for training and 2710 samples for test, i.e., $n_{wind\_s}=20, n_{wind\_q}=2710$. The data distribution is shown in

Table 1. Data distribution.

	Training Set	Testing Set
Normal	10	905
Defective	10	1805
Sum Up	20	2710

. Figure 8 contains examples of normal and defective aerial images.

3.3. Defect Visualization

Figure 9 shows some examples of defect visualization. It illustrates that our DMnet is capable of providing accurate identification for cracks, coating breakage, and corrosion in samples. Furthermore, even small target defects in a more difficult situation can be pinpointed.

We have selected a sample to show the dynamic recognition process under the proposed activation mapping strategy, as illustrated in Figure 10. We can see that in the early stage, the model is not yet good at targeting defects for recognition, so features in abruptly discordant locations are more likely to be noticed, such as edge lines and spindle connection of the blade. In the intermediate stage, some of the noise has been filtered out and edge lines are no longer mistaken for “defects”. The spindle connection, however, is more difficult to distinguish since it has a more similar form to defects. In the later stage, the defective parts have already been well focused. Overall, as training epochs increase, the strategy directs the model’s attention toward the parts that are more highly associated with the defect in the picture, ultimately helping the model to accurately identify and locate the defect.

3.4. Comparison with State-of-the-Art Methods

For a more objective evaluation, we use several indicators here, including accuracy (Acc), precision (Pre), recall (R), and F1-score. Specifically,

$$Acc=\frac{TN+TP}{FP+TN+TP+FN}$$

(14)

$$Pre=\frac{TP}{FP+TP}$$

(15)

$$R=\frac{TP}{TP+FN}$$

(16)

$$F_1=\frac{2\ast Pre\ast R}{Pre+R}$$

(17)

TP is the number of positive samples that are correctly distinguished. TN is the number of negative samples that are correctly distinguished. FP and FN, respectively, represent the negative and positive samples that were misclassified.

To verify the effectiveness of our method, we set up three groups of comparison models:

• Conventional machine learning algorithms with manual feature extraction: LBF for feature extraction and SVM for classification (one of the most widely used methods).
• Classical image classification algorithms based on deep learning: VGG, Res2net.
• Classical few-shot learning algorithms: MetaBaseline [35], RelationNet [36], Baseline-plus [37], and NegMargin [38].

Table 2. Comparative experiments.

	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
LBF + SVM	49.89	62.41	62.27	62.34
VGG-16	63.32	65.02	66.74	62.80
Res2net-50	62.25	60.84	61.48	62.87
MetaBaseline	69.82	67.58	69.24	67.85
RelationNet	69.96	66.34	66.52	66.42
Baseline-plus	73.76	71.73	73.86	72.13
NegMargin	73.25	71.09	73.06	71.49
DMnet	80.41	78.80	75.70	76.83

compares the model performance on the metrics. The recognition accuracy of the first group does not exceed 50%, which indicates that the manually extracted features are likely to be poorly adapted to WT surface defect detection. The second group showed an overall increase in performance, but the f1-score was essentially the same as the first group. Since models in the third group are designed specifically for the few-samples situation, they performed significantly better than the first two groups. Clearly, the proposed DMnet achieves the best scores on every metric. Compared to the second place, DMnet reflects an improvement ranging from 3% to 7% on each metric.

Two additional notes on the data for Table 2:

• Training vgg16 and Res2net from scratch with only 20 samples would be difficult to converge due to the unreliable empirical risk minimizer described in 2.1; thus, the results here are based on ordinary supervised learning in the context of pre-training weights with ImageNet [39].
• To keep the irrelevant variables consistent in the comparison, the embedding models for both DMnet and the four classical few-shot learning algorithms in Table 2 are VGG.

3.5. Ablation Experiments

To further validate the respective validity of the innovation points in this paper and disentangle the contributions made by each component, we set up two sets of ablation experiments, as depicted in

Table 3. The ablation experiments.

Embedding Model	Applied Strategy	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
VGG	baseline	63.32	65.02	66.74	62.80
	Baseline + approach 1	76.94	74.22	72.52	73.19
	Baseline + approach 2	73.91	70.65	68.24	69.02
	Baseline + approach 1 + approach 2	80.41	78.80	75.70	76.83
Res2net	baseline	62.25	60.84	61.48	62.87
	Baseline + approach 1	70.85	67.66	68.42	67.96
	Baseline + approach 2	66.46	67.21	69.31	65.74
	Baseline + approach 1 + approach 2	74.10	70.78	69.34	69.90

Note: Baseline: general supervised learning model (i.e., the second set of comparison models in Table 2). Approach 1: the three-stage construction introduced in Section 2.3.2, Section 2.3.3 and Section 2.3.4 Approach 2: the dynamic activation mapping strategy introduced in Section 2.3.5.

.

From the first four rows, the three-stage meta-learning framework provides a significant boost to the experimental results, enabling the model to achieve an increase of almost 14% and 11% in accuracy and F1-score, respectively. The dynamic activation mapping strategy also contributes further gains to the final results, with each metric improving by 3.5–4.5%. This demonstrates that every component of our method is practical and effective.

The last four rows were experimented under the condition that the embedding model was changed to res2net, and the results illustrate a similar pattern to the first four rows. With different embedding models, DMnet is always able to make great progress compared to baseline, which shows good generalizability and scalability.

4. Conclusions

In this paper, we propose a new few-shot WT surface defect detection framework—DMnet. To address the few-shot issue, a cross-task training strategy is designed. Searching for a set of generalized defect identification criterion, the hypothesis space shrinks to achieve rapid learning and fast adaptation to new tasks in a few-shot scenario. Moreover, a metric-learning-based classification module is developed to learn a general similarity measure that is transferable between tasks. Its non-parametric structure allows the network to adapt more quickly to matching (trained and tested) samples under the same task. Additionally, we introduce a new activation mapping strategy with a dynamic feedback session for training to improve the recognition accuracy for small targets.

To ensure a few-shot scenario, we specified to conduct experiments with a training set of only 20 wind turbine defect samples. Under the same experimental conditions, our method has the best defect recognition ability compared with conventional machine learning algorithms, classical deep-learning-based image classification algorithms, and classical few-shot learning algorithms. In particular, compared to the most commonly used deep learning methods, ours reflects improvements ranging from 9% to 14% in accuracy, precision, recall, and F1-score metrics. The ablation results illustrate that by chunking the model, it is confirmed that both the three-stage construction and the dynamic activation mapping strategy are valid components and each contributes to the gain. Further, the results present consistent patterns after replacing the base model, which indicates that DMnet can be adapted to different deep learning models, which is highly scalable.

In general, DMnet alleviates the information scarcity caused by insufficient defect samples by constructing a large number of similar tasks to obtain prior knowledge, which is applied to solve few-shot defect detection problems. Additionally, it provides the first dynamic interpretability within the network during training by formulating a feedback mechanism, which greatly improves the sensitivity of recognizing defects in small and concealed targets, and finally achieves high-precision universal defect detection, which is of great significance to the research of equipment diagnostics and condition monitoring.

Due to the data limitation, this paper only discusses whether the wind turbine is defective or not. In the next step, we consider subdividing the defect categories by collecting more samples of different defect types. Additionally, for the more difficult new tasks, cosine distance may not be effective enough to reflect the similarity between samples owing to its limitations as a predefined metric. The similarity measure that is dynamically updatable (e.g., a learnable network) may be a good choice for greater expressiveness.