Insulator Contamination Grade Recognition Using the Deep Learning of Color Information of Images

Abstract

To implement the non-contact detection of contamination on insulators, a contamination severity assessment methodology using the deep learning of the colored image information of insulators can be used. For the insulator images taken at the substation site, a mathematical morphology-improved optimal entropic threshold (OET) method is utilized to extract the insulator from the background. By performing feature calculations of insulator images in RGB and HSI color spaces, sixty-six color features are obtained. By fusing the features of the two color spaces using kernel principal component analysis (KPCA), fused features are obtained. The recognition of contamination grades is then accomplished with a deep belief network (DBN) that consists of a three-layered restricted Boltzmann machine. The experimental results of the images taken on-site show that the fused features obtained by the KPCA can fully reflect the contamination state of the insulators. Compared with the identification obtained using RGB or HSI color-space features alone, accuracy is significantly improved, and insulator contamination grades can be effectively identified. The research provides a new method for the accurate, efficient, and non-contact detection of insulator contamination grades.

1. Introduction

With the expansion of the power grid and increasingly serious environmental pollution, insulator contamination flashover has become the biggest threat to the safe and stable operation of the power system. It is a traditional measure for contamination flashover prevention to formulate a cleaning plan according to the distribution of contamination areas and to clean insulators regularly. This methodology can reduce the occurrence of contamination flashover to a certain extent and improve the reliability of the power grid. However, due to the lack of actual contamination grades for insulators, the formulation of cleaning plans is often inaccurate. Another disadvantage of this method is that due to the complexity of the climate and the environment, as well as on-site contamination accumulation, the cleaning effect is difficult to maintain for a long time, while contamination flashover caused by poor cleaning quality can also occur. In addition, the long power failure time and the risks of high-altitude operations also restrict extensive regular cleaning. Relying on regular cleaning to prevent contamination flashover requires high maintenance costs and significant human resources, which do not meet the requirements of an intelligent power grid. Advanced technical means can be used to monitor the contamination severity, flexibly arrange the cleaning work according to the contamination severity, and promote the transformation from planned maintenance to condition-based maintenance (CBM), which is the focus of contamination flashover prevention in the future. In order to accurately detect insulator contamination grades, researchers have carried out a lot of research. At present, the detection methods for assessing the contamination level of insulators mainly include evaluating the equivalent salt deposit density (ESDD), measuring the leakage current, using infrared temperature measurement methods, and so on [1,2,3,4,5,6,7,8]. The ESDD method is widely used today for contamination measurement. Since the acquisition of salt deposit density and non-soluble deposit density samples requires turning off the power, the process is complex and cannot facilitate efficient detection of contamination in the field. In the leakage current method, the leakage current on the surface of the insulator is analyzed to identify the contamination state or to issue a flashover warning. However, the leakage current is affected by the operating voltage, ambient temperature, humidity, type of insulator, and string length [9,10,11]. These issues have hindered the practical application of the leakage current method. In the infrared temperature measurement method, the heat generated by the leakage current is used to identify the degree of contamination [12]. An infrared imager is used to measure the surface temperature of the insulator, and the state of contamination is then identified by analyzing the temperature features. Since the surface temperature of an insulator in the field can be affected by several factors, including the heat dissipation condition, radiation from sunlight, and ambient temperature, many issues remain to be resolved for the infrared method. In theory, the detection of partial discharge intensity using ultraviolet images or ultrasonic signals can facilitate the monitoring of contamination severity. However, as the obvious discharge phenomenon can only be formed on the surface of contaminated insulator when the environmental humidity is high, the application of this method is restricted by environmental factors. Moreover, the relationship between partial discharge intensity, contamination grade, environmental humidity, and other factors is complex, and there are still many problems that need to be further studied. Compared to these methods, the method of identifying the contamination grade by analyzing a visible-light image is not affected by the ambient temperature and humidity, is cost effective, requires no power outage, and is non-contact. RGB and HSI are two widely used color space models for color images. For the RGB color model, red (R), green (G), and blue (B) components are utilized to represent color images [13,14]. for the HSI color model, hue (H), saturation (S), and intensity (I) components are utilized to represent color images. The two color models reflect the color differences of insulators with different contamination grades in the two color spaces. The comprehensive utilization of the information contained in the two color spaces can help detect the contamination state of insulators more accurately.

Compared to the conventional shallow learning method, a deep learning algorithm has a better capability of discovering and describing the complicated internal characteristics of a problem. Because it has a powerful learning ability, deep learning is able to obtain deep features from raw information directly [15]. Deep belief network (DBN) is a kind of representative deep network composed of restricted Boltzmann machines (RBMs). It has the capability of discovering deep structure information and learning complicated nonlinear features from data.

In this work, a method for identifying the contamination state using multimodal deep learning of image color information is proposed. The mathematical morphology-improved optimal entropic threshold (OET) segmentation method is used to extract information from the image of an insulator disk and eliminate the influence of background. Feature calculations of visible-light images of insulators in the field are carried out separately in the RGB and HSI color spaces, and the original features from the two color spaces are fused with kernel principal component analysis (KPCA) to obtain the fused features. Finally, classifiers are constructed using a DBN to realize the accurate recognition of insulator contamination grades using the fused features. Compared with image recognition algorithms such as convolutional neural network (CNN) and graph convolutional network(GCN), the scheme proposed in this paper uses surface color information to realize contamination grade recognition, avoid the influence of background, edge, structure, and other parameters independent of contamination severity, and realize a method that is more suitable for the research objectives and application scenarios. The process of insulator contamination grade recognition is shown in Figure 1.

2. Acquisition of Insulator Images

In multiple substations under the jurisdiction of the China Southern Power Grid in the Guangdong province of China, photographs were taken of the high-voltage-side insulating sleeves and low-voltage-side pillar insulators of the main transformer, as well as of simulated insulators in the station. The photographed insulators were all made of brown porcelain, and the acquisition and testing of salt deposit and non-soluble deposit samples followed the photography. For each insulator, the equivalent salt deposit density (ESDD) and non-soluble deposit density (NSDD) were determined, and then the contamination grade was determined according to the ESDD on the surface of the insulator. The photoshooting distance was approximately 1.5 m and the image of the insulator was zoomed to fill the entire frame to reduce background effects. Figure 2 shows that contamination is most severe on the upper plate surface of the pillar insulator. The contamination on the lower plate surface is much less and does not represent the overall contamination state. Therefore, the upper plate surface was chosen as the target for photography and identification. An angle of 45° between camera and pillar insulator worked well.

The appropriate camera angle range for photography is shown in Figure 3. Figure 4 shows insulators with contamination grades from I to IV. It can be seen that there are obvious differences in the color of insulators with different contamination grades. Using artificial intelligence (AI) to learn the color differences can facilitate the recognition of contamination grades.

3. Insulator Extraction in Visual Images

3.1. OET Segmentation

By analyzing the entropy of a gray image, the OET segmentation method can find the appropriate threshold that maximizes the amount of information of target and background. Assuming that the gray range of the image is $\left\{0,1,\cdots ,L-1\right\}$, pixels with a gray level lower than t constitute the target area O, pixels with a gray level higher than t constitute the background area B, while the entropy of the target region $H_O\left(t\right)$ and the background region $H_B\left(t\right)$ are defined as:

$$H_O\left(t\right)=-{\displaystyle \sum }_i\frac{p_i}{p_t}lg\frac{p_i}{p_t}$$

(1)

where $i=0,1,\cdots ,t$, $p_i$ is the probability of occurrence of the i-th gray scale.

$$H_B\left(t\right)=-{\displaystyle \sum }_i\frac{p_i}{1-p_t}lg\frac{p_i}{1-p_t}$$

(2)

where $i=t+1,t+2,\cdots ,L-1$. The entropy function is defined as:

$$\varphi \left(t\right)=H_O\left(t\right)+H_B\left(t\right)=lg{p}_t\left(1-p_t\right)+\frac{H_t}{p_t}+\frac{H_L-H_t}{1-p_t}$$

(3)

where $H_t=-{\displaystyle \sum }_i{p}_{i}lg{p}_i$,$i=0,1,\cdots ,t$ and $H_L=-{\displaystyle \sum }_i{p}_{i}lg{p}_i, i=0,1,\cdots ,L-1$.

In this paper, the color image is transformed into a gray image. The gray value that maximizes the entropy function $\varphi \left(t\right)$ is the segmentation threshold. The formula for converting a color image into a gray image is as follows

$$V_{Gray}\left(x,y\right)=0.299V_R\left(x,y\right)+0.587V_G\left(x,y\right)+0.114V_B\left(x,y\right)$$

(4)

where $V_{Gray}\left(x,y\right)$ is the gray value at the pixel $\left(x,y\right)$. $V_R\left(x,y\right)$, $V_G\left(x,y\right)$, $V_B\left(x,y\right)$ are the values of R, G and B respectively.

3.2. Mathematical Morphological Processing

Mathematical morphology makes use of operators from set theory to analyze images. The morphological operator defines the local transformation, and the pixel values to be expressed are regarded as a set. The basic operations of mathematical morphology include expansion and corrosion, and their different combinations constitute opening and closing operations.

Assume that B is a structural element consisting of 0 and 1, which defines the size and shape of morphological operations. A is an image, and the expansion operation of A with structural element B is defined as follows:

$$D\left(A\right)=A\oplus B=\left\{\left(x,y\right)|B_{xy}{\displaystyle \cap }A\ne \varnothing \right\}$$

(5)

Equation (5) shows that when the origin of B moves to point $\left(x,y\right)$, the intersection of B and A is not empty, then point $\left(x,y\right)$ belongs to the expanded binary image $D\left(A\right)$. It can be seen that expansion will cause the original image to expand. Corrosion is the dual operation of expansion, which is defined as follows:

$$E\left(A\right)=A\ominus B=\left\{\left(x,y\right)|B_{xy}\subseteq A\right\}$$

(6)

Equation (6) shows that when the origin of B moves to point $\left(x,y\right)$, all points in B belong to A, then point $\left(x,y\right)$ belongs to the binary image $E\left(A\right)$ after corrosion. It can be seen that corrosion will cause the original image to shrink. The combination of expansion and corrosion can realize opening and closing operations, and can carry out more complicated image processing. The opening operation of $A$ using structural element $B$ is defined as follows:

$$A·B=\left(A\ominus B\right)\oplus B$$

(7)

The above formula shows that the process of opening A with structural element $B$ means using the same structural element to corrode the image first and then expand it. The closing operation of A using structural element $B$ is defined as follows:

$$A·B=\left(A\oplus B\right)\ominus B$$

(8)

The above formula shows that the process of closing $A$ with structural element $B$ means using the same structural element to expand the image first and then corrode it.

The expansion operation expands the image boundary, and the corrosion operation shrinks the image boundary. The opening operation can break the narrow discontinuity and eliminate the fine protrusion, and the closing operation can bridge the narrow discontinuity and slender gap, eliminate the small hole, and fill the fracture in the contour line.

After OET segmentation, a 10 × 10 square structure element is utilized to close the image for the elimination of the holes on the insulator disc surface. Using a grade Ⅲ insulatoras an example, the result of disk surface extraction is shown in Figure 5.

In Figure 5b, the segmented background is replaced with a black region. It can be seen that the insulator is extracted completely, and only a small part of the reflective area is segmented as background. The follow-up research focuses on the disk area extracted by the proposed segmentation method above.

4. Theory of KPCA and DBN

4.1. KPCA Feature Extraction

KPCA maps the input vector X to a high-dimensional feature space F through nonlinear mapping so that the input vector has better separability, and then conducts a linear principal component analysis on the mapped data in space $F$ to obtain the nonlinear principal component.

Assume that the size of the original characteristic parameter matrix X is M × N, where M is the dimension of feature parameters and N is the number of samples. The principal components extracted from the eigenvectors corresponding to several features with a large covariance matrix are used as new features. For the original parameter matrix X, its row vector is expressed as $x_1,x_2,\cdots ,x_m$, then $X={\left(x_1,x_2,\cdots ,x_m\right)}^T$. Through nonlinear mapping $\phi$, the original space $R$ is mapped to the feature space F and the image of $x_i$ in F is $\phi \left(x_i\right)$. Assuming that the image data is zero mean, the covariance matrix of $\phi \left(X\right)$ is:

$$\overline{C}=\frac{1}{n}{\displaystyle \sum }_{i=1}^M\phi \left(x_i\right)\phi {\left(x_i\right)}^T$$

(9)

For matrix $\overline{C}$, its eigenvalue is $\lambda$, the eigenvector is V, and $\lambda V=\overline{C}V$. The inner product of each sample and the formula can be obtained as:

$$\lambda \left[\phi \left(x_k\right)·V\right]=\left[\phi \left(x_k\right)·\overline{C}V\right]\hspace{1em}k=1,\cdots ,M$$

(10)

The eigenvector of $\overline{C}$ can be expressed linearly as:

$$V={\displaystyle \sum }_{j=1}^M{\alpha }_j\phi \left(x_j\right)$$

(11)

where ${\alpha }_j$ is the correlation coefficient. Substituting Equation (11) into Equation (10), it can be obtained that:

$$\lambda {\displaystyle \sum }_{j=1}^M{\alpha }_j\left[\phi \left(x_k\right)·\phi \left(x_j\right)\right]=\frac{1}{M}{\displaystyle \sum }_{j=1}^M{\alpha }_j[\phi \left(x_k\right)·{\displaystyle \sum }_{j=1}^M\phi \left(x_i\right)]\left[\phi \left(x_i\right)·\phi \left(x_j\right)\right]$$

(12)

Defining a symmetric matrix K with size of M × N:

$$K_{ij}=\left[\phi \left(x_i\right)·\phi \left(x_j\right)\right]$$

(13)

Equation (12) can be expressed as $M\lambda K\alpha =KK\alpha$. It can be simplified as:

$$M\lambda \alpha =K\alpha$$

(14)

In combination with Equation (11), V can be derived from α, the eigenvector of matrix K, and the principal component direction of the mapping space can also be obtained. Matrix K can be determined by the selection of the kernel function.

Matrix K is diagonalized, and its eigenvalues are represented by ${\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_N$ while the corresponding eigenvectors are represented by ${\alpha }^{\left(1\right)},{\alpha }^{\left(2\right)},\cdots ,{\alpha }^{\left(N\right)}$. Assume that ${\lambda }_p$ is the first non-zero eigenvalue, and the standardization of ${\alpha }^{\left(p\right)},\cdots ,{\alpha }^{\left(N\right)}$ is realized by the normalization of V. Supposing that:

$$V^{\left(k\right)}{V}^{\left(k\right)}=1\hspace{1em}k=p,\cdots ,N$$

(15)

The following equation can be obtained by substituting Equation (11) into the above equation:

$${\displaystyle \sum }_{i,j=1}^M{\alpha }_i^{\left(k\right)}{\alpha }_j^{\left(k\right)}\left[\phi \left(x_i\right)·\phi \left(x_j\right)\right]={\displaystyle \sum }_{i,j=1}^M{\alpha }_i^{\left(k\right)}{\alpha }_j^{\left(k\right)}{K}_{ij}={\alpha }^{\left(k\right)}K{\alpha }^{\left(k\right)}={\lambda }_k\left({\alpha }^{\left(k\right)}{\alpha }^{\left(k\right)}\right)=1$$

(16)

In order to extract the principal component features, the projection of the mapping data on the eigenvector $V^{\left(k\right)}$ is calculated as:

$$[V^{\left(k\right)}·\phi \left(x\right)]={\displaystyle \sum }_{i=1}^M{\alpha }_i^{\left(k\right)}\left[\phi \left(x_i\right)·\phi \left(x\right)\right]$$

(17)

This projection is the nonlinear principal component of X obtained by nonlinear mapping $\phi$. The above algorithm is derived on the assumption that the mapping data is zero mean. In fact, this assumption is usually not established. Therefore, it is necessary to centralize the mapping data. Substitute K in Equation (14) with $\overline{K}$ as follows:

$$\overline{K}=K-I_{M}K-K{I}_M+I_{M}K{I}_M$$

(18)

where I_M is a M × N identity matrix with a coefficient of 1/M. The eigenvalue formula is:

$$\overline{\lambda }\overline{\alpha }=\overline{K}\overline{\alpha }$$

(19)

The principal component of the kernel function is:

$$\left[{\overline{V}}^{\left(k\right)}·\phi \left(x\right)\right]={\displaystyle \sum }_{i=1}^M{\overline{\alpha }}_i^{\left(k\right)}\left[\overline{\phi }\left(x_i\right)·\overline{\phi }\left(x\right)\right]={\displaystyle \sum }_{i=1}^M{\overline{\alpha }}_i^{\left(k\right)}\overline{K}\left(x_i,x\right)$$

(20)

When the nonlinear principal element is obtained by Equation (20), the inner product calculation is required. In practice, the kernel function is utilized to replace the inner production. The Gaussian radial basis function has the best classification effect, and its formula is:

$$K\left(x,y\right)=e^{\left[-\frac{\parallel x-y{\parallel }^2}{2{\sigma }^2}\right]}$$

(21)

where the value of $\sigma$ is selected as 1 in this study.

4.2. Deep Belief Network

The fused features obtained from the KPCA are utilized for the training of a DBN to achieve the recognition of contamination grades. Figure 6 shows a DBN model consisting of three restricted Boltzmann machines (RBMs). In the procedure of unsupervised layer-wise learning, the output of the lower RBM is passed up layer by layer as the input of the higher RBM. The results are passed layer by layer to form a feature representation that is more abstract and has characterization capabilities at the upper layer rather than at the lower layer. Then, on the basis of increasing the classifier, the recognition ability of the DBN is optimized through supervised back propagation fine-tuning.

(1)

Forward unsupervised layer-wise learning

Each node in the RBM has two states: activated (represented by 1) and inactive (represented by 0). Assuming that the RBM has $n$ input nodes, represented by $v=\left(v_1,v_2,\cdots ,v_n\right)$, and that there are m hidden nodes, represented by $h=\left(h_1,h_2,\cdots ,h_m\right)$, the following energy function may then be defined:

$$E\left(v,h;\theta \right)=-{\displaystyle \sum }_{i=1}^n{a}_i{v}_i-{\displaystyle \sum }_{j=1}^m{b}_j{h}_j-{\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j=1}^m{v}_i{w}_{ij}{h}_j$$

(22)

where $v_i$ represents the state of the i-th input node, $h_j$ represents the state of the j-th hidden node, $\theta =\left(w_{ij},a_i,b_j\right)$ represents the parameter value when training the RBM, $w_{ij}$ is the connection weight between the i-th input node and the j-th hidden node, and $a_i$ and $b_j$ are the offset values of the i-th input node and the j-th hidden node. According to this energy function, the joint probability distribution of $\left(v,h\right)$ can be given as:

$$P\left(v,h;\theta \right)=\frac{1}{Z\left(\theta \right)}{e}^{-E\left(v,h;\theta \right)}$$

(23)

Here are the conditional probabilities of the input layer and the hidden layer:

$$P\left(v|h;\theta \right)=\frac{P\left(v,h;\theta \right)}{P\left(h;\theta \right)}={{\displaystyle \prod }}_{i}P\left(v_i|h;\theta \right)$$

(24)

$$P\left(h|v;\theta \right)=\frac{P\left(v,h;\theta \right)}{P\left(v;\theta \right)}={{\displaystyle \prod }}_{j}P\left(h_j|v;\theta \right)$$

(25)

Since the nodes are independent and there is no connection within a layer, the activation probabilities are deduced as:

$$P\left(v_i=1|h;\theta \right)=\frac{1}{\left[1+e^{\left(-a_i-{{\displaystyle \sum }}_j{w}_{ij}{h}_j\right)}\right]}$$

(26)

$$P\left(h_j=1|v;\theta \right)=\frac{1}{\left[1+e^{\left(-b_j-{{\displaystyle \sum }}_i{v}_i{w}_{ij}\right)}\right]}$$

(27)

where $\theta$ can be obtained by maximizing the log-likelihood function, that is:

$$L\left(\theta ;v\right)={{\displaystyle \prod }}_{v}L\left(\theta |v\right)={{\displaystyle \prod }}_{v}P\left(v\right)$$

(28)

The maximum value of $L\left(\theta \right)$ is usually obtained using the ascending gradient method, however, due to the normalizing factor $Z\left(\theta \right)$, the direct calculation of this is difficult.

A fast calculation method of contrastive divergence (CD) was proposed by Hinton to obtain $\theta$ [16]. Using Equations (26)–(28), as well as the CD calculation method, the update rule for $\theta$ can be obtained:

$$\{\begin{array}{c}{w}_{ij}^{k+1}=w_{ij}^k+\epsilon \left(\langle v_i{h}_j\rangle {}_{data}-\langle v_i{h}_j\rangle {}_{recon}\right)\\ a_i^{k+1}=a_i^k+\epsilon \left(\langle v_i\rangle {}_{data}-\langle v_i\rangle {}_{recon}\right)\\ b_j^{k+1}=b_j^k+\epsilon \left(\langle h_j\rangle {}_{data}-\langle h_j\rangle {}_{recon}\right)\end{array}$$

(29)

where ε represents the learning ratio, ${\langle ·\rangle }_{data}$ represents the expectation defined by the observed data, ${\langle ·\rangle }_{recon}$ represents the expectation defined by the refactored data.

(2)

Backward supervised fine-tuning

The parameters gained by forward learning were taken as the initial values of backward supervised fine-tuning. The fine-tuning optimization starts with the last layer and then fine-tunes the parameters of the lower layers using the labeled data. A softmax logistic regression model was utilized as the classifier in the last layer. For a DBN consisting of a one-layer RBM, where $x$ is the initial input data, then the output vector of the last layer is $u^l\left(x\right)$:

$$u^l\left(x\right)=\frac{1}{1+e^{\left[a^l+w^l{u}^{l-1}\left(x\right)\right]}}$$

(30)

After l layers of forward unsupervised RBM learning, the probability that the i-th sample belongs to category y_i, where $y_i\in \left(1,2,\cdots ,c\right)$ is:

$$P\left(y_i=k|u^l\left(x_i\right),V^l,c^l\right)=\frac{e^{\left[V_k^l{u}^l\left(x_i\right)+c^l\right]}}{{{\displaystyle \sum }}_{k=1}^c{e}^{\left[V_k^l{u}^l\left(x_i\right)+c^l\right]}}$$

(31)

where V represents the parameter coefficient. The category with the maximum probability is selected as the final determined category of the softmax classifier. The error function of the l-th layer is expressed as:

$$J\left({\lambda }^l\right)=-\frac{1}{m}\left[{\displaystyle \sum }_{i=1}^m{\displaystyle \sum }_{k=1}^{c}1\left\{y_i=k\right\}log \rho \right]$$

(32)

where $\rho$ represents the weight decay ratio, ${\lambda }^l=\left\{w^l,a^l,c^l,V^l\right\}$, and $1\left\{y_i=k\right\}$ is the logistic indicator function that assumes the value of 1 when $y_i=k$; otherwise, its value is 0:

$${\nabla }_{{\lambda }^l}J\left({\lambda }^l\right)=-\frac{1}{m}{\displaystyle \sum }_{i=1}^m\left[u^l\left({\widehat{x}}_i\right)\left(1\left\{y_i=k\right\}-h^l\left({\widehat{x}}_i\right)\right)\right]$$

(33)

The fine-tuning parameter is:

$$\widetilde{{\lambda }^l}={\lambda }^l-\epsilon {\nabla }_{\lambda }J\left({\lambda }^l\right)$$

(34)

where $\epsilon$ represents the learning ratio. Proceeding toward lower layers in an analogous fashion, the relevant parameters are fine-tuned until the initial layer is reached.

5. Recognition of the Contamination Grade

After the photographing, the contamination sampling and testing of insulators are carried out. The on-site photographing and contamination sampling are shown in Figure 7. In the laboratory, the contamination grade test is conducted with reference to GB/T 16434-1996 to determine the contamination grade of each insulator. The standard for the insulator contamination grades is shown in

Table 1. Standard for theinsulator contamination grades.

	Contamination Grade
	0	I	II	III	IV
ESDD (mg/cm2)	≤0.03	0.03~0.06	0.06~0.10	0.10~0.25	>0.25

.

In the contamination test, no grade 0 samples were found in the sampled insulators, therefore the follow-up research only focuses on the four contamination grades of I to IV.

For the insulator image taken on site, the mathematical morphology-improved OET segmentation method is used to extract the insulator from the background, and the extracted insulator area is selected as the region of analysis. Sixty-six features of the region—namely, the mean, median, maximum, minimum, common, range, variance, skewness, kurtosis, energy, and entropy of the six color components of R, G, B, H, S, and I—are used as the original feature quantities. The calculation formulas for several of these features are:

Mean $r_{aver}$:

$$r_{aver}={\displaystyle \sum }_{i}t\left(i\right)p_t\left(i\right)$$

(35)

Variance $r_{vari}$:

$$r_{vari}={\displaystyle \sum }_i{\left[t\left(i\right)-r_{aver}\right]}^2{p}_t\left(i\right)$$

(36)

Skewness $r_{skew}$:

$$r_{skew}=\frac{{{\displaystyle \sum }}_i{\left[t\left(i\right)-r_{aver}\right]}^3{p}_t\left(i\right)}{\sqrt[3]{r_{vari}}}$$

(37)

Kurtosis $r_{kurt}$:

$$r_{kurt}=\frac{{{\displaystyle \sum }}_i{\left[t\left(i\right)-r_{aver}\right]}^4{p}_t\left(i\right)}{r_{vari}^2}$$

(38)

Energy $r_{ener}$:

$$r_{ener}={\displaystyle \sum }_i{t}^2\left(i\right)p_t\left(i\right)$$

(39)

Entropy $r_{entr}$:

$$r_{entr}={\displaystyle \sum }_i-p_t\left(i\right)lg\left(p_t\left(i\right)\right)$$

(40)

where $t\left(i\right)$ is the value of a color component in the insulator disk area, $p_t\left(i\right)$ is its distribution probability, and ${\displaystyle \sum }_i{p}_t\left(i\right)=1$. In order to improve the comparability of data and the operation speed of the classifier, each group of features needs to be normalized. The normalization formula is as follows:

$$\overline{r_i^{\left(k\right)}}=\frac{r_i^{\left(k\right)}-r_{min}^{\left(k\right)}}{r_{max}^{\left(k\right)}-r_{min}^{\left(k\right)}}$$

(41)

where $\overline{r_i^{\left(k\right)}}$ is the normalized value of the feature, $r_i^{\left(k\right)}$ is the original value of the feature, $k=1,\cdots ,11$ is the number of features, $i$ is the sample serial number, and $r_{min}^{\left(k\right)}$ and $r_{max}^{\left(k\right)}$ are the minimum and maximum values of the k-th feature, respectively.

Feature extraction is then performed on the 66 original features of each image using KPCA. The DBN is trained by the fused features from the KPCA to create a recognition model for the insulator contamination grades. The proposed method is validated using test samples. For the KPCA, the dimension of the input feature is 66 and the dimension of the output fused feature is 20. For the DBN, the input node number (i.e., the input feature dimension) is 20. The node numbers of the three hidden layers in the DBN are set to 40, 20, 10, respectively. The output node number is set to 4. For the remaining parameters, the learning ratio $\epsilon$ and the weight decay ratio $\rho$ are set to 0.1 and 0.0001, respectively. For the output state of the DBN, [0,0,0,1], [0,0,1,0], [0,1,0,0], and [1,0,0,0] represent the contamination grade from I to IV, respectively. For each contamination grade, 120 images are utilized as training samples for the training of the DBN. For each contamination grade, 40 images are utilized as test samples, and the trained classifier is used to identify them.

To verify the improvement of the contamination grade recognition derived from information fusion, contamination grade recognition was performed using the proposed algorithm on the 33 features of the R, G, and B color components and the 33 features of the H, S, and I color components of the same set of samples. For the KPCA, the dimension of the input feature is 33 and the dimension of the output fused feature is 10. For the DBN, the input node number (i.e., the input feature dimension) is 10. The node numbers of three hidden layers are 40, 20, and 10, respectively. The output node number is set to 4. All the other parameter values are the same as given previously. The recognition results are shown in

Table 2. Results of contamination grade recognition.

Contamination Grade	Number of Samples	Number of Correct Recognitions
		RGB	HSI	Fusion
I	40	34	34	39
II	40	32	33	38
III	40	31	34	38
IV	40	33	35	38
Accuracy Rate	------	81.25%	85%	95.625%

.

Results show that the accuracy of contamination grade recognition using the color features of RGB and HSI alone are 81.25% and 85%, respectively. In contrast, the recognition accuracy using the fused features obtained by the KPCA has been significantly improved, reaching 95.625%. Compared with the RGB and HSI color-space features before fusion, the fused feature makes full use of the information of the two color spaces to more comprehensively reflect the contamination state of insulators. It has better separability and has significantly improved the accuracy of contamination grade recognition. Figure 8 shows the confusion matrices of the contamination grade recognition.

In order to verify the improvement of the recognition accuracy of the deep learning method compared with the shallow learning method, classifiers were constructed using BP and SVM to realize contamination grade recognition. The input of the two classifiers is consistent with the above DBN, which is a 20-dimensional fused feature extracted by the KPCA. The structure of the BP network is 20-16-4 and the learning rate is set to 0.1. The kernel function of the SVM adopts RBF and the penalty coefficient is set to 10. The training samples and test samples are consistent with the samples used by the DBN. After testing, the recognition accuracy of BP and SVM are 80.625% and 83.75% respectively. It can be seen that deep learning can make better use of feature information, and the recognition accuracy of the DBN is obviously better than the two classical shallow learning methods.

In order to study the impact of each feature on the performance of the algorithm, ablation analysis is used to remove one input feature at a time from the best performance, observe the change of the accuracy of the algorithm, and determine the features that have greater impacts on the performance. Figure 9 shows the recognition accuracy after removing each RGB color feature. The ablation analysis diagrams of HSI color features are shown in Figure 10.

It can be seen from Figure 9 and Figure 10 that 11 features, such as R-mean, R-maximum, G-minimum, G-skewness, B-mean, B-common, H-mean, H-median, H-common, H-skewness and I-skewness, have a greater impact on the performance of the algorithm, and the accuracy reduction caused by removing any feature alone is more than 10%.

The parameters of the computer used in this study are i5-9500 CPU, 3 GHz basic frequency and 16GB RAM. Running in the MATLAB 2019 environment, the disk segmentation of 480 test images takes 104.4 s, the feature calculation and feature fusion takes 115.2 s, the DBN network training takes 585.6 s, and the whole training process takes 805.2 s in total. The algorithm performs well in terms of time complexity. In the follow-up research, the operation time can be further reduced by using a commercial algorithm framework or hardware platform.

The current research is focused on brown porcelain external insulation. In the future, research can be carried out on white porcelain insulators, glass insulators, and composite insulators to build contamination grade classifiers for different materials and types of insulators. Furthermore, after further improvement, the contamination grade recognition method proposed in this paper can be combined with inspection platforms, such as robots and UAVs, to realize remote and unmanned detection of insulator contamination status.

6. Conclusions

In order to obtain accurate information about the contamination grade of insulators, a methodology to fuse RGB and HSI color-space information using a deep-learning method to achieve contamination grade recognition is proposed in this paper. The mathematical morphology-improved OET segmentation is used to extract the insulator disk region from the image taken in the field. A total of sixty-six color features of insulator disk area are calculated in the RGB and HSI color spaces. These color features are then fused using KPCA. After that, the fused features are utilized to train a DBN for the recognition of the contamination grade. The proposed method is verified using images taken at the substation site. The results show that the precise recognition of the contamination grade can be achieved by utilizing the fused color-space features and a DBN. Compared with recognition using RGB or HSI color features alone, the fused feature has significantly improved the identification accuracy by making full use of the information of the two color spaces. This research provides a method with high accuracy and reliability for the non-contact measurement of insulator contamination grades.