Recognition Method of Digital Meter Readings in Substation Based on Connected Domain Analysis Algorithm

Abstract

Aiming at the problem that the number and decimal point of digital instruments in substations are prone to misdetection and missed detection, a method of digital meter readings in a substation based on connected domain analysis algorithm is proposed. This method uses Faster R-CNN (Faster Region Convolutional Neural Network) as a positioning network to localize the dial area, and after acquiring the partial image, it enhances the useful information of the digital area. YOLOv4 (You Only Look Once) convolutional neural network is used as the detector to detect the digital area. The purpose is to distinguish the numbers and obtain the digital area that may contain a decimal point or no decimal point at the tail. Combined with the connected domain analysis algorithm, the difference between the number of connected domain categories and the area ratio of the digital area is analyzed, and the judgment of the decimal point is realized. The method reduces the problem of mutual interference among categories when detecting YOLOv4. The experimental results show that the method improves the detection accuracy of the algorithm.

1. Introduction

Substations are in an indispensable and important position in the power system, and they are responsible for the major tasks of converting and distributing electrical energy. In recent years, the safe and normal operation of substations has also become a key issue that needs attention [1]. Much equipment in substations reflects their actual operation through meter readings, and testing whether the equipment is operating normally can be transformed into monitoring its meter readings.

The traditional substation inspection is mainly based on manual inspection and recording, and the inspection efficiency is low [2]. Due to the large number of meters, the complex external environment and the reliance on the original equipment of the substation, real-time performance and data reliability cannot be effectively guaranteed.

In the actual scene of the substation, the decimal point of the digital instrument is often found to be less than 32 × 32 pixels during the data collection process. It means that the decimal point belongs to the category of small targets. Therefore, it is prone to miss detection and false detection. At the same time, the existence of small targets tends to cause intercategory interference to other targets, making it difficult to accurately obtain image features, and affecting the accuracy of meter reading recognition. This poses a major challenge for accurately identifying digital meter readings in substations. On the basis of satisfying the real-time detection of the substation, improving the accuracy of digital instrument recognition as much as possible is a key issue that needs to be solved.

1.1. Motivation

With the development of science and technology in recent years, in order to minimize labor costs and reduce the adverse effects of environmental factors on manual operations, the realization of unattended [3] substation inspection has also become one of the focus of substation work. Most of the indoor equipment in large substations are equipped with meters.

Most of the indoor equipment in large substations are equipped with meters. Presently, the mainstream method used by researchers is to collect image data from digital meters in various substations, process the image data through a computer, extract image features, convert the data into corresponding text forms, and finally realize the purpose of readings information for collection and storage. The core of this method lies in the construction of image processing and recognition algorithms. The difficulty is that the method needs to have good real-time, accuracy and robustness to the background [4]. The thesis focuses on improving the detection accuracy in the substation scenario.

1.2. Contributions

Our contribution in this article is as follows:

• Propose a digital meter readings detection algorithm based on Faster R-CNN (Faster Region Convolutional Neural Network), YOLOv4 (You Only Look Once) convolutional neural network and connected domain analysis.
• Contrast with the direct use of deep learning convolution neural network for readings recognition.
• Propose a data expansion method for substation digital instrument data.

1.3. Organization

The rest of this article is organized as follows. Section 2 investigates the related research work. In Section 3, we describe a digital readings recognition method based on connected domain research domain analysis. In Section 4 and Section 5, we made comparative experiments and obtained the results.

2. Related Work

At present, there are detection algorithms for instrument detection, such as template matching method, threading method, five-segment feature extraction method, and image recognition method based on deep learning [5,6,7,8,9]. The template matching method establishes the template of each number from 0 to 9. Then the algorithm denoises the digital image and the template image, binarization and normalization into the same size difference, and takes the number with the smallest difference as the corresponding number. The process is simple, but different templates need to be designed for different numbers, and the robustness is poor. The threading method determines the number by equally dividing the area of the digital dial and scanning the form of pixels. This method is not suitable for dials with decimal points and is subject to environmental interference factors. Huang et al. [10] proposed a digital meter readings recognition algorithm based on a sample matching algorithm. This algorithm is simple and easy to implement. However, in the complex environment of substation instruments, it is necessary to construct a sample with the same image and stable structure. The robustness of this method is poor, so it is difficult to meet the needs of the project. Chen et al. [11] proposed an algorithm for rapid recognition of digital meters based on feature detection. Based on the seven-segment feature extraction method for digital tube, the algorithm adds intersecting lines to detect special positions and determine specific numbers. This method can determine whether there is a decimal point, but it needs an appropriate shooting angle, and there is a high false detection rate in the actual scene of the substation. Liu et al. [12] used the deep learning CNN (Convolutional Neural Network) model to operate on the entire line of text. The accuracy is higher in the synthetic data set, but the accuracy rate drops significantly in the real scene. The main reason is that the synthetic sample has a gap with the real object.

In view of the problems that the decimal point of digital instrument in the actual scene of substation is prone to false detection and leak detection, and the accuracy of number identification is not high. The paper innovatively combines Faster R-CNN [13,14,15] and YOLOv4 [16] target detection algorithms with connected domain analysis methods. A target detector was formed, and experiments were carried out on the image data actually collected by the substation and compared with the detector composed of Faster R-CNN and YOLOv4. Using Faster R-CNN positioning [17] accuracy, YOLOv4 accuracy and real-time balance characteristics. This algorithm can remove the interference information in the image data, judge the number category and determines whether the decimal point is included by analyzing the attributes of the connected domain [18,19] in the image results. Experiments show that the detector combined with the connected domain analysis method can effectively solve the problem of mutual interference between categories in the neural network, therefore improving the detection performance.

3. Digital Instrument Connected Domain Analysis Algorithm

Usually, the meter recognition algorithm generally includes two steps, positioning and detection. The image components containing the dial region are located and obtained and the image of the target region is segmented [20]. In order to accurately extract features and improve the performance of the training model, the acquired image components are enhanced by the localization algorithm. The detection algorithm classifies and regresses the detection object, and returns the required category and position coordinates.

To solve the problem of whether the detection of decimal point is misdetected or missed in digital instrument, this paper adopts Faster R-CNN as the positioning network and YOLOv4 as the target detection network for morphological analysis [21]. It is found that the number with a decimal point at the end and the number without a decimal point at the end have greater intercategory interference. The study found that there is greater interference between the numbers with a decimal point in the tail and the numbers without a decimal point in the tail. Combined with the method of analyzing connected regions in image processing [22], a digital meter readings recognition based on connected domain analysis algorithm is proposed. Method, by comparing the area ratio of connected domains, reducing the mutual interference of categories in the detection network and improving the detection performance. Figure 1 is the overall framework of digital instrument readings recognition.

3.1. Faster R-CNN Regional Positioning

In 2015, Kaiming He [13] and others proposed the Faster R-CNN algorithm. So far, this method is still one of the most accurate algorithms [23], using a small RPN (Region Proposal Network) to generate the candidates area. The method improves the processing speed of the image. Digital region, as a key component, can be localized to effectively reduce the influence of background [24] and reduce the scale of target detection and search [15]. In order to ensure the accuracy of the positioning area, the Faster R-CNN network is used to locate the required digital meter area. The image is input into the feature extraction network [25] to obtain the feature map. The paper uses Resnet50 [25] as the backbone, and uses the RPN structure to generate candidate frames, and the candidate frames generated by RPN Projected to the feature map to obtain the feature matrix, each feature matrix is scaled to a 7 × 7 size feature map through the ROI (Region of interest) pooling layer, and the feature map is flattened and the prediction result is obtained through the fully connected layer [26]. Faster R-CNN regional positioning is shown in Figure 2.

3.2. Dial Area Readings Detection

After obtaining the dial area through the Faster R-CNN positioning network [27], the readings detection is performed on the image of the dial area. Divide the dial area readings detection algorithm into two parts. In the first part, the YOLOv4 algorithm does not distinguish whether the tail contains a decimal point in the readings detection of the dial area. If the number contained in the region is i (i is a certain number between 0 and 9), the number (or the decimal point contained in the number and the tail) is classified as one class, to reduce the influence of the decimal point on the training process and reduce the interference between categories in the YOLO detection network. The second part is the connected domain analysis method of digital area based on seed filling method. It analyzes the digital area truncated from the YOLOv4 network and analyzes the attributes of the connected domain to determine whether the number tail contains a decimal point. Dial area readings detection YOLOv4 algorithm is shown in Figure 3.

3.2.1. Data Preprocessing and Data Expansion

Before entering the YOLOv4 detection network, it is necessary to perform data expansion operations on the image of the dial area to avoid the problem of model training caused by imbalance in the input data. Commonly used image expansion methods include image sharpening, adding Gaussian noise, and contrast enhancement. In order to avoid the influence of the image shooting angle in the process of collecting data, image enhancement measures that perform perspective transformation on the image are adopted.

3.2.2. YOLOv4 Detection Network

YOLO is a classification algorithm that uses regression networks to achieve target detection. Compared with the traditional network based on area candidates, the YOLO network directly uses the target detection task as a regression task to ensure the detection speed and ensure the real-time detection [28,29]. The YOLOv4 detection network introduced the architecture of CSP (Cross-Stage-Partial-Connections) [16], built CSPDarknet53 [8] as the backbone feature extraction network, and pooled the output of CSPDarknet53 through the SPP (Spatial Pyramid Pooling) [30,31]. The method enhances the receptive field of the network. As an additional module of Neck, SPP uses multi-scale pooling to effectively improve the detection accuracy of the training model, and uses a series of up-sampling and down-sampling of PANet (Path Aggregation Network) to fuse the features of each target scale. Compared with YOLO [32,33,34], YOLOv2 [35], YOLOv3 [36,37], YOLOv4 greatly improves the detection accuracy of the model while ensuring speed, and has the highest accuracy among all current real-time target detection algorithms [38], the map on the coco dataset is 43.5%. YOLOv4 network structure is shown in Figure 4.

3.2.3. Connected Domain Analysis Method of Digital Area Based on Seed Filling Method

In the image processing, the domain [39] can be added to the mark for processing. The thesis proposes an analysis and judgment algorithm to solve the problem of whether there is a decimal point in the digital area of the dial. A single digital area may appear without a decimal point or with a decimal point. The connected domain attribute of the digital area can be classified as the criterion. Connected area analysis is mainly for the image after binarization. First, the connected area is marked on the binarized image by the seed filling method. The main steps are as follows:

(1)

Scan the image until the pixel point value A (x, y) is 1;

1)

Use A as a seed, mark it as a label, and then push the same pixel value in the adjacent pixel with the value of A point 4 into the stack;

2)

Pop the top pixel of the stack, mark it with the same label, and then push the same pixel value in the adjacent pixel of the top pixel 4 into the stack;

3)

Repeat step 2) until the stack is empty, i.e., obtain a connected area, which is marked with a label;

(2)

Repeat step (1) until the end of the scan.

After marking the connected domain, the category and size of the connected domain are analyzed and the number of categories of each connected domain is calculated. Then the area of the connected domain is calculated in pixels. Finally, the number of categories and the area of the connected domain are used to judge whether the decimal point is included. The specific judgment strategy is: if the number of categories is 1, it is judged that the number area does not contain a decimal point. When the area of two or more connected domains is divided, if it is greater than a certain threshold value, it is judged that there is a decimal point in the digital region. If less than the threshold, the numeric region does not contain a decimal point. The algorithm flow chart of digital region connectivity domain is shown in Figure 5.

4. Experiment

4.1. Experimental Environment

The experiment platform is Windows10 operating system, the software used is CUDA10.0, cuDNN7.4.1.5, the experiment is carried out under the deep learning framework based on Pytorch1.2, and the computer configuration is shown in

Table 1. Experimental Hardware Configuration.

Name	Type
CPU	Intel Core I7-8700
GPU	Nvidia RTX 2080
Hard Disk	4T SSD
Memory	32 G

.

4.2. Experimental Data Set

This experimental data set comes from the daily inspection collection of Guyang Xingshun West Wind Farm Substation. This data set belongs to the engineering application data set in a specific scenario, involving the company’s related intellectual property rights. It is a non-public data set. In the Faster R-CNN positioning network, the number of positioning data sets is 530 when it is not expanded, the expanded data set is 5301, and the detection data set is not expanded to 530, and the expanded data set reaches 5765. The Experimental Data Set is shown in

Table 2. Experimental Data Set.

Dataset	Number of Samples
Locate Unextended Datasets	530
Locate Extended Data	5301
Detect Unexpanded Datasets	530
Detect Extended Datasets	5765

. In the positioning and detection training process, 85% of the data set is used as the training set, and 15% of the data set is used as the verification set. The initial learning rate is set to 0.01, the number of iterations is 200, and the IoU (Intersection over Union) is 0.5.

4.3. Performance Evaluation Index

There are usually three evaluation indicators for target detection algorithms: detection accuracy, detection speed, and recall. Average Precision (AP) is the percentage of the number of correctly identified objects in the total number of objects, and mAP (Mean Average Precision) is used as an index to evaluate the accuracy of detection of all categories [40]. The average recall rate is the percentage of the number of correctly identified objects in the number of objects in the test set. The true positive (TP), false negative (FN) and false positive (FP) are used to calculate precision, recall. In the experiment, the accuracy rate is calculated by (1), the recall rate is calculated by (2). Then we calculate mAP and F-Measure by precision rate (P) and recall rate (R). mAP is calculated by (3), and F-Measure is calculated by (4).

$$Precision=\frac{TP}{TP+FP}\times 100\%$$

(1)

$$Recall=\frac{TP}{TP+FN}\times 100\%$$

(2)

$$mAP={\int }_0^{1}P\left(R\right)dR$$

(3)

$$F-Measure=\frac{2\times P\times R}{P+R}$$

(4)

4.4. Positioning of the Instrument Dial Area

Use the labeling tool to mark the bounding box [41] of the dial area, generate the corresponding xml file. the file contains the position coordinates of the dial area and other information, then train the digital instrument data set through the Faster R-CNN neural network. After the training is completed, a model file of the relevant data set is generated. Input the test picture, and predict the position coordinates of the upper left corner and lower right corner of each number or the number with a decimal point and the corresponding category, and finally return the position coordinates and the number category to which it belongs. Predict the overall data set, intercept and save according to the predicted coordinates, and obtain an image of the dial area. Figure 6 and Figure 7 are schematic diagrams of the original images collected. Figure 8 and Figure 9 are schematic diagrams of the dial areas after locating.

4.5. Dial Readings Detection without Connected Domain Analysis Module

To verify the important role of the connected domain analysis module in this algorithm, after locating and acquiring the dial area, use the imgaug tool to expand the data set of the obtained dial area image data set, sharpen the dial area image, add Gaussian noise, data preprocessing operations such as contrast enhancement and perspective transformation. In the dial readings detection, the YOLOv4 detector is used directly to divide the result categories into numbers without a decimal point at the end and numbers with a decimal point at the end. There are 20 categories of “0”–“9” and “0.”–“9.”.

After the model training is completed, the precision, recall, mAP and F-Measure of each category are calculated in the verification set. The study found that the PR (Precision–Recall) curve of “7.” and its mAP, F-Measure is lowest, and the false detection rate of “9.” is high. The PR curve of “7.” is shown in Figure 10, and the PR curve of “7” is shown in Figure 11. The experimental results of each category are shown in

Table 3. Performance Indicators For Each Category.

Class	Precision%	Recall%	AP%	F-Measure
0	99.52	99.58	99.74	1.00
1	99.34	99.44	99.52	0.99
2	99.88	98.73	98.97	0.99
3	98.52	97.90	98.37	0.98
4	80.23	97.99	98.19	0.88
5	97.79	98.66	99.06	0.98
6	89.77	93.99	95.37	0.92
7	99.46	99.66	99.90	1.00
8	100.00	94.81	96.51	0.97
9	75.00	95.60	95.50	0.90
0.	99.89	98.94	99.15	0.99
1.	99.95	97.67	97.94	0.99
2.	99.86	98.95	99.20	0.99
3.	100.00	99.80	99.88	1.00
4.	99.64	98.94	99.06	0.99
5.	99.88	94.53	96.69	0.97
6.	99.75	99.88	99.88	1.00
7.	99.57	91.82	94.35	0.96
8.	99.89	99.45	99.45	1.00
9.	71.82	99.25	99.00	0.83

. The experiment also found that the existence of the decimal point is enough to cause a missed detection when it is classified. Figure 12 is a schematic diagram of the missed detection.

4.6. Dial Readings Detection with Connected Domain Analysis Module

Combined with the YOLOv4 target detector of the connected domain analysis method, the detection targets in the input image are divided into 10 categories “0–9” by number. It can reduce the mutual interference between data categories, therefore improving the performance of the detector. Experiments show that the method increases the average accuracy of the YOLOv4 target detector from 98.29% to 99.92%, and all types of F-Measure are also increased to 1.00. The data shows that this method is suitable for the actual digital instrument data set of the substation. The experimental results of each category are shown in

Table 4. Performance Indicators For Each Category.

Class	Precision%	Recall%	AP%	F-Measure
0	99.81	99.96	100.00	1.00
1	99.94	99.94	100.00	1.00
2	99.95	99.93	99.97	1.00
3	99.99	99.91	99.96	1.00
4	100.00	99.63	99.63	1.00
5	100.00	99.81	99.81	1.00
6	99.81	99.95	100.00	1.00
7	100.00	100.00	100.00	1.00
8	99.96	99.87	99.96	1.00
9	99.75	99.83	99.92	1.00

.

After directly using YOLOv4 to classify targets with and without a decimal point in the tail, the PR curve of the “7.” number and its average accuracy have a lower F value, and the false detection rate of the “9.” category is higher.

To reduce the YOLOv4 network parameters and reduce the mutual interference between categories, the digital panel data obtained in the first stage of detection is marked as 10 categories. If the numbers are the same, regardless of whether there is a decimal point, they are classified into the same category by numbers. The study found that by reducing the interference between the categories with a decimal point at the tail and the category without a decimal point at the tail, the accuracy, recall, average accuracy, and F-number have been significantly improved. The performance indicators of each category are shown in Table 4, and the average accuracy of each category is shown in Figure 13. Figure 14 is a schematic diagram of successfully detecting missing digital areas after joining the connected domain analysis module.

Obtain the area of a single number (possibly with a decimal point), binarize the area, remove noise, morphologically process and find the connected components in the area, and find the area ratio of all two or more connected components, according to whether there is an area ratio, it is judged that the number area contains a decimal point within the set threshold. During the experiment, setting the threshold between 70 and 100 can judge whether the test set contains a decimal point. Figure 15 is part of the actual effect picture.

5. Conclusions

By combining the connected domain attributes of the image in traditional image recognition, the detection difficulty of the convolutional neural network in target detection can be reduced, the model calculation amount is reduced, the final detection accuracy is significantly improved, and the detection efficiency is also improved. Faster R-CNN, as a two-stage detection algorithm, further accurately detects the pre-screened suggested areas, and is suitable for positioning the dial area of digital instruments as a positioning network. YOLOv4, as the most accurate single-stage detection algorithm today, directly performs regression classification on the digital area. Although ensuring real-time performance, the detection accuracy is guaranteed. the connected domain analysis algorithm reduces the interference between the decimal point and the number category in the YOLOv4 detection process. After the model is fully trained, the accuracy and detection speed have been significantly improved.

When the contrast of the collected data is not strong enough, the image data results obtained may be affected to different degrees. In the future, it should not be limited to a certain substation. It is also necessary to conduct a more in-depth study of the substation data with different contrasts using this algorithm. We need to integrate more relevant attributes of traditional image recognition into the actual application scenarios of neural networks, construct related algorithms, improve algorithm versatility, and improve model performance.