Nonlinear Autoregressive Neural Network Models for Prediction of Transformer Oil-Dissolved Gas Concentrations

Abstract

Transformers are one of the most important part in a power system and, especially in key-facilities, they should be closely and continuously monitored. In this context, methods based on the dissolved gas ratios allow to associate values of gas concentrations with the occurrence of some faults, such as partial discharges and thermal faults. So, an accurate prediction of oil-dissolved gas concentrations is a valuable tool to monitor the transformer condition and to develop a fault diagnosis system. This study proposes a nonlinear autoregressive neural network model coupled with the discrete wavelet transform for predicting transformer oil-dissolved gas concentrations. The data fitting and accurate prediction ability of the proposed model is evaluated in a real world example, showing better results in relation to current prediction models and common time series techniques.

1. Introduction

As the transformer is one of the most important unit of an electrical system, it is natural that efforts are made to preserve its integrity and increase its availability [1]. For these purposes, maintenance policies and procedures are planned and applied to ensure the least interruption of such equipment [1,2,3]. In fact, any failure in this equipment can affect the whole network, compromise other elements in the grid and generate significant economic impacts [1,2].

Especially regarding oil-filled transformers, the maintenance operations should be carried out with additional caution to minimize the potential problem of flammability of the thermal insulation material [4]. Due to its complexity and importance, the problems of aging degree of paper insulation has been object of study in many recent works [2,3,4]. Several other tests of insulation items have been an important part of transformers fault diagnosis systems, with emphasis on chromatographic oil-dissolved gas analysis, namely dissolved gas analysis (DGA) [4,5,6,7,8,9].

In this context, methods based on the dissolved gas ratios allow values of gas concentrations to be associated with the occurrence of some faults, such as partial discharges and thermal faults. Power transformers faults usually take to the degradation of the insulating materials, which results in the release of certain gases that are dissolved in the oil. From a certain concentration level, these gases act as a thermal insulator bringing forth the equipment overheating and, simultaneously, decreasing the oil dielectric vigor, which may cause electric isolation failure. On the other hand, the overheating of the oil increases the levels of some gases such as methane and ethylene, for example. So, an accurate prediction of oil-dissolved gas concentrations is a valuable tool to monitor the transformer condition and to develop a fault diagnose system [10].

Over the last years, the analysis of dissolved gases in the transformers oil, based on International Electrotechnical Commission (IEC) [11] and Institute of Electrical and Electronics Engineers (IEEE) guidelines [12], became a widespread practice followed by all electric power companies. In this context, the use of artificial intelligence techniques combined with chromatographic analysis of oil-dissolved gases (DGA) deserves a highlight. In general, intelligent approaches have been proposed to circumvent the limitations of purely traditional DGA-based methods, with emphasis on artificial neural networks approaches, including generalized regression neural network [10], support vector machine (SVM) [5,13,14], expert systems (EPS) [15], and fuzzy systems [16]. A recent survey by Cheng and Yu [4] shows that the use of these techniques has produced promising results in the development of high precision fault diagnosis systems. However, recent results indicate that these techniques still present limitations in the prediction of oil-dissolved gases, leaving room for further improvements.

Regarding expert systems, for example, an accurate simulation of the experience, skill, and reasoning process of the experts strongly depends on the quality of the established knowledge base, which is one of the main limitations of this approach in most cases. In general, the knowledge base hardly considers all possible cases which leads to errors in identifying symptoms of faults not present in the database [4,6].

In relation to neural networks, the basic idea is to map a highly nonlinear input and output relationship and, from this relation, output a diagnosis conclusion about the fault [4]. Despite satisfactory results, this traditional approach is not able to predict multi-step ahead values and, mainly, the performance is influenced by the input data and it is limited by training samples and parameters. In [10], for example, the authors propose the use of principal components analysis (PCA) to improve prediction accuracy by selecting the most representative inputs for network training. In fact, the use of PCA in this context is widespread [10,17,18]. Moreover, the usual application of non-recurrent models, such as the Generalized Regression Neural Network (GRNN), performs only non-linear mapping between inputs and outputs and is not appropriate for future estimation of oil-dissolved gas values.

Another important difficulty in neural network approaches is the adjustment of training parameters. GRNN models, for example, are strongly dependent on the smooth factor parameter. The authors of [10] overcame this limitation by applying an optimization method only to select a suitable value of the smooth factor. That approach has the limitation of needing to recalculate the smooth factor whenever the data is updated. Those authors also apply the principal components analysis to reduce the influence of the input data on the model.

Despite producing relatively accurate results, with good generalization capacity and little over-fitting, the SVM in regression problems is also strongly influenced by the quality of the input vectors [14].

On the other hand, despite addressing some issues regarding data uncertainty, the use of fuzzy logic increases the dependence on expert knowledge to create a set of fuzzy rules, which describes the relationships between input and output variables. Many authors propose the creation of a set of rules based on standards (IEC 599, IEEE), which clearly is not efficient, since combinations of different gas ratios contemplated by the standard may not occur in practice, leading to a serious problem of indecision or non-decision in diagnosis.

Considering this situation, we propose here a combination of a nonlinear autoregressive neural network model with the discrete wavelet transform, for predicting dissolved gas concentrations and gas concentration ratio in transformer oil. The objective of this approach is to create a model that is invariant in relation to the time delay parameter and allows a less sensitive prediction to long-term time dependencies, besides presenting better generalization and learning capacities, resulting in a high-accuracy multi-step ahead forecast of in-oil gas concentrations.

Wavelet transform localizes features in the input data and concentrates its features in a few wavelet coefficients without affecting the data quality [19]. As a result, we have a set of input data of simplified complexity that leads to a high accuracy prediction, without any human intervention. So, the hypothesis is that the use of wavelets functions to create sparse versions of the initial data can increase the prediction accuracy of the model, increasing confidence in multi-step ahead predictions and reducing the effect of the delay parameter in a high precision fault diagnosis.

As the proposed model can predict future values of the oil-dissolved gas concentrations, it is applied in a transformer condition monitoring system, in conjunction with reliability techniques, to provide an early diagnosis of possible faults and to estimate the remaining life of the transformer based on historical data and events accumulated over a given period.

It should be noted that the proposed model is not intended to produce a conclusion about the transformer fault diagnosis, but rather a high precision prediction of concentrations and gas ratios at future time points, allowing the operator to anticipate possible faults and proceed with recommended protective measures. Thus, the main contributions of the paper are as follows:

The development of a gas prediction model based on combination of wavelet functions with a nonlinear autoregressive network, insensitive to the delay parameter and type of wavelet function;

High precision forecasts of future values of the concentrations and ratio of oil-dissolved gases, contributing to increase the reliability of the monitoring system to anticipate possible faults;

An alert system that monitors future values of gas concentrations and allows the anticipation of abnormal situations and to carry out appropriate protection measures.

The data fitting and accurate prediction ability of the proposed model is evaluated in a real-world example, showing better results in relation to several current prediction models and common time series techniques.

2. Related Theory

2.1. Artificial Neural Network

The Artificial Neural Network (ANN) is a mathematical method that aims to simulate the human brain in the knowledge acquisition process, with successful applications in nonlinear mapping between input and output variables, pattern recognition and classification, optimization, just to name a few [20,21,22].

As a kind of a human brain biology model, ANN sets up some components that define essential properties of the biological neuron, such as synapses, neural weights, and transfer functions [20].

The artificial neurons are the processing elements, which are organized in successive interconnected layers that receive the information (input variables) and propagate it towards the output layer. The information received is weighted by a synaptic weight that determines whether the next neuron will be activated by the activation function, usually the sigmoidal function in non-linear problems.

Network learning takes place as the weights are adjusted along the layers, according to the relationship between the inputs and the desired outputs.

One of the most basic models is Multilayer Perceptron Network (MLP), which is widely used in the approximation of non-linear functions that describe complex relationships between independent and dependent variables in many applications.

Nonlinear Autoregressive Neural Network

The Nonlinear autoregressive neural network is a kind of ANN appropriate for estimation future values of the input variable. The NAR Network enables the prediction of future values of a time series, supported by its history background, by means of a re-feeding mechanism, in which a predicted value may serve as an input for new predictions at more advanced points in time [23,24]. The network is created and trained in an open loop, using the real target values as a feedback and thus ensuring greater accuracy in training. After training, the network is converted into a closed loop and the predicted values are used to supply new feedback inputs to the network. The architecture of the open and closed loop is illustrated in Figure 1.

Mathematically, the model forecasts future values of a time series y(t), based on its historical values y(t − 1), y(t − 2), ..., y(t − d), (NAR model) utilizing additionally an external time series x(t − 1), x(t − 2), ..., x(t − d) (NARX model), where d is the time delay parameter.

The network training is performed, generally, by the backpropagation algorithm, and uses steepest descent method to minimize the squared error between the real values and the predicted ones.

2.2. Discrete Wavelet Transform

Wavelet transform is a signal processing technique, widely applied and developed in many different areas [25,26,27]. Due to its success in many applications, several works attempt to motivate and explain the basic ideas behind wavelets [28]. In this context, we highlight the lifting technique that seeks to explore correlations in the data to construct a sparse approximation in the spatial domain, which makes it more efficient than approaches based on the frequency domain [28].

Mathematically, the lifting approach receives a non-random signal $S^j$, of length $2^j$ as illustrated in Equation (1), and constructs an approximation of this signal by writing each odd sample as an average of two adjacent samples. The accuracy of the approximation is defined by a set of detail coefficients, which are calculated as the difference between an odd sample and its computed approximation, as expressed by (2),

$$S^j=\left\{s_1^j,s_2^j,s_3^j,\dots ,s_{2^j}^j\right\}$$

(1)

$$d_k^{j-1}=s_{2k+1}^j-\left(s_{2k}^j+s_{2k+2}^j\right)/2.$$

(2)

An additional operation, called update, is performed to preserve the average value of the original signal, according to (3).

$$s_k^{j-1}=s_{2k}^j+\left(d_{k-1}^{j-1}+d_k^{j-1}\right)/4.$$

(3)

The approximation signal, $S^{j-1}$, represents a high fidelity simplified and sparse version of the original data.

3. The Proposed Prediction Model

The presence of a small concentration of oil-dissolved gases in the transformer is a natural consequence of the normal operation of this equipment, due to the electric field, humidity, and oxidation [10]. However, an increase in the concentration of these gases, which includes hydrogen (H₂), carbon monoxide (CO), carbon dioxide (CO₂), methane (CH₄), acetylene (C₂H₂), ethylene (C₂H₄), and ethane (C₂H₆), may be related to the occurrence of failures and abnormalities. The elevation in methane and ethylene concentrations, for example, may indicate the occurrence of some thermal failure in the transformer, while variations in hydrogen and acetylene are indications of electrical faults [12].

Therefore, the variation of the gas concentrations over time is a critical issue in the transformer fault diagnosis analysis. Moreover, as some of these gases have a strong correlation in a situation of failure, many gas concentration ratios also must be considered.

So, the proposed prediction model (NAR–DWT) is applied to predict future values of the seven kinds of oil-dissolved gas and the IEC and Rogers ratios (CH₄/H₂, C₂H₂/C₂H₄, C₂H₄/C₂H₆), according to the following steps:

Step 1: A set of historical oil-dissolved gas data is collected from a transformer equipped with a GE Kelman-Transfix (GE—General Electric, São Paulo, Brazil) and GE Intellix BMT 330 (GE—General Electric, São Paulo, Brazil).

Step 2: The collected data set is evaluated using the discrete wavelet transform to create a sparse and simplified version with good approximation properties.

Step 3: Nonlinear autoregressive neural network models are trained and validated according a k-fold cross validation approach.

Step 4: Apply the created neural network model to predict the oil-dissolved gas concentrations and ratios.

A flowchart of the proposed prediction model is presented in Figure 2.

Prediction of In-Oil Gas Concentrations

The nonlinear autoregressive model has been applied and the accuracy of the prediction is evaluated using the mean squared error performance between given target and predicted values.

A training function was applied based on Bayesian regularization, random data division with 80% for training and 20% for testing. We also applied a k-fold cross-validation, in which the data were divided into 10 subsets and the training repeated 10 times, using one of the 10 subsets at a time to test/validation, while the other 9 subsets forming a single training set. The error estimation is averaged over all 10 trials.

The wavelets Symlets and Daubechies are applied to create a sparse and simplified version for the gas concentrations and gas ratios data, respectively.

In order to enable a comparison regarding prediction accuracy and validity we adopted the same evaluation criteria of [1]:

(a)

The relative percentage error between target and predicted values (avg_err)

$$avg\_err=\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N\left|\frac{{\tilde{x}}_i-x_i}{x_i}\right|\times 100.$$

(b)

The maximum relative error (max_err)

$$max\_err=max\left|\frac{{\tilde{x}}_i-x_i}{x_i}\right|$$

in which N is number of data samples, $x_i$ and ${\tilde{x}}_i$ are target and predicted value, respectively.

The proposed model is evaluated in real world example using a set of oil-dissolved gas concentration data from a transformer in a 13.8–230 kV, 190 MVA substation located in Brazil. The device has been equipped with a GE Kelman-Transfix that is featured with a photo-acoustic detection technology to measure the gas concentrations. The data set is composed of seven months of daily observations, carried out in the period from November 2016 to May 2017, corresponding to 176 samples of the gases H₂, CO, CO₂, CH₄, C₂H₂, C₂H₄ and C₂H₆.

4. Numerical Results

First of all, we evaluated the effect of the time delay parameter d on the performance of the training process, evaluated using the mean squared error (mse) and the coefficient of determination R, which is a goodness-of-fit measure for linear regression between the target and the predictions. In this case, we set d from 2 to 10, with step 1, and the error averaged over all 10 trials according to the cross-validation approach described above.

Table 1. Effect of the parameter d on the performance of training process for CO₂ gas.

d	mse	R
2	0.0000007	0.99
3	0.0000010	0.99
4	0.0000007	0.99
5	0.0000006	0.99
6	0.0000005	0.99
7	0.0000006	0.99
8	0.0000008	0.99
9	0.0000008	0.99
10	0.0000008	0.99

illustrates the results for this evaluation using the concentration of CO₂ as an example. The NAR–DWT model presents a very accurate fit and a small mse independent of the value of d. The results for the other gases are similar.

We also tested the performance of the wavelet functions and the effect of this factor on the accuracy of the proposed model. For this we selected some usual functions from three traditional wavelet families, Daubechies (db), Symlets (sym), and Coiflets (coif), and evaluated results regarding mse, R, max_err, and avg_err. Corresponding results are highlighted in

Table 2. Performance of the wavelet functions and the effect of this factor on the accuracy of the proposed model.

Wavelet	mse	R	avg_err (%)	max_err (%)
sym2	0.0000004	0.99	0.06	0.34
sym3	0.0000010	0.99	0.08	0.50
sym4	0.0000007	0.99	0.08	0.52
db1	0.0000007	0.99	0.15	1.02
db3	0.0000010	0.99	0.08	0.52
db5	0.0000002	0.99	0.03	0.19
coif1	0.0000011	0.99	0.10	0.60
coif3	0.0000004	0.99	0.06	0.35
coif5	0.0000003	0.99	0.04	0.28

.

The prediction results of oil-dissolved gas concentrations and the gas ratios are presented in

Table 3. NAR–DWT prediction error for oil-dissolved gas concentrations.

Gas Type	avg_err (%)	max_err (%)
H2	0.46	6.03
CO	0.08	0.79
CO2	0.06	0.34
CH4	0.10	0.89
C2H6	0.29	2.34
C2H4	0.32	3.11
C2H2	0.33	2.37

and

Table 4. NAR–DWT prediction error for gas concentrations ratios.

Gas Type	avg_err (%)	max_err (%)
CH4/H2	0.58	3.35
C2H2/C2H4	2.02	10.48
C2H4/C2H6	0.66	9.82

, respectively. It is possible to see that the proposed model has a great accuracy regarding the max_err and avg_err. Additionally, an illustration of the output and target plot and error for the gas H₂ is presented in Figure 3, corroborating the good degree of fit of the NAR–DWT model. From Figure 3 it is possible to verify the high accuracy in the prediction of the proposed model. The output and target plot clearly illustrates that the model can accurately reproduce the oscillatory behavior of the input data, presenting a target-output error of less than 0.01. Even at points in which the gas variation is greater, as in the 40 s and 128 s time instants, the model can keep up with the variation despite producing a slightly larger average error in this case.

The results generated by the proposed method have been compared with important current prediction methods from the literature: KPCA-FFOA-GRNN, FFOA-GRNN, KPCA-GRNN, GRNN, BPNN from [10] and SVM from [13].

Some time series techniques were also used to compare the results of the in-oil dissolved gas concentrations prediction values, and their ratios. The following statistics models were used: autoregressive moving average model (ARMA), autoregressive integrated moving average models (ARIMA), seasonal autoregressive integrated moving average model (SARIMA), Autoregressive model for conditional heteroscedasticity (ARCH), and the generalized autoregressive conditional heteroscedasticity (GARCH) [29].

The selection of the most suitable prediction model among all the options tested was performed using the Akaike information criteria (AIC) and Bayesian (BIC), which indicated the model most adjusted to the data by relative quality analysis of the statistical models.

This comparison is illustrated in

Table 5. Prediction error for the ethylene gas by different models.

Prediction Model	avg_err (%)	max_err (%)
NAR–DWT	0.32	3.11
KPCA-FFOA-GRNN	3.27	11.34
FFOA-GRNN	5.04	12.81
KPCA-GRNN	7.09	15.14
GRNN	7.93	14.06
BPNN	8.72	19.52
SVM	4.21	10.65
GM	6.69	15.77
AIC	9.15	22.33
BIC	18.4	40.87

for the prediction of the ethylene gas concentration.

For this test we used 141 samples for training and 35 samples for testing.

Application in the Transformer Fault Diagnosis Analysis

The NAR–DWT model was applied to a transformer monitoring system, based on the evaluation of the equipment condition from the analysis of the oil-dissolved gases. The model was used to predict the concentration of each gas for a time horizon of 50 days ahead, as shown in Figure 4. The results of each forecast were evaluated against the limit values defined by the standards guidelines adopted in Brazil, related to statistical quality control ideas, as highlighted in Figure 4.

5. Discussion

This section presents the discussion about the application of the proposed model in a real-world example, in relation to the accuracy of data fitting for predicting oil-dissolved gas concentrations in transformer.

Regarding the evaluation of the performance of the wavelet functions and the time delay parameter d on the training process, and the effect of these factors on the accuracy of the proposed model, the robustness of the model is clear. The combination of nonlinear autoregressive neural network and wavelet transform enable to produce high precision prediction of oil-dissolved gas concentrations and ratios. The low influence of the wavelet function on the accuracy of the prediction indicates that the proposed model is less sensitive to variations in input data for training, a problem that is usual in approaches based on neural networks [4]. In addition, the adoption of the cross-validation approach increases the reliability of the model and leads to better quality forecasts.

Results from a real example showed that the proposed model presented good accurate predictions, higher than that obtained by the current tested methods such as Generalized Regression Neural Network (GRNN), support vector machine (SVM), back propagation neural network (BPNN), and usual time series techniques. Specifically, regarding the ethylene gas, the maximum prediction error of the NAR–DWT model was about 70% smaller than the other tested models. The superior performance presented was independent of the delay parameter, d, used in the model. This result is coherent with the literature that states that nonlinear autoregressive network models are less sensitive to long-term time dependencies, besides presenting better generalization and learning capacities [22]. Moreover, an additional increase in performance is due to the application of the discrete wavelet transform to create simplified and sparse versions of the original data.

The ability to accurately predict future values of gas concentrations allows the reliability of the monitoring system to be increased and to anticipate possible faults in the system. In this sense, a prediction of an increase in the concentration of a gas above the limit value issues an alert indicating the need for an immediate check of the operation status of the transformer. In addition, multiple gas concentration predictions are used to estimate the average time, in days, to an alert condition, as well as the lower and upper limits of the 95% confidence interval for that average. This information is then used to calculate the remaining life of the equipment.

6. Conclusions

This study proposes a combination of a nonlinear autoregressive neural network model with the discrete wavelet transform for predicting power transformer oil-dissolved gas concentrations.

The model presents a low sensitivity in relation to the training parameters, the wavelet functions and the time delay parameter d, which confers a robustness necessary for a high precision prediction of oil-dissolved gas concentrations and its ratios. As the model is able to predict multiple future values of the gas concentrations, it was applied in the context of a transformer fault diagnosis analysis system, allowing the issuance of alerts in case of possible faults and providing reliable interval estimates for the average time to the occurrence of an abnormality.

A drawback of the model is the need for retraining as new measurements are incorporated into the database, as is common to every model based on artificial neural networks. Future work may include investigating the performance of such approach for predicting the causes of a failure phenomenon from a database of occurrences, which can be performed reliably only when the fault database contains a sufficient number of cases.