Deep Neural Network-Based Autonomous Voltage Control for Power Distribution Networks with DGs and EVs

Featured Application

Medium voltage networks that face voltage regulation issues due to high penetration of distributed generation.

Abstract

This paper makes use of machine learning as a tool for voltage regulation in distribution networks that contain electric vehicles and a large production from distributed generation. The methods of voltage regulation considered in this study are electronic on-load tap changers and line voltage regulators. The analyzed study-case represents a real-life feeder which operates at 10 kV. It has 9 photovoltaic systems with various peak installed powers, 2 electric vehicle charging stations, and 41 secondary substations, each with an equivalent load. Measurement data of loads and irradiation data of photovoltaic systems were collected hourly for two years. Those data are used as inputs in the feeder’s model in DigSilent PowerFactory where Quasi-Dynamic simulations are run. That will provide the correct tap positions as outputs. These inputs and outputs will then serve to train a Deep Neural Network which later will be used to predict the correct tap positions on input data it has not seen before. Results show that ML in general and DNN specifically show usefulness and robustness in predicting correct tap positions with very small computational requirements.

1. Introduction

The increase in distributed generation (DG) has changed the traditional direction of power flows in medium voltage (MV) networks. Up until recently, power flew from higher voltage towards the loads which are connected on the low voltage (LV). Now, in instances when the generation from plants that are connected on the LV side is higher than the load itself, the power will flow upstream [1,2]. There are several reasons for the popularization of distributed generation (DG). The first one is the rapid decline of costs of devices and technologies that are needed to build and operate plants based on Renewable Energy Sources (RESs). One of the significant price declines is that of the photovoltaic panels, which became 99% cheaper in 40 years, as the price dropped from 105.7 USD/Watt in 1975 to only 0.2 USD/Watt in 2020 [3].

Other reasons are the public’s raised interest and awareness towards climate change, governmental incentives, and common goals from various international organizations [4,5,6].

Without doubt, RESs have their advantages, the main one being the fact that they decrease the overall CO₂ emissions. RESs, however, also have their disadvantages. Their unpredictable nature poses new challenges for the operation of the power system [7]. One of the issues this causes is over-voltages [8]. Due to that, this paper investigates two voltage regulation mechanisms that will be controlled by a machine-learning (ML) algorithm. Although voltage is closely related to reactive power, no reactive power methods of voltage regulation such as reactors or capacitors are considered in this paper. Instead, the voltage regulation mechanisms considered are the line voltage regulator (LVR) and the electronic on-load tap changer (E-OLTC). The intention is to use an ML algorithm as the controller of the correct tap positions of E-OLTCs and LVRs, in such a manner that voltages are kept within ±10% of the nominal voltage. Different geographical areas have different Distribution Codes that decide the allowed voltage range. In the feeder used in this study case, the allowed voltage range is ±0.1 Vn in normal operation and down to −0.15 Vn in critical conditions.

The effectiveness of LVRs and E-OLTCs depends greatly on the topology of the feeder and their positioning in it. Since LVRs are installed longitudinally, they become useful in long feeders that do not have many side branches, while E-OLTCs perform better in branchy feeders. In cases where feeders seem to have a main long axis, LVRs can prove to be an excellent choice if installed at a point along the axis where there are significant voltage drops or under-voltages [9,10]. That usually is valid for villages and other extra urban feeders that go on for tens of kilometers. On the other hand, for feeders in cities, LVRs may not be helpful due to these feeders typically having several branches. In the latter case, E-OLTCs that are part of the usual MV/LV transformers are better suited. E-OLTCs make it possible to directly regulate the voltage of their busbars to which the loads are connected, without affecting a larger portion of the feeder [11,12]. This paper does compare their individual and combined effects on voltage regulation.

ML algorithms are receiving increasing attention in many fields, with no exception for power systems [13]. For instance, in [14], the authors used an ML and geographic information systems (GIS) approach for electrical load assessment to increase resilience in distribution networks. In [15], the authors conducted research on the ML-based application to predict cascading failure propagation. Paper [16] proposes a bottom-up data-driven multistage adaptive robust optimization (MARO) framework to address the power systems’ renewable transition under uncertainty, while paper [17] proposes a machine-learning method to evaluate the low-frequency oscillation stability of the power system accurately and efficiently considering the random response data containing the uncertainties of the power grid. In the field of ML applied to voltage control, paper [18] is a step forward in the path of reinforcement learning applications of Multiterminal Soft Open Points (M-SOPs) using Deep Deterministic Policy Gradient (DDPG). It offers quicker-than-before adjustments because it is data-driven and performs voltage control locally using a Markov Decision Process (MDP) and real-time measurement data. Paper [19] makes use of similar tools. Once again, reinforcement learning and Deep Deterministic Policy Gradient (DDPG) is used, including the Markov Decision Process (MDP) as a means to formulate the problem. It possesses the uncommon advantage of a model which is expected to perform well when the network data is unreliable or missing.

The remaining four sections of the paper are organized as follows: Section 2 explains the proposed methodology for incorporating ML in voltage regulation. Section 3 is concerned with the creation and preparation of data for loads and PVs. Section 4 covers the Quasi-Dynamic (Q-D) simulations performed using PowerFactory. Section 5 represents the building, training, and testing of the Deep Neural Network (DNN). Section 6 measures the effect of incremental DG penetration levels. Section 7 offers a potential improvement in the lifecycle of tap changers by reducing their operations. Finally, Section 8 contains the concluding remarks.

2. Proposed Methodology

Speaking in high-level terms, the methodology is to first gather data of loads, PVs and EVs which will serve as inputs. Then, a real-life feeder is modeled using PowerFactory 2022 software, on which the prepared and cleaned inputs are inserted. Quasi-Dynamic analyses are run and the software’s results are exported. These results are then prepared, cleaned, and divided into the training set and the test set. These datasets are used to build a Deep Neural Network architecture and train/tweak it as necessary. After a good ML model is built, it is tested using the validation set, then the model’s predictions on the test set are exported back to PowerFactory, where the final comparison of the results given by DNN and PowerFactory software are made. Besides these, secondary analyses are run to make the understanding more profound, such as the effects of PV penetration level changes and tap operations number reduction.

This section further explains the approach to gathering the relevant data that will later be used to train and test the DNN. The topology of the 10 kV network of this study-case is shown in Figure 1. The feeder lies in a rural mountainous area, and it has around 60 km of lines in total.

In Figure 1, the primary substation is shown in a red square, the secondary substations are shown in blue triangles, and the lines are shown in yellow. There are 41 equivalent loads, each connected to a secondary substation (SS). As of now, there are no Electric Vehicle (EV) charging stations and little to no photovoltaic generation. To study the effects of high DG, nine photovoltaic systems and two EV charging stations will be added. To better understand the role of the penetration level of DGs, five different analyses were conducted, each with a relative percentage of power generation, starting at 60% Pn and ending at 140% Pn.

The voltage regulation will be performed by the automatic tap changer of the HV/MV transformer, by four LVRs installed at various points in the feeder and by transformers equipped with E-OLTCTs. The primary substation has 21 tap positions, ranging from −10 to +10, each with ±1.25%Vn per tap. LVRs and E-OLTCS have five taps with a regulation range of ±2.5% per tap when moved from the neutral position. The feeder’s model made in PowerFactory is shown in Figure 2.

To train the DNN, we need as many training examples as possible. These training examples will have to contain input data and outputs. The inputs, called features, will be the active loads, reactive loads, charging stations loads, and photovoltaic generations. The outputs, called labels, will be the correct tap positions of each E-OLTC and LVR. The data are real and come from smart meter measurements exported hourly. The data pertain to two full years, which translates to 17,520 values for each parameter. With 109 parameters in total, there will be 1,909,680 values. This feeder has a peak load of around 2.5 MW, while the aggregate peak installed power PVs is around 1.9 MWp. That means the ratio of PV generation to the peak load is roughly 76%. However, it is important to note that the peak load of the feeder and peak generation of PVs do not occur at the same hours as these are two different characteristics which depend on the location, which means that the ratio may well go beyond 100%.

To better assess the effect of the penetration level of DG for a fixed peak load, a characteristic against voltage constraint violations will be plotted later. Various scenarios are tested to have a clearer understanding of the individual impact of E-OLTCs and LVRs on this MV feeder. Scenario I corresponds to the actual conditions where neither the E-OLTCs nor LVRs have automatic tap changing. Scenario II is the case where LVRs have automatic tap changing but E-OLTCs do not. Scenario III is the case where E-OLTCs have automatic tap changing while LVRs do not. The last one, Scenario IV, is the case where both E-OLTCs and LVRs are equipped with automatic tap changing, so they both serve as real-time voltage regulators. This paper is not concerned with mechanical tap-changing transformers which require de-energization when operated.

The results from each of these scenarios contain inputs and outputs which are then exported, prepared, and used as examples to train the DNN. The training examples are first shuffled, then separated into a training set (around 75%) and a test set (around 25%). The training set will adjust the weights of the DNN and make the model learn by maximally decreasing the loss function. After the model has been trained, the DNN is fed the test set which it has not seen before, so it will predict the outputs by itself.

The performance of the ML model is measured using the loss function and by being compared to tap positions found by the deterministic method through PowerFactory. A simplified methodology flowchart is provided in Figure 3.

3. Creating Loads and PV Generation Data

This section reports the details of the characteristics that are built based on loads and PV productions. These profiles will be the inputs in PowerFactory and the ML model. The load profiles are real as they are supplied by real measurements. On the other hand, PV generation characteristics are built using irradiation data for the relevant location using PVGIS [20].

3.1. Secondary Substation Load Data

Using smart meters, the data is collected in real time and exported hourly in a database. These meters measure both active and reactive components of loads. Active power of loads for the period from September 2021 to September 2022 is shown in Figure 4. Since there is only one load per secondary substation LV side, it means this is the equivalent or sum of all LV feeders connected to it.

The reactive powers of loads are shown in Figure 5.

There are 41 SSs in total, each has active and reactive components. That gives us 82 vectors of hourly data. Then, four more vectors belong to the hourly loads of the two charging stations, each with two ports, one 11 kW and the other 50 kW. Getting these vectors together builds the matrix of loads ($L_{\mathrm{17,520}\times 86}$). This matrix is exported to “.csv” format and processed in Python. This matrix will serve as input for the training examples which are needed by the DNN.

3.2. Secondary Substation PV Generation Data

Since the PV plants considered here are fictive, their power generation data cannot be taken from measurements. Instead, the production profiles of PVs are built using irradiation data converted to output power. There are several online tools that provide such information by choosing the geographical location of interest. In this paper, the SARAH2 database of PVGIS was used, which is a tool offered by the European Commission. It is possible to obtain the data for a unit 1 kWp PV plant and build the unit production profile for that geographical territory. That is much easier than processing parameters such as DNI (Direct Normal Irradiation), DHI (Diffuse Horizontal Irradiation), or GHI (Global Horizontal Irradiation).

Upon building the unit production vector, it can be used for all other PVs, as they are all subject to the same irradiation, only their installed peak power changes. Next, that process is explained in steps.

First, using the 1 kWp PV plant’s outputs for two years, the unit production vector is built.

$$I_{pv}={(\mathrm{0,0},0,\dots ,0.89,0.93,0.88,\dots )}_{1\times \mathrm{17,520}}$$

(1)

Then, another vector containing the installed peak powers of the PVs is built:

$$P_{pv}={(PP1,PP2,\dots ,PP9)}_{1\times 9}$$

(2)

When multiplying $I_{pv}$ transposed and $P_{pv}$, we get the matrix which contains the hourly power outputs for all nine plants for the entire two years:

$$P_0={({I_{pv}}^{\prime }\times P_{pv})}_{\mathrm{17,520}\times 9}$$

The PV outputs for the first half of the data are shown in Figure 6. There are nine joint characteristics, so a zoomed-in picture is shown in Figure 7.

Similar to the loads, the matrix’s columns represent a 2-year characteristic for each PV. Those columns are exported individually to separate ‘.csv’ files. They are first used in PowerFactory, and later, they will also be used as inputs to build and train the DNN. The reactive components of the PVs are added although their power factor is assumed to be fixed in these simulations.

4. Quasi-Dynamic Simulation in PowerFactory

Since the input data is now ready and for each parameter there is a separate comma-separated value file, they can be assigned to their respective elements on PowerFactory. The next step is to run the Quasi-Dynamic (Q-D) simulation. The Q-D simulation is run in hourly steps for a period of two years. For software reasons, in the backstage, the simulations are run separately for each year and then concatenated. The Q-D analysis means that PowerFactory will run a load flow for each hour, every single time with different data for the inputs. And for each of the 17,520 h, it will store the correct position of all automatic tap changers. When the simulation is finished, the results can be exported externally and processed as needed. The exported data will be organized into inputs, or features, that were fed to PowerFactory, and the outputs, or labels, which were calculated by PowerFactory. Data for each hour will serve as one training example and it will be used to train the DNN. The tap positions of all E-OLTCs and LVRs that change hour by hour are shown in Figure 8. The figure shows visually how the taps are adjusted to keep the voltage within the set range of ±10%Vn. Depending on the transformer, the tap changer will be on either the HV side or the LV side.

The four scenarios mentioned in Section 2 are run individually to better understand and compare E-OLTCs and LVRs as voltage regulating methods:

• S1: E-OLTC off and LVR off (Figure 9)
• S2: E-OLTC off and LVR on (Figure 10)
• S3: E-OLTC on and LVR off (Figure 11)
• S4: E-OLTC on and LVR on (Figure 12)

Since the DG penetration is very high, the feeder experiences unacceptable levels of over-voltages in cases where there is no type of voltage regulation in place such as in scenario S1.

The over-voltage constraint violations are counted for each scenario and summarized in

Table 1. Over-voltage violations in one year.

	E-OLTC Off LVR Off	E-OLTC Off LVR On	E-OLTC On LVR Off	E-OLTC On LVR On
V > 1.05 p.u	15,652	6754	473	443
V > 1.10 p.u	3742	655	0	0

. The count considers each busbar for each hour of the entire duration of the analysis. So, the theoretical maximum of constraints would be 17,520*9.

The results are within expectations. The more regulations, the fewer violations. That is why, when combining both LVRs and E-OLTCs, there are the fewest instances of over-voltages. The second-best method is using E-OLTCs only. Significantly worse performance is noted if only LVRs are used. It must be noted that this may be due to the specific topology of this feeder, and as such, it does not represent a generalized finding. Lastly, as expected, the highest number of over-voltage instances occur when there is no voltage regulation at all. It could be said that the topology and the length of lines, the load distribution throughout the feeder, and DG penetration, are all contributing factors to render one method better or worse than the other. In any case, combining both methods yields the best results in any feeder configuration.

5. Over-Voltage’s Dependence on Penetration Levels of DG

A core question is how much DG can be injected relative to its feeder’s load without causing too much trouble? An analysis was performed on this feeder when both E-OLTCs and LVRs were on. The input data comes from only a random portion of the dataset.

Five levels of DG penetration levels were simulated. Each level represents the percentage of the nominal installed DG. This is done by using scaling factors ranging from 0.6 Pn to 1.4 Pn in increments of 0.2 Pn. For each level, a count of over-voltage constraint violations has been taken. The results are presented in

Table 2. Over-voltage violations in one year as a function of penetration level of PVs.

	$60\%{\mathit{P}}_{\mathit{n}}$	$80\%{\mathit{P}}_{\mathit{n}}$	$100\%{\mathit{P}}_{\mathit{n}}$	$120\%{\mathit{P}}_{\mathit{n}}$	$140\%{\mathit{P}}_{\mathit{n}}$
$\mathrm{V}>1.05\mathrm{p}.\mathrm{u}$	0	0	265	894	1643
$\mathrm{V}>1.10\mathrm{p}.\mathrm{u}$	0	0	0	1	80

.

Figure 13 is a graphical representation of the number of over-voltages depending on the percentage of DG.

6. DNN Modeling, Training, and Testing

At this point, the input data or features and the output data or labels are ready to form training examples. The next step is for the DNN model to be created. The training examples will first be randomly shuffled, then divided into two sets, a training set and a test set, usually in a ratio of around 70/30 or so, respectively. The training set will be used by the DNN to learn, while the test set will not be seen until the learning is completed.

The training data makes up 75% of the total 17,520 training examples of

• 104 inputs, from which

  i.

  41 Loads–active power,

  ii.

  41 Loads–reactive power,

  iii.

  9 PVs–active power,

  iv.

  9 PVs–reactive power,

  v.

  4 EV charging station active powers (2 ports for each charging station),

• 86 outputs, a neuron for each of the following transformer taps:

  i.

  45 neurons: 5 taps × 9 E-OLTCs,

  ii.

  20 neurons: 5 taps × 4 LVRs,

  iii.

  21 neurons: 21 taps × 1 Primary SS.

Each of the output neurons will assume values between zero and one. That value represents the probability of that certain tap of that certain transformer to be the correct one.

As for the machine-learning algorithm, a significant amount of time was taken trying out different learning rates, batch sizes, number of epochs, number of neurons in the hidden layer, activation functions, and various optimization algorithms and architectures.

This specific algorithm was chosen while keeping in mind that the voltage in busbars in reality can vary around 10% without breaching any limits, which means that the immediate transformer’s tap position can assume more than one correct position. That automatically leads us to a multi-output output layer and a non-exclusive algorithm with an adequate loss function which allows for multiple neurons to show a high probability simultaneously.

Considering all that, the following settings were chosen:

• The DNN is ‘multi-output’.
• The algorithm is ‘non-exclusive’.
• Classification should be binary, using the ‘Sigmoid activation function’ in the output layer.
• The loss function is ‘Binary Cross-entropy’.
• In the input layer, there are 104 neurons.
• There are two hidden layers: the first, with 96 neurons, and the second, with 90 neurons.
• The number of neurons in the output layer will be 5∙Ndevices (E-OLTC, LVRs, and PSS). So, 86 neurons in total.

The DNN has basic architecture as shown in Figure 14.

Speaking at a high level, the DNN has the following flow process:

• The input (feature) and output (label) data are imported.
• The dataset is shuffled randomly and split into a training set and a test set.
• The data is scaled.
• The DNN is built by choosing the number of neurons in layers, the activation functions, the optimizer, etc.
• The DNN is trained.
• The DNN is tested.
• The DNN predictions are exported.

When the DNN has learned well enough, the test set will be fed to it. This way, the DNN will have to predict the outputs for inputs it has never seen before. The loss function that measures the numerical performance of the DNN in this model is Binary Cross-entropy. It is a logarithmic function, expressed as follows:

$$CE=-t_1\log \left(f\left(s_1\right)\right)-(1-t_1)\log (1-f(s_1\left)\right)$$

where $f\left(s_1\right)=\frac{1}{1+e^{s_1}}$.

This loss function is represented in Figure 15. It can be seen from this logarithmic graph that DNN is learning fast, as it sees a significant decrease after only around 15 epochs. In order to avoid overfitting, some measures are taken such as the use of a Cross-Validation set, setting up the Early-Stop command, employing a 50% Dropout in the hidden layer, etc.

Attention must be paid to the fact that the loss function of the validation set is one epoch behind the training set’s loss function; hence, it is significantly lower.

After the DNN has been trained and tested, its predictions can be measured in a deterministic way by running the results back to PowerFactory. Now, differently from before, PowerFactory will use characteristics for correct tap positions hour by hour, but which were created by the ML algorithm instead of coming from measurement data or software calculations. When that is done, the voltages will be monitored, and the over-voltage instances will be counted. The voltages of the Q-D simulation for the duration of the test set are shown in Figure 16.

The test set is 25% of the 2 years’ hourly data, which translates to 6 months or around 4380 h. Plugging the ML predictions of taps back into the software shows that there are only 143 instances with an over-voltage going beyond 5% and no instances of over-voltages going beyond 10%. These results are comparable to PowerFactory’s own results for this same test dataset, which is 96 such instances.

According to these results, the conclusion is that a well-trained DNN is a robust tool whose tap position predictions give good results which may even be comparable to those of expensive specialized software.

7. Reducing the Tap Position Changes

Another issue that may be raised is the fact that with such voltage regulation methods, regardless of whether the taps are being controlled by a specialized software or an ML algorithm, changing the tap position every hour will cause more wear and tear than usual. A quick deterioration of tap changing equipment translates to a costlier operation decision. A way to mitigate that is if the tap position changes are reduced yet kept at a level that still achieves satisfactory voltage regulation.

In this paper, an initial try was made on the dataset of tap predictions made by the ML algorithm. That was done by only allowing the positions at timestep t + 1 to be changed if the position at timestep t differs for more than one position in either direction. Otherwise, the position at timestep t + 1 is kept the same as the one of the previous timestep. After imposing this condition on this dataset, around 20,538 tap position changes were ignored. That is roughly 1/3 of all possible tap position changes that may happen at every timestep. However, if the voltage regulation resolution is lowered due to the reduced allowed changes, the voltage regulation will not be as good. As shown in

Table 3. Over-voltage violations in one year with ML reduced adjustments.

	PF Hourly Adjustments	ML Hourly Adjustments	ML Reduced Adjustments
V > 1.05 p.u	96	143	1621

, when the Q-D simulation is run with this test set and the tap changes per timestep are reduced, there will be 1621 instances of over-voltages, which is much more than the 143 instances when the tap movements are not reduced, or the 96 instances when the regulation is done by the software itself.

At least for this dataset, this movement adjustment method does not seem very successful.

8. Conclusions

This paper offers an ML approach to voltage regulation in MV networks using LVRs and E-OLTCs as control mechanisms. The first conclusion is that E-OLTCs and LVRs are both good tools for voltage regulation; however, their effectiveness depends largely on the topology of the feeder, the length of its lines, the distribution and amount of load throughout the feeder, and the positioning and sizes of DG.

The second conclusion is that ML in general and DNN specifically show usefulness and robustness in predicting correct tap positions with very small computational requirements. Soon, this algorithm could be further sophisticated to include the control of reactive power methods of voltage regulation. Perhaps to even communicate directly with the inverters of each PV plant on the feeder and send/receive adjustments.

A tool such as this, practically free to develop and tailor to any feeder, quick and with almost zero memory requirements, can be easily integrated into any simple company computer at any substation. It can then be connected to various systems such as SCADA, EPMS, or any system that is concerned with monitoring and control. Most importantly, it can easily be fully automated and work without the need for human inputs. This would lower the workload of personnel, save vast amounts of funds that are normally allocated to expensive specialized software, and it would make the system safer and easier to operate.