GK-SPSA-Based Model-Free Method for Performance Optimization of Steam Generator Level Control Systems

Abstract

The Steam Generator (SG) is a crucial component of a Nuclear Power Plant (NPP), generating steam to transfer heat from the primary loop to the secondary loop. The control performance of the Steam Generator Level Control System (SGLCS) plays a crucial role in the normal operation of the SG. To improve the system’s performance, the parameters of the control system should be optimized. However, the steam generator and its corresponding control system are highly complex, exhibiting nonlinearity and time-varying properties. Conventional parameter-setting methods mainly rely on engineers’ experience, and are laborious and time-intensive. To tackle the aforementioned challenges, a Model-Free Optimization (MFO) method based on Knowledge-informed Historical Gradient-based Simultaneous Perturbation Stochastic Approximation (GK-SPSA) is applied to the performance optimization of the steam generator level control system. The GK-SPSA algorithm is a variant of the traditional SPSA algorithm. The fundamental idea of this revised algorithm is to maximize the utilization of historical gradient information generated during the optimization process of the SPSA algorithm, with the aim of enhancing overall algorithm performance in a model-free optimization context. Based on the effective utilization of historical gradient information, the GK-SPSA algorithm exhibits two improvements over the SPSA algorithm. The first improvement is related to the recognition of the online optimization progress, utilizing the state of the optimization progress to dynamically adjust the optimization step size. The second improvement is related to gradient estimation compensation, employing compensation rules to enhance the accuracy of gradient estimation, thus improving the optimization efficiency. Through simulation experiments, it can be observed that there is not much difference in the final iteration values among the GK-SPSA, IK-SPSA, and SPSA methods. However, the iteration count of GK-SPSA is reduced by about 20% compared to SPSA and by 11.11% compared to Knowledge-informed SPSA (IK-SPSA). The results indicate that this method can significantly improve the efficiency of parameter tuning for the liquid level control system of a steam generator.

1. Introduction

The Steam Generator (SG) plays a critical role in the nuclear steam supply system of a nuclear power plant, and transfers heat from the primary circuit to the secondary circuit, ultimately generating steam [1,2]. During operation, it is of paramount importance to maintain the level within the predetermined programmed setpoint. Deviations from specified level setpoints can compromise plant productivity and safety [3,4,5]. Reports indicate that approximately 70% of emergency shutdowns in nuclear power plants result from inadequate control of the SG water level [6]. Therefore, the efficacy of SG level control assumes paramount significance in ensuring the safe, stable, and economically efficient operation of the system. However, the SG is an exceptionally intricate system, exhibits nonlinearity, and undergoes time-dependent variations [7,8,9,10]. One notable dynamic characteristic of the steam generator involves the occurrence of the shrink and swell phenomenon. Within this phenomenon, the water level exhibits a temporary reversal in response to variations in water mass inventory. This intricacy becomes more pronounced during plant start-up or periods of low turbine load. As a result, accomplishing the optimization of the performance of SG level control is a challenging and complex task.

The control performance of the Steam Generator Level Control System (SGLCS) is influenced by various factors, including the inherent process characteristics, the configuration of the control system, and the setting of controller parameters [11,12]. The parameter-setting process of the SGLCS is similar to the quality control of the injection-molding process and can be considered as a batch process. By optimizing the production parameters of the batch process, it not only reduces resource wastage and energy consumption, but also enhances product stability [8]. This study delves into the data-driven method, and the data-driven optimization method is applied to real industrial processes. This empowers industrial manufacturing and propels production methodologies towards intelligence and sustainability [13]. However, for certain complex intermittent processes like the parameter setting of the SGLCS, determining optimal settings is often empirically driven due to the challenging correlation between configuration parameters and production outcomes, entailing a laborious and time-intensive process. Therefore, how to search for the optimal values to achieve efficiency under the set parameters becomes a significant issue during the operation of a steam generator.

The most common methods for determining optimal controller parameters for a control system can be categorized as follows:

• Experience-based methods. These methods heavily rely on the engineer’s expertise. They operate without depending on the controller-performance model and instead directly seek the optimal settings through a model-free method. These can be categorized as “Fundamental model-free techniques” encompassing methods such as trial and error, the Design of Experiments (DOE), and expert system-based control. These methods offer the benefit of straightforward implementation. However, their drawbacks are evident, as they heavily depend on operator or expert knowledge, are laborious and time-intensive, and it can be challenging to ascertain the true optimal settings [10,14].
• Model-based methods. Engineers should first obtain a suitably precise model to represent the relationship between parameters and performance. Following that, a tuning formula can be devised using this model. Ultimately, the selection of parameters can be influenced by the model coefficients and the formula [15,16,17,18]. However, accurately determining the relationship between parameters and performance in complex industrial systems can be challenging, making it very difficult to apply this method to the SGLCS.
• Model-free methods. Unlike model-based optimization methods, this method does not rely on the issue of model reliability. However, because of the unique nature of this approach, which does not necessitate the construction of an objective function model, it often needs more experimental data to find the optimal solution. This process may require additional computational resources and time. This drawback can make the parameter-setting process quite tedious, ultimately leading to poorer control performance [16]. However, Geng et al. proposed a variety of intelligent optimizations without model methods and applied them in the performance-optimization process of the SGLCS in nuclear power plants. Based on simulation experiments, it is shown that these optimization methods can obtain better control performance. However, this method requires a large number of simulation experiments and cannot be optimized online in the nuclear power plant site [19]. Kong et al. presented a structured and effective optimization method based on Simultaneous Perturbation Stochastic Approximation (SPSA) for the control of steam generator levels and the quality control of injection molding [20]. Through experimental validation, they demonstrated that this method is a fast, cost-effective, and highly efficient parameter-tuning method.

The development history and pros and cons of the above three parameter search methods are shown in Figure 1. The SPSA algorithm is a common algorithm applied in model-free optimization methods. It is a gradient-approximation algorithm suitable for application in multi-dimensional and noisy environments [20,21]. This method has been proved to be an effective method for parameter setting of the SGLCS. Nonetheless, as a method that does not rely on models, there is room for enhancing the efficiency of the SPSA-based optimization method. Taking into account the limitations of conventional SPSA methods, Geng, etc. proposed a Knowledge-Informed Simultaneous Perturbation Stochastic Approximation (IK-SPSA) performance-optimization strategy for the SGLCS based on historical information [19]. While this method endows the traditional SPSA with the capability of adaptive optimization searching, it only utilizes information from adjacent iteration points in the process of optimization.

However, the information of adjacent iteration points is not the only information in the historical data. Therefore, this paper introduces a Knowledge-informed Historical Gradient-based Simultaneous Perturbation Stochastic Approximation (GK-SPSA) algorithm. During the optimization process, the algorithm emphasizes the significance of historical gradient information, employs it to steer the optimization process, and adjusts both the iteration direction and step size. In this research, the method is applied to the SGLCS of a nuclear power plant. The effectiveness and efficiency of the proposed method are systematically validated.

The structure of this paper is as follows: In Section 2, we describe the problem of optimizing the performance of the steam generator level control system. Section 3 provides an overview of the knowledge-informed optimization method, along with an explanation of the fundamental principles of the GK-SPSA method, including its development. In Section 4, we apply the GK-SPSA method to optimize the performance of the SGLCS, thereby validating the optimization efficiency of this method. We then present a comprehensive discussion based on the experimental findings. Finally, the paper concludes with a summary of the main findings.

2. Performance Optimization of the SGLCS

The parameter-setting process of the SGLCS is accomplished by modulating the feedwater flow to meet the set requirements for the current SG water level height [22,23]. The process of SGLCS parameter setting, which is also the framework for performance optimization, is illustrated in Figure 2.

The brief introduction of each part is as follows:

(1)

Optimization method

The optimization process leverages historical data generated throughout the iterative procedure to steer the optimization, thus improving efficiency and minimizing costs whenever feasible. In Section 3 of this paper, this method leverages historical gradient information to achieve adaptive adjustments of the optimization step size.

(2)

Performance evaluation

Selecting an appropriate method to assess the performance of the SGLCS is essential for establishing a closed-loop optimization method. Control system performance is frequently quantified in terms of control errors observed under specific disturbances. A typical step load change can be chosen as the test condition. A typical control evaluation index can be expressed as Integral Time Absolute Error (ITAE) [19,22]. However, the ITAE value can change significantly compared to the scale of parameter changes, leading to violent oscillations during the optimization process. Therefore, the revised evaluation index, ITAE (lg), is used in this paper, and the calculation formula is as follows:

$$\mathrm{ITAE}(\lg )=\lg {\displaystyle {\int }_0^{T}t\left|e(t)\right|dt}$$

(1)

where

$e(t)$ is the disparity between the observed water level and the desired water level, $t$ represents time, and

$T$ is the preset time interval designated for assessing the transient performance of the system.

3. GK-SPSA-Based Model-Free Optimization

This section comprises three subsections: In the first part, the concept and solution of hybrid process knowledge with SPSA-based model-free optimization are discussed and proposed to address the aforementioned challenges. The second subsection introduces the GK-SPSA method and compares it with the conventional SPSA optimization method. The third subsection provides a brief elucidation of the iterative termination criterion for achieving holistic closed-loop optimization.

3.1. The Fusion Method of Process Knowledge and MFO

Given the iterative aspect of quality control, the optimization procedure produces a sequence of historical iteration details, effectively encapsulating the process knowledge pertaining to specific batch processes. In conventional Model-Free Optimization (MFO) methods, this information is not fully leveraged. Taking the example of model-free optimization based on SPSA, each iteration relies solely on the current optimization state to search for the next iteration point.

By preserving the historical information generated during the optimization process and utilizing these data to evaluate and adjust the progress of optimization, this research integrates data-driven optimization methods using iterative information with a model-free framework, as illustrated in Figure 3.

This section starts from the idea of iteratively fusing knowledge in the optimization process and delves into a historical gradient approximation-based SPSA knowledge-indexing strategy. By employing this strategy, key information from the iterative process can be efficiently utilized to improve optimization efficiency. This paper presents a viable implementation mechanism, which is utilized to enhance the performance optimization efficiency of the SGLCS, as shown in Figure 4.

3.2. GK-SPSA Strategy

In the case of random error interference, the SPSA algorithm generates two perturbation points, one positive and one negative, at the current iteration point according to certain rules, and uses the two perturbation points and their corresponding index values to generate an estimated gradient, which guides the generation of the next iteration point, so as to continuously approximate the optimal point.

Due to the fixed step size mechanism, the SPSA algorithm has some problems, such as slow convergence, ease of falling into local optimization, and oscillation of the optimization process. Therefore, this research used the historical gradient information to adjust the step size of the SPSA algorithm to form a knowledge-guided parallel perturbation stochastic approximation method utilizing historical gradient approximations. In this method, the gradient of the next iteration point is compensated by the historical gradient information. The gradient information of the current iteration point and the historical gradient information are used as the method to judge the optimization process. The step size is adjusted according to the optimization direction determined by this method.

The gradient compensation rule is as follows:

$$\widehat{G}(X_k)={\rho }_k\widehat{G}(X_{k-1})+\left(1-{\rho }_k\right)G(X_k),\hspace{0.33em}\hspace{0.33em}\widehat{G}(X_1)=G(X_1)$$

(2)

where

$X_k$ is the kth iteration point,

$G(X_k)$ is the Current Simultaneous Perturbation Gradient Approximation (CSPGA) at $X_k$,

$\widehat{G}(X_k)$ is the kth compensated Composite Gradient Approximation (CGA) at the $X_k$,

${\rho }_k$ is the gradient compensation coefficient at the kth iteration point.

This rule uses ${\rho }_k$ to combine the current gradient information and historical gradient information to provide the improvement direction for the iteration step size of the next iteration point. At the same time, the gradient estimation compensation mechanism is used to improve the accuracy of gradient estimation, and then the search efficiency of the method is improved. Finally the optimization efficiency is improved.

In this algorithm, the current iteration point gradient information and historical gradient information are used to evaluate the optimization process, as shown in Figure 5.

The angle between CSPGA and current CGA is used as the Status Indicator (SI) for the optimization process, and the SI is used to determine whether the current optimization direction is consistent with the historical optimization direction.

If the directions of the two are consistent, the angle approaches zero. This indicates that the historical optimization direction and the current optimization direction are aligned, signifying that the optimization direction is correct. In this case, the step size is increased. Conversely, if the angle is close to $\pi$, it indicates that the current optimization direction deviates from the correct direction. In this case, the step size is decreased.

Using this method to adjust the step size based on the optimization direction guided by the current gradient information and historical gradient information, the specific adjustment formula is as follows:

$$d_k=a_k\times \left(\frac{R}{\pi k^{\omega }}\left(\arctan \left(2\pi \left(\cos \left(SI\right)-\frac{1}{2}\right)\right)\right)+1\right)$$

(3)

where

$d_k$ is the new iteration step size,

$R$ is the tuning coefficient for iteration step size ($0<R<2$),

$\omega$ is a coefficient that counts on the iteration number effects on $R$,

${\rho }_k$ is the gradient compensation coefficient at the kth iteration point.

In this formula, $\cos \left(SI\right)$ represents the cosine value of the angle between CSPGA and CGA. The range of $\cos \left(SI\right)$ is [−1, 1]. Its purpose is to determine whether the optimization direction is consistent.

$\left(\frac{R}{\pi k^{\omega }}\left(\arctan \left(2\pi \left(\cos \left(SI\right)-\frac{1}{2}\right)\right)\right)+1\right)$ limits the range of d_k correction to [1 − 0.5R, 1 + 0.5R], allowing the range of step size variation to be adjusted based on different application needs. Through this rule, the adjustment of the current step size is realized based on the historical gradient estimation information, guiding the optimization process to operate more efficiently.

The core concept of GK-SPSA lies in the feedback of historical information generated during the optimization process into the optimization procedure to advance the progress of the optimization process. Compared to the traditional SPSA algorithm, this method takes into account historical gradient information that is often overlooked in traditional SPSA, assessing the optimization state based on both the current gradient and historical gradient, and calculating the current optimization status indicator. A smaller SI indicates better consistency in the gradient direction between the current iteration point and the previous one. Therefore, using this SI allows for the adjustment of the iteration step size based on the estimated status of the current iteration point. The comparison of GK-SPSA and SPSA frameworks is shown in Figure 6, with differences in the algorithms highlighted in red text and outlined in red lines in the figure [20].

3.3. Iteration Termination Control

Traditional SPSA achieves iterative termination based on a fixed maximum number of iterations. To improve optimization efficiency and prevent the continuation of ineffective or inefficient optimization processes, it is crucial to analyze historical iteration information and promptly terminate the iteration process when necessary.

In the iteration termination control method, the optimization process is evaluated using the historical iteration points. By calculating the degree of smoothness in the optimization curve, we can determine whether the optimization is continuing. This method connects the historical iteration point data with the optimization process. By analyzing the historical data, the optimization process can be terminated and unnecessary optimization processes can be avoided.

The termination control steps for iteration can be briefly described as follows [19]:

• Step1: Historic iteration sequence updating. Visualizing the data corresponding to historical iteration points and the current iteration point lays the foundation for iteration termination control.
• Step2: Relatively optimality sequence updating. Because model-free optimization involves randomness, it can have a detrimental impact on the assessment of convergence. This step involves a simple ranking of the loss function values corresponding to historical iteration points, thereby creating a relatively optimal sequence. It eliminates the interference caused by randomness in the assessment of convergence.
• Step3: Smoothing tendency sequence updating. Conduct a global analysis of the optimization process using the relatively optimal sequence obtained in step 2. Update the iteration sequence through a smooth average method to better represent the optimization progress.
• Step4: Smoothing termination sequence updating. Using historical data from the iteration process, transform the iteration sequence obtained in step 3 into a smoothing tendency sequence that better reflects the optimization process trend.
• Step5: Differential control sequence updating. Convert the smoothing tendency sequence into a differential sequence and then assess the relative progress of the optimization process.
• Step6: Iteration termination factor calculation. Use the ratio of the improvement value at the current iteration point to the loss function value at the current iteration point as the iteration termination factor, and decide whether to terminate the iteration process based on the evaluation of this iteration factor. The primary objective of this study is to introduce the GK-SPSA algorithm into the performance-optimization process of the SGLCS. For a specific description of the iteration termination criterion, please refer to the reference [19]. The specific method is shown in Figure 7.

4. Results and Discussion

4.1. Experimental Setup

In real industrial processes, the Steam Generator Level Control System comprises the Steam Generator level process and its Distributed Control System (DCS) [22,23]. The DCS modifies the relevant settings of the controller using externally provided controller parameters and integrates them into the control system. Subsequently, the DCS collects operational data and forwards them to the performance evaluation system.

In order to maintain the research’s universality, the classical steam generator level control system model is used instead of the actual industrial process, and this research utilizes the SG mathematical model proposed by E. Irving as the simulation experiment model [7]. The controller in this simulation system comprises six controller parameters: the Principal Regulator (PR) Proportional-Integral-Derivative (PID) and the Auxiliary Regulator (AR) PID. This can be represented in this article as $X^{*}={[X_1,X_2,X_3,X_4,X_5,X_6]}^T$.

Table 1. Parameter feasible region [24].

Variable	Description	Feasible Range of Parameters
X1	kP of the PR	[0.077, 0.3]
X2	kI of the PR	[2.3 × 10−4, 2.3 × 10−3]
X3	kD of the PR	[−0.6, 2.65]
X4	kP of the AR	[1, 1.5]
X5	kI of the AR	[0.3, 0.8]
X6	kD of the AR	[0, 0.5]

provides a detailed description and constraint conditions for the six controller parameters [24].

4.2. Effectiveness Test

GK-SPSA adheres to the fundamental architecture of the SPSA method, thereby ensuring a convergence profile that remains congruent with SPSA. In its capacity as a stochastic search algorithm, the efficacy of GK-SPSA finds validation through its foundational design principles as well as rigorous numerical and statistical analyses. The widespread adoption and acknowledgment of this technique, both within academic circles and industrial applications, underscore its robustness and significance [21]. To ensure randomness, in the effectiveness test, the Monte Carlo method is used to randomly select two initial points in the parameter feasible region $X_1^{*}={[0.077,3\times {10}^{-4},0.2,1,0.5,0]}^T$ and $X_2^{*}={[0.1,3\times {10}^{-4},0,1.5,0.8,0.5]}^T$ [19]. To implement the optimization strategy of GK-SPSA, tests and analyses were conducted on two different initial points. The optimization processes for these two different initial points are presented in Figure 8, Figure 9, Figure 10 and Figure 11.

(1)

Trajectories of iteration

As can be seen from Figure 8, as the optimization process progresses, the ITAE consistently exhibits a significant decrease, indicating that the GK-SPSA optimization method can be applied to this optimization problem, and it also indicates a noteworthy enhancement in the performance of the SGLCS. Moreover, in the whole optimization process, the number of iterations of GK-SPSA is not always smaller than that of SPSA, and the ITAE index of GK-SPSA is not always smaller than that of SPSA, so it is necessary to conduct batch efficiency tests to test the optimization efficiency of GK-SPSA.

(2)

Trajectories of step size change

Analyzing the trend in Figure 9, with the increase in iteration times, the search step sizes of GK-SPSA and SPSA decrease continuously. This reduction effectively reduces the oscillations around the optimal value. In addition, small fluctuations in the GK-SPSA step size indicate that GK-SPSA is capable of evaluating the current state of the optimization process and adjusting the step size according to the current optimization state.

(3)

Parameter change trajectory

The adjustment of these essential controller parameters signifies the continuous enhancement of the control system’s performance. The controller parameter path is visually traced in Figure 10, revealing that, as the optimization process progresses, the controller parameters will continue to change dynamically, and the controller parameters at adjacent iteration points will not be the same. Throughout the iteration process, the parameters of each controller undergo changes. However, these changes are relatively minor, resulting in subtle variations in the parameters as depicted in Figure 10b.

(4)

Level change curve before and after optimization

Once the criteria for terminating the iteration process are met, the optimization procedure concludes promptly pinpointing an optimal control parameter configuration. As depicted in Figure 11, the transient response profiles of the SGLCS undergo a transformation throughout the optimization process, effectively illustrating the modifications in control system performance prior to and following optimization, thus substantiating the effectiveness of the optimization method. The enhanced response performance of the optimized control system stands as compelling evidence of the effectiveness of the GK-SPSA method.

4.3. Efficiency Test

In the efficiency test, the sequential Latin hypercube sample method is adopted for experimental sample extraction. Based on the original Latin hypercube sampling, this method adopts the sequential method, uses the information of existing sample points to guide the next sampling, and uses the Monte Carlo method to randomly select initial points in batches on each layer, so that the initial points can cover most of the space of the feasible region [19]. Under these initial points, evaluate the efficiency of the three optimization methods, SPSA, GK-SPSA and IK-SPSA, as shown in Figure 12 and Figure 13.

(1)

Comparison of the number of iterations

The comparison of the number of iterations of SPSA, GK-SPSA, and IK-SPSA is shown in Figure 12. First, the average iterations of SPSA are 20 times and the scatter points of the iterations are relatively concentrated, while the box lengths of GK-SPSA and IK-SPSA are both larger than SPSA and the mean iterations are smaller than SPSA. Note that both of these methods can improve the performance of SPSA. Secondly, the comparison between GK-SPSA and IK-SPSA shows that the average number of iterations of GK-SPSA is still smaller than that of IK-SPSA when both outliers exist, indicating that GK-SPSA can find the best advantage faster and reduce the experiment cost to a greater extent.

Due to the enhancements introduced by GK-SPSA and IK-SPSA to the original SPSA search methodology, both GK-SPSA and IK-SPSA exhibit a substantial reduction in iteration counts during the iterative process, when compared to the conventional SPSA algorithm. This improvement contributes to an accelerated optimization pace for GK-SPSA and IK-SPSA, showcasing the remarkable performance in the realm of optimization problems. Consequently, they offer a swifter and more reliable approach for approximating optimal solutions to intricate problems, thus providing a potent tool for tackling complex challenges.

(2)

Comparison of iteration final values

The comparison of the iteration final values of the three optimization methods is shown in Figure 13. As shown in Figure 13, the final iteration values of SPSA and GK-SPSA are similar and higher than for IK-SPSA. In addition, it can also be observed that there are some outliers that deviate from the mean value in GK-SPSA optimization. Even in this case, the overall box diagram is still flatter than for the SPSA method. This phenomenon shows that the stability of the overall value is superior. It can be concluded that the optimization results obtained by the optimization method are more reliable and less likely to cause abnormal conditions in the control system.

The situation depicted in Figure 13 arises because both GK-SPSA and IK-SPSA have enhanced the search methodology of the original SPSA, without altering the fundamental framework of SPSA. Therefore, it can be reasonably expected that these three sets of search results should exhibit similarity. By fine-tuning and optimizing the original SPSA method, GK-SPSA and IK-SPSA have made beneficial refinements to the search methodology while retaining the core framework, aiming to achieve improved search performance.

5. Conclusions

This study introduces the GK-SPSA optimization method under the model-free optimization framework, and it is used to optimize the performance of the SGLCS. This method integrates process knowledge with model-free optimization techniques, effectively leveraging the guiding role of process data in the optimization process, and fully utilizing historical information generated during the optimization process. First of all, in the effectiveness test, GK-SPSA, like SPSA, showed a decrease in ITAE value under limited iterations, representing an improvement in the performance of the control system. Secondly, compared with traditional SPSA and IK-SPSA in the efficiency test, the average number of iterations of GK-SPSA decreased by about 20% and 11.11%, respectively, and the average final value decreased by about 0.7% and increased by 2.67%, respectively. These data indicate that the GK-SPSA optimization method enhances the search speed of control system parameters with minimal or even slight reduction in control system performance. The experimental results indicate that this algorithm serves as a rapid method for parameter setting in the SGLCS of nuclear power plants, achieving swift convergence to optimal parameter settings. This algorithm possesses broad application potential and can be employed for optimizing controller parameters in various contexts, including nuclear power plant process control, such as turbine generator control and cooling system control. Additionally, it can also be applied to parameter-setting issues in batch processes, such as parameter setting in injection-molding processes and temperature control in food processing. This algorithm serves as an effective optimization tool for control systems in different domains and holds the promise of enhancing system stability, efficiency, and performance, thereby exerting a positive impact on both industrial and scientific fields.