A Knowledge-Informed Simplex Search Method Based on Historical Quasi-Gradient Estimations and Its Application on Quality Control of Medium Voltage Insulators

Abstract

Quality control is of great significance for the economical manufacturing and reliable application of medium voltage insulators. With the increasingly stringent quality control requirement, traditional quality control methods in this field face a growing challenge on their efficiency. Therefore, this study aims to achieve quality specifications by optimizing process conditions with the least costs. Thus, a knowledge-informed simplex search method was proposed based on an idea of knowledge-informed optimization to enhance the optimization efficiency. Firstly, a new mathematical quantity, quasi-gradient estimation, was generated following a reconstruction of the simplex search from the essence and the development history of the method. Based on this quantity, the gradient-free method possessed the same gradient property and unified form as the gradient-based methods. Secondly, an implementation of the knowledge-informed simplex search method based on historical quasi-gradient estimations (short for GK-SS) was constructed. The GK-SS-based quality control method utilized the historical quasi-gradient estimations for each simplex generated during the optimization process to improve the method’s search directions’ accuracy in a statistical sense. Finally, this method was applied to the weight control of a kind of post insulator. The experimental simulation results showed that the method is effective and efficient in the quality control of medium voltage insulators.

1. Introduction

Medium voltage insulators, which are widely used in electrical switches, electrical control devices, and other electrical equipment [1,2], are manufactured by the epoxy resin automatic pressure gelation process (APG) [3]. As the APG process parameters significantly impact product quality, insulators’ quality control is mainly achieved via the tuning of process parameters [4,5]. However, the APG process is a typical batch process with high complexity and nonlinearity [6]. The challenging task, developing quality control methods with high efficiency, is at high priority.

Traditional quality control methods for batch processes become increasingly inefficient to meet the stringent quality control requirement. The trial-and-error and the design of experiments (DOE) are suboptimal, time-consuming, and experience-dependent [7]. The model-based optimization (MBO) methods rely on reliable quality models, which cannot avoid model mismatch and is highly inefficient on model building. To address the above challenges, a systematic model-free optimization (MFO) framework for a type of batch processes with a short cycle time and a low operational cost was proposed [7,8,9]. In the MFO framework, direct experiments are utilized to substitute the role of the model in MBO. Hence, it can retain the high-efficiency optimization characteristics of MBO and simultaneously avoids high modeling costs. To reduce the experimental costs of MFO further, Zhao et al. proposed an iterative modeling and the trust-region optimization method (IMTO) by integrating the advantages of both the MFO and the MBO together [10]. Lu et al. proposed a model-free quality optimization method based on the natural gradient for the injection molding processes [11]. These methods significantly improved this kind of batch processes’ quality control efficiency. However, as the operational cost and the cycle time for a single batch of the APG are relatively higher, more efficient method is needed.

With the current emerging trend on data-driven technology, data-driven control or optimization is becoming a research hotspot. Spall and Dong et al. tried to revise the traditional model-free algorithms to enhance efficiency [12,13]. Hou et al. focused on the development of the data-driven strategy to improve the efficiency of model-free control (MFC) [14,15,16]. Xing et al. had proposed a series of knowledge-based algorithms and learning-guided algorithms [17,18]. In all the above works, knowledge generated during the optimization or obtained prior is utilized to promote method efficiency.

Based on a knowledge-informed idea, Kong et al. has proposed different kinds of knowledge-informed Simultaneous Perturbation Stochastic Approximation (SPSA) methods and applied them to medium voltage insulators’ quality control [4,5,19]. However, the quality control of medium voltage insulator is mostly a low-dimensional problem, while SPSA, as a gradient-based method, is more suitable for high-dimensional optimization problems. Another kind of gradient-free method, the simplex search method, which is especially effective for low-dimensional optimization, was concerned. As a popular algorithm, its efficiency has gained much attention. Researchers have tried to investigate it further and modify it in different forms [20,21,22,23,24,25]. These modifications, however, do not suitable for the scenarios of MFO-based quality control. By now, there are no modifications in the simplex search method from the perspective of using iterative optimization process knowledge. For the SPSA-based MFO, different kinds of knowledge generated during the optimization process have been extracted and utilized to improve its optimization efficiency. In this study, it was expected that the same knowledge-informed idea might be employed for the simplex search method. The development and the characteristics of the traditional simplex search method were detailed and analyzed. On the basis, a revised simplex search method, knowledge-informed simplex search based on historical gradient approximations (GK-SS), was proposed. As a method based on an idea of knowledge-informed optimization, the GK-SS integrates a kind of iteration knowledge, the quasi-gradient estimations, generated during the optimization process to improve the efficiency of quality control for a type of batch process with relatively high operational costs.

The remainder of this paper is organized as follows. Section 2 illustrates the quality control problem of medium voltage insulators. Section 3 presents an overview of the knowledge-informed optimization strategy, followed by the simplex search method’s basic principles. The development history of the traditional simplex search was detailed based on the idea of a knowledge-informed optimization strategy, and a revised knowledge-informed simplex search method based on historical quasi-gradient estimations (GK-SS) was proposed and delineated. In Section 4, the GK-SS method was further verified on the weight control of a kind of post insulator. A comprehensive discussion is carried out to assess GK-SS’s superiority over the SPSA and the traditional simplex search method based on the experimental results. Finally, conclusions were summarized.

2. Quality Control for Medium Voltage Insulators

In our previous paper (Referred to [4]), we had defined and illustrated the problem of quality control for medium voltage insulators in detail. According to the previous definition, we mainly achieve the medium voltage insulators’ quality control in this study by adjusting the APG process’s parameters. Accordingly, the quality control of the insulators was transformed into searching for the optimal process parameters.

The relationship between the quality response of medium voltage insulators and the process parameters is illustrated in Figure 1. The mathematical expression of the quality model is defined as below:

$$Q=f\left(x\right)+e,x={\left[x_1,\cdots x_n\right]}^T$$

(1)

where $Q$ is the quality response of the quality index to be controlled, $x$ is an n-dimensional vector for the APG process parameters, and $x_i\left(i=1,\cdots ,n\right)$ is the ith process parameters of $x$, $n$ is the dimension of $x$, $e$ represents the process uncertainty, while $f\left(x\right)$ is the internal mechanism between the quality index and the corresponding process parameters. As analyzed previously, the explicit expression of the quality model is usually unavailable.

In this study, quality control of finding the optimal process conditions can be summarized as below:

$$\begin{array}{ll}\underset{x}{\min }& Q_E=\left|Q-Q_t\right|\\ s.t.& R_i^L\le x_i\le R_i^H,i=1,2,\cdots ,n;\\ & x={\left[x_1,\cdots x_n\right]}^T.\end{array}$$

(2)

where $Q_E$, as the objective function, is the absolute quality error between the actual quality response $Q$ and the quality target $Q_t$. $R_i^{L}and{R}_i^H$ are the lower and upper bounds of $x_i$, respectively.

According to Equation (2), quality control of medium voltage insulators is essentially a nonlinear optimization problem with bounded constraints. The target of quality control is to attain the optimal process parameters within quality specifications at the least costs. Considering that the APG process’s quality model is usually unavailable, the model-free optimization framework (MFO) is supposed to be suitable for the quality control of such batch processes with run-to-run repetitive characteristics. The schematic diagram of APG’s quality control is illustrated in Figure 2. In the MFO framework, the quality evaluation is conducted online, which substitutes the quality model’s role in the MBO. With this closed-loop and online quality control strategy, medium voltage insulators’ quality specifications could be achieved iteratively. The detailed procedure of the MFO can be referred to [7]. This methodology initially sprung from considering the quality control for a type of batch process with a short cycle time and a low operation cost. However, the application scenario is slightly different in quality control of the medium voltage insulators, which has a relatively higher price for each batch. As discussed in [4], the APG’s quality control is sensitive to the iteration number. Thus, the iteration costs on quality control should be minimized to improve efficiency. In this study, the simplex search method has been revised to utilize historical iteration knowledge from an idea of knowledge-informed optimization to achieve the above goal.

3. Knowledge-Informed Simplex Search Methodology

3.1. Knowledge-Informed Optimization Strategy

Since the operational cost of the APG is relatively high, the number of batches on quality control of medium voltage insulators has a significant impact on the economy of the quality control process. The rapidity, therefore, is of paramount importance for the quality control of medium voltage insulators. It is necessary to reduce the number of iterative experiments in the quality optimization process as much as possible, thus reducing the cost of quality control. Even though in the optimization framework, as shown in Figure 2, the MFO method has excellent performance on quality control of a type of batch processes with lower operational cost. As the scenario is slightly different for the APG, the traditional MFO, however, faces the relatively high optimization price challenge.

Up to the present, there is no method superior to the MFO that exists to address this challenge. Considering that the MFO still has its advantages in the medium voltage insulators’ quality control, a possible way to promote quality control efficiency is to revise the conventional MFO method. Under the framework of MFO, the optimization method is the key. Consequently, it is critical to further enhance the optimization method’s efficiency under the existing optimization framework. During the optimization, a series of iterative process information, which contains process knowledge, will be generated dynamically. In traditional optimization methods, most of this knowledge is discarded or not used effectively. If this knowledge can be extracted and utilized effectively, it is possible to further enhance the optimization method’s efficiency. It is feasible to reduce the optimization number and improve the quality control efficiency for a type of batch process with relatively high operation cost by mining the process knowledge generated during the optimization. Based on the knowledge-informed idea, different knowledge-informed optimization strategies for SPSA had already been formulated. It was verified that appreciable improvement could be achieved.

The simplex search method, as a gradient-free optimization strategy, is different from SPSA. Even though the simplex search method’s principle is quite different from SPSA, the basic idea of the knowledge-informed optimization strategy for both of them is similar. Like SPSA, a certain amount of process knowledge will be generated during the iterative process of the simplex search optimization. The simplex search takes each simplex, which consists of n + 1 points, as an iteration point. It determines its current search direction and the step size only according to the current simplex information at each iteration. The knowledge of the historical sequential simplices, such as simplex sequences, centroids, and historical reflection points, is completely discarded. However, the historical knowledge can be utilized to enhance the optimization method’s efficiency if appropriate mechanisms could be built according to the characteristics of the simplex search. For the quality control of medium voltage insulators, this knowledge can be used to improve the simplex search method. Therefore, the knowledge-informed quality control strategy for the simplex-search-based MFO is promising. The framework of the knowledge-informed quality optimization via the simplex search method is shown in Figure 3. Stem from the above-mentioned knowledge-informed idea, this section discusses a kind of knowledge-informed simplex search method based on historical quasi-gradient estimations in-depth. A feasible implementation mechanism of the GK-SS strategy was put forward.

3.2. Basic Principles of Simplex Search Methodology

As a gradient-free multivariate optimization method, the simplex search method is a direct search method for nonlinear optimization. This method is formulated based on a concept of simplex, which is a geometric object that is a convex hull of n + 1 points—not lying in the same hyperspace—in n-dimensional Euclidean space $R^n$ (n is the dimension of the optimization problem understudy) [26]. This method’s origin can be traced back to the idea of evolutionary operation (EVOP), which was proposed by Box in 1957 [27]. Considering how EVOP might be automatic, Spendley et al. proposed the original idea of the sequential simplex search [28]. To further improve this method’s efficiency, Nelder and Mead made the simplex search method have the capability to adapt itself to the local landscape [29]. Due to its simplicity and effectiveness, the Nelder-Mead simplex search method becomes widely used in industrial quality improvement areas. As a result, the Nelder-Mead simplex search (called simplex search by default) is the baseline method in this project. The procedure of the simplex search method, as can be seen in Figure 4, is detailed as follows:

Step 1: Methodology initialization.

The initial process conditions are preset as $X_1$, which represents the 1st iteration point of the optimization. As each process condition has a different operating range, each is scaled into the same range [0, 100] to facilitate the optimization. The scaled initial conditions are represented as ${\overline{X}}_1$. Meanwhile, the coefficients of the methodology, $\left\{\alpha ,\beta ,\gamma ,\delta \right\}$, are set at this stage. $\alpha$ is the reflection coefficient, $\beta$ is the contraction coefficient, $\gamma$ is the expansion coefficient, $\delta$ is the shrink coefficient. Set the coefficients according to the suggestion of Nelder and Mead [29].

Step 2: Initial simplex construction.

As shown in Figure 5, a particular sequential perturbation strategy is proposed to construct a feasible initial simplex. Set the simplex iteration number s = 0, and set the first vertex $V_1^s={\overline{X}}_1$. The construction rule for other n vertices is expressed as below:

$$\{\begin{array}{l}{\overline{X}}_{{}^{k+1}}={\overline{X}}_{{}^1}+\tau e_k,{\overline{X}}_1\left(k\right)\le 50\\ {\overline{X}}_{{}^{k+1}}={\overline{X}}_{{}^1}-\tau e_k,{\overline{X}}_1\left(k\right)>50\end{array},V_{k+1}^s={\overline{X}}_{{}^{k+1}},k=1,\cdots ,n$$

(3)

where $e_k$ is a column vector, which has 1 in the kth element and zeros in the other elements, ${\overline{X}}_{{}^{k+1}}$ represents the (k+1)th iteration point of the optimization; $\tau$ is the perturbation coefficient that should be set according to the range of all the process conditions, which has a range of (5, 50). After the perturbation, the initial simplex is constructed as $V^s=\left\{V_1^s,V_2^s,\cdots ,V_{n+1}^s\right\}$. Experiments are then conducted to obtain the corresponding quality responses vector $F^s=\left\{F_1^s,F_2^s,\cdots F_{n+1}^s\right\}$, where F represents the function response for each simplex vertex.

Step 3: Simplex Sorting.

The simplex iteration number s is updated as s = s + 1. The vertices in the current simplex $V^s$ are sorted according to their corresponding quality responses $F^s$. The sorted simplex may be denoted as $V^s{}^{*}$, which has the following characteristics:

$$F_1^{s*}\le F_2^{s*}\le \cdots F_n^{s*}\le F_{n+1}^{s*}.$$

(4)

Thus, $V_1^{s*}$ is the vertex with the best response, $V_n^{s*}$ is the vertex with the next-to-the-worst response, $V_{n+1}^{s*}$ is the vertex with the worst response.

Step 4: Reflection.

The reflection operation would be carried out to obtain the reflection point $V_{ref}^s$ according to the following expression:

$$V_{ref}^s=(1+\alpha )V_c^s-\alpha V_{n+1}^{s*}.$$

(5)

$V_c^s$ is the centroid of all the vertices except $V_{n+1}^{s*}$, which is expressed as:

$$V_c^s=\left({\displaystyle {\sum }_{i=1}^n{V}_i^{s*}}\right)/n.$$

(6)

Experiment to evaluate the quality response $F_{ref}^s$ at the reflection point. If $F_{ref}^s<F_1^{s*}$, which means that the performance of the process conditions along the reflection direction may be promising, the expansion operation should be executed; hence, the procedure would go to Step 5. If $F_1^{s*}\le F_{ref}^s\le F_n^{s*}$, which means that the performance of the process conditions along the reflection direction may be neutral, the current reflection point should be accepted, and the worst vertex would be substituted by the reflection point; thus, the procedure would go the Step 8. If $F_{ref}^s>F_n^{s*}$, which means that the performance of the process conditions along the reflection direction may be gloomy, the contraction operation should be conducted, the procedure would go to Step 6.

Step 5: Expansion.

The expansion operation would be carried out to obtain the expansion point $V_{exp}^s$ according to the following expression:

$$V_{\exp }^s=(1-\gamma )V_c^s+\gamma V_{ref}^s.$$

(7)

Then, experiment to evaluate the quality response $F_{\exp }^s$ at the expansion point. If $F_{\exp }^s\le F_{ref}^s$, the expansion is successful, thus $V_{n+1}^{s*}$ would be replaced by $V_{exp}^s$. Otherwise, the worst vertex would be substituted by the reflection point. At last, the procedure would go to Step 8.

Step 6: Contraction.

The contraction operation could be categorized into two types, viz. (i) inside contraction, (ii) outside contraction. Generally, the contraction point $V_{ct}^s$ could be calculated according to the following formula:

$$V_{ct}^s=(1-\beta )V_c^s+\beta V_{\max /ref}^s.$$

(8)

where $V_{\max /ref}^s$, the reference point for contraction, maybe $V_{n+1}^{s*}$ or $V_{ref}^s$. The choice of $V_{\max /ref}^s$ depends on the contraction types.

Case 1: When $F_{ref}^s\ge F_{n+1}^{s*}$, this is inside contraction.

In this case, $V_{\max /ref}^s=V_{n+1}^{s*}$, $F_{\max /ref}^s=F_{n+1}^{s*}$; conduct experiments to evaluate the quality response $F_{ct}^s$. If $F_{ct}^s\le F_{n+1}^{s*}$, the contraction is accepted.

Case 2: When $F_n^{s*}\le F_{ref}^s<F_{n+1}^{s*}$, this is outside contraction.

In this case, $V_{\max /ref}^s=V_{ref}^s$, $F_{\max /ref}^s=F_{ref}^s$; conduct the experiments to evaluate the quality response $F_{ct}^s$. If $F_{ct}^s\le F_{ref}^s$, the contraction is accepted.

If the contraction is accepted in the above cases, replace $V_{n+1}^{s*}$ with $V_{ct}^s$ and go to Step 8. Otherwise, the contraction is refused, then go to Step 7 for a shrink operation.

Step 7: Shrink.

The shrink operation could be expressed as:

$$V_i^{s*}=\left(1-\delta \right)V_1^{s*}+\delta V_i^{s*},i=2,\cdots ,n+1.$$

(9)

All the vertices, except the best vertex, are sequentially updated and substituted by the new vertices generated based on Equation (9). The quality responses of the updated vertices are obtained and updated via experiments.

Step 8: Termination.

Since the simplex search is utilized for the quality control, the method would be terminated when the maximum iteration number is achieved or the quality response of the best vertex is satisfied within a preset quality tolerance.

3.3. Basic Ideas for Knowledge-Informed Simplex Search Methodology Based on Quasi-Gradient Estimations

Considering the genesis and the development process of the simplex search method, it, to some extent, can be regarded as a particular type of knowledge-informed strategy. Different knowledge fusion mechanisms accompany the development of the simplex search method. To achieve more effective integration of process knowledge with the simplex search method, it is necessary to attain some inspiration from its development. This method was initially proposed by Spendley et al. in 1962 [28]. Moreover, its origins can be dated back to Box’s EVOP strategy in 1955 [29]. The basic philosophy of the EVOP method is that the improvement of the industrial process is very similar to the biological evolution process. At the current working point, EVOP continuously sets a series of controlled, slight process parameter variants and observes their responses to find the direction of progressive improvement of the process parameters iteratively. Each advancement of EVOP relies on the knowledge of the current working point and a series of controlled variants at its vicinity, reflecting an intrinsic characteristic of a knowledge-informed mechanism. However, the final optimization operation is still determined by the operation engineers in combination with their experience. To realize the automation of the EVOP, Spendley devised a sequential simplex-design-based optimization methodology, which inherits the basic principle of the EVOP. Under Box’s EVOP methodology, the execution and the search direction of the optimization operation are determined by the operator or the manager. Hence, it is a hybrid optimization strategy relying on both operation knowledge and experience. However, with Spendley’s method, the optimization operation could be determined entirely by the current simplex. The optimization, therefore, gets rid of the constraint of the operator’s experience and becomes automatic and more efficient. However, the iteration of the simplices relies only on the reflection operations in Spendley’s method. The reflection point will substitute the worst vertex, discarded from the current simplex, to form a new simplex. It is a mirror-symmetric reflection that the size of simplices remains unchanged in the optimization process. From the perspective of iterative optimization, the size of the simplex determines the optimal step size. As a result, the optimal step size of the method is kept unchanged. Hence, this is a fixed step size optimization strategy, which seriously restricts its optimization performances. Considering Spendley’s method’s rigidity, Nelder and Mead proposed a revised simplex method that can adapt itself to the optimization’s local landscape. The Nelder-Mead method still follows the idea of sequential simplex design, but it adds additional operations, such as expansion and contraction. These additional operations are conducted depending on the local landscape’s different scenarios; thus, the methodology could adapt to the process landscape. Hence, the revised method becomes a variable-step-size method with the step size of the simplices that could be adjusted dynamically according to the state of the optimization process. This method could contract to the optimal settings more efficiently. The Nelder-Mead method’s significant improvement can be attributed to the deep mining and utilization of knowledge information in the current iteration simplex. This method essentially utilizes the relationships between the reflection points and their subsequent operation points and the present simplex vertices’ knowledge information to realize the step size’s dynamic adjustment. To sum up, the method’s efficiency attribute to the utilization of iteration knowledge and the simplex search method’s development process reveals that making full use of the historical information generated during the optimization process can effectively improve optimization efficiency. Then, from the perspective of using iterative process knowledge to improve the efficiency of MFO, where is the next improvement direction of the simplex search method?

From the idea of knowledge-informed optimization, the traditional simplex search method is reviewed. It does not make full use of the historical iterative information of the optimization process. Some other historical knowledge information generated during the iteration of simplices has been ignored before. Only part of the knowledge is utilized. All the knowledge can be classified into two types according to their relationships with the current simplex. The classification of the knowledge is shown in Figure 6. The first dimension represents the knowledge within the current simplex, while the second dimension contains the historical knowledge outside the current simplex.

The traditional simplex search method only utilizes the first dimension knowledge, and excellent optimization performance is achieved with an appropriate knowledge integration mechanism. Then, could the iteration knowledge in the second dimension be utilized to facilitate the optimization process? The answer must be yes. However, the critical question lies in how to dig, define, and finally use this iteration knowledge. The fusion mechanism of the iteration knowledge, including knowledge digging, definition, and utilization, is central to knowledge-informed strategy.

The genesis and essential characteristics of the simplex search method are analyzed to find sufficient knowledge that can be used to facilitate performance improvement. The simplex search method is widely regarded as a gradient-free algorithm. However, as pointed out by Spendley, the original EVOP method is essentially a rudimentary steepest descend optimization strategy that senses the steepest descend direction and climbs down. As a sequential-simplex-design EVOP strategy, Spendley’s method is a kind of steepest-descend method. Its direction is estimated from the simplex’s centroid through the hyperface. The hyperface is opposite to the vertex with the worst response. The advance direction is determined by a rough gradient estimation at the current simplex, as $d^s=-\nabla G^s$, in essence. For the Nelder-Mead method, the optimization direction is determined by the exact mechanism. The steepest descend direction, which deviates from the actual gradient, is determined solely by the current simplex knowledge. In a sense, this method is still not out of the gradient algorithm’s scope, which could be treated as a special case. The calculation strategy of the operations can be generalized to reveal the essence further. According to the basic principle of the simplex search, the reflection operation could be reviewed and correspondingly transformed into a different form:

$$V_{ref}^s=(1+\alpha )V_c^s-\alpha V_{n+1}^{s*}=V_c^s-\alpha \left(V_{n+1}^{s*}-V_c^s\right).$$

(10)

The expansion operations could be revised as below:

$$\begin{array}{ll}{V}_{\exp }^s& =(1-\gamma )V_c^s+\gamma V_{ref}^s=V_c^s-\gamma \left(V_c^s-V_{ref}^s\right)=V_c^s-\gamma \left\{\alpha \left(V_{n+1}^{s*}-V_c^s\right)\right\}\\ & =V_c^s-\alpha \gamma \left(V_{n+1}^{s*}-V_c^s\right)\end{array}.$$

(11)

The contraction operations could be revised as below:

$$V_{ct}^s=(1-\beta )V_c^s+\beta V_{\max /ref}^s=V_c^s-\beta \left(V_c^s-V_{\max /ref}^s\right)$$

(12)

Hence, the inside contraction and the outside contraction would be transformed into a similar form as below:

$$\begin{array}{ll}{V}_{inside\_ct}^s& =V_c^s-\beta \left(V_c^s-V_{n+1}^s\right)=V_c^s+\beta \left(V_{n+1}^{s*}-V_c^s\right),\\ V_{,outside\_ct}^s& =V_c^s-\beta \left(V_c^s-V_{ref}^s\right)=V_c^s-\beta \left\{\alpha \left(V_{n+1}^{s*}-V_c^s\right)\right\}\\ & =V_c^s-\alpha \beta \left(V_{n+1}^{s*}-V_c^s\right)\end{array}$$

(13)

From the above equivalent formula, except for shrink operation, other operations of the simplex search method attempt to find a new vertex in a unified way, which can be concluded as below:

$$V_{vertex\_new}^s=V_c^s-\xi \left(V_{n+1}^{s*}-V_c^s\right).$$

(14)

where $\xi$ can be viewed as the step size of the method, which can be adjusted dynamically according to the current optimization status estimated by the knowledge of the current simplex. According to the general settings, $\xi$ is set to {2, 1, 0.5, −0.5} corresponding to the expansion, reflection, outside contraction, and inside contraction. It is obvious that $\xi$, in fact, plays the role of the step size. According to the unified iteration strategy, it can be seen that the simplex search method has a mechanism similar to the gradient-based methods. Thus, the vector from the centroid to the worst point is defined as quasi-gradient estimation, which is a rough estimation of the gradient at the centroid of the simplex. The quasi-gradient estimation can be expressed as follows:

$${\tilde{G}}_c^s=V_{n+1}^{s*}-V_c^s.$$

(15)

Correspondingly, the iteration strategy of the simplex search method can be represented as below:

$$V_{vertex\_new}^s=V_c^s-\xi {\tilde{G}}_c^s.$$

(16)

Like the gradient-based method, each simplex during the optimization can be viewed as an iteration point. Hence, the simplex search method with rough gradient estimation behaves like a gradient-based method. Like the SPSA, although the quasi-gradient estimations at every simplex iterations are rough estimations, the method could statistically find the steepest descend route. Thus, its efficiency can be guaranteed. We had proposed a knowledge-informed SPSA based on historical gradient approximations (GK-SPSA) in our previous work. The basic idea of the GK-SPSA lies that if we can appropriately make use of the historical gradient estimation information, then the gradient estimations’ accuracy may be improved to enhance the efficiency of the optimization method. Considering their similarity, the iterative quasi-gradient estimations generated during the simplex search optimization may also be gathered and utilized. This knowledge certainly carries some degree of information on the gradients’ tendency that it may be used to compensate for the gradient estimations at each simplex iteration. Once the gradient estimations’ accuracy could be improved, the simplex search method’s efficiency would be enhanced accordingly. A schematic diagram of the gradient estimation compensation mechanism can be seen in Figure 7.

Based on the aforementioned knowledge-informed idea, a revised simplex search method, which incorporates the historical quasi gradient estimations to facilitate the optimization, was proposed. The different optimization mechanisms of the traditional simplex search and the revised method are illustrated in Figure 8. The revised method utilizes the historical quasi-gradient estimations to compensate for the search direction moderately at each simplex iteration. Like the knowledge-informed SPSA, the revised method was named knowledge-informed simplex search method based on historical quasi gradient estimations (GK-SS for short). This method is a typical knowledge-informed optimization strategy. An implementation scheme for the GK-SS is proposed and illustrated in detail in the following subsection.

3.4. An Implementation Scheme of Knowledge-Informed Simplex Search Method

According to the traditional simplex search method principles, GK-SS’s implementation scheme is formulated and detailed based on the above knowledge-informed idea. In this mechanism, the simplex operations should be performed along the compensated search direction, which is different from the traditional method.

The simplex search method’s iteration strategy was reformulated from the steepest-descent characteristics of the traditional simplex search just discussed above. An intermediate quantity that has an apparent mathematical meaning was revealed and formulated. This quantity represents a search direction determined by each simplex, which, in a sense, is the approximate gradient information of the current simplex. Thus, this quantity of the current simplex is called the estimated quasi-gradient for current simplex (EGCS). The definition of the EGCS could be expressed as Equation (15), where ${\tilde{G}}_c^s$ represents the EGCS at the sth simplex.

With the above formula Equations (15) and (16), we construct a kind of quasi-gradient estimation information equivalent to the gradient approximation of the gradient-based strategies for the gradient-free simplex search method. In the traditional simplex search procedure, this kind of information does not exist explicitly. After each simplex is formed and sorted in the GK-SS, the EGCS of the current simplex would first be calculated before the corresponding simplex operations are conducted. Then, it can be recorded into a sequence in an orderly manner. The EGCS sequence, which grows with the simplices’ iteration, stores the historical quasi-gradient estimations generated by each simplex.

The EGCS provides a potential link between gradient-free methods and the gradient-based methods. The EGCS, however, is a slightly rough estimation of the gradient at the current simplex due to the operation rule. To integrate the historical knowledge to compensate the gradient estimation accuracy at the current simplex, a new compensated composite gradient for the current simplex (CCG) is defined as follows:

$${\widehat{G}}_c^s={\rho }_s{\widehat{G}}_c^{s-1}+\left(1-{\rho }_s\right){\tilde{G}}_c^s,{\widehat{G}}_c^1={\tilde{G}}_c^1,$$

(17)

where ${\widehat{G}}_c^s$ is the sth compensated composite gradient (CCG) at the sth simplex, ${\widehat{G}}_c^{s-1}$ is the CCG at the (s−1)th simplex, ${\widehat{G}}_c^1$ is the CCG at the 1st simplex.

${\rho }_s$ is a gradient compensation coefficient at the sth simplex, reflecting the gradient information’s current weights. According to Equation (17), the CCG at the current simplex is determined by all the historical EGCS generated during the previous optimization process. The historical EGCSs of the simplices nearing the current simplex plays a more critical role for the CCG at the current simplex. Considering that the credibility of the EGCS of the neighboring simplex is higher than that far away from the current simplex, the proposed mechanism is reasonable.

On the other hand, according to the simplex search method’s characteristics, the accuracy of CCG will gradually improve with the simplex iterations’ development. Hence, the coefficient should not be a fixed constant. Contrarily, it should be a parameter that changes dynamically with the iteration of the simplices. Therefore, a dynamic mechanism for the coefficient can be formed as follows:

$${\rho }_s={\rho }_F-\frac{\Delta {\rho }_{init}}{s^{\tau }},$$

(18)

where ${\rho }_F$ is an upper limit that a CCG can be approached, which is set to 0.5 by default, $\Delta {\rho }_{init}$ is the initial deviation between ${\rho }_F$ and its lower limit, which is set to 0.2 by default, and $\tau$ is an exponential coefficient, which counts on the iteration number effects on the accuracy of CCG. The coefficient ${\rho }_s$ represents the tendency of the cumulative effect of the historical quasi-gradient estimations at each simplex on compensated gradients.

From Equation (17), the CCG at the sth simplex is a linear combination of the previous CCG at the (s−1)th simplex and the EGCS at the sth simplex. The historical gradient estimations generated by each simplex are integrated into the revised simplex search method through the above mechanism. The CCG is utilized to substitute the implicit EGCS, as shown in Equation (16), in the traditional simplex search method’s operations. The historical quasi-gradient estimations are gained iteratively and stored sequentially. The ratio relationship of the EGCS on the synthesis of the CCG at each iteration is showcased in Figure 9.

With the CCG, the original reflection operation, as shown in Equation (5), would be substituted as below:

$${\overline{V}}_{ref}^s={\overline{V}}_c^s-\alpha {\widehat{G}}_c^s,$$

(19)

where $\alpha$ is the reflection coefficient that is kept unchanged.

The expansion operation would be revised accordingly as follows:

$${\overline{V}}_{\exp }^s=(1-\gamma ){\overline{V}}_c^s+\gamma {\overline{V}}_{ref}^s={\overline{V}}_c^s-\gamma {\overline{V}}_c^s+\gamma \left({\overline{V}}_c^s-\alpha {\widehat{G}}_c^s\right)={\overline{V}}_c^s-\alpha \gamma {\widehat{G}}_c^s.$$

(20)

Correspondingly, the outside contraction is:

$${\overline{V}}_{oct}^s=\left(1-\beta \right){\overline{V}}_c^s+\beta {\overline{V}}_{ref}^s={\overline{V}}_c^s-\beta {\overline{V}}_c^s+\beta \left({\overline{V}}_c^s-\alpha {\widehat{G}}_c^s\right)={\overline{V}}_c^s-\alpha \beta {\widehat{G}}_c^s.$$

(21)

Moreover, the inside contraction is:

$${\overline{V}}_{ict}^s=(1-\beta ){\overline{V}}_c^s+\beta {\overline{V}}_{n+1}^s={\overline{V}}_c^s+\beta \left({\overline{V}}_{n+1}^s-{\overline{V}}_c^s\right)={\overline{V}}_c^s+\beta {\widehat{G}}_c^s.$$

(22)

The flowchart of the GK-SS is illustrated in Figure 10. As shown in the figure, except for the reflection, expansion, and contraction operations, the other operations and the GK-SS’s basic procedures are the same as the traditional method. It is natural since these operations are the key to the simplex search method. Therefore, the revised simplex operation could provide an appropriate mechanism for the fusion of historical simplex information.

This GK-SS scheme provides a feasible way from a gradient view, which is relatively peculiar for a gradient-free method, for the fusion of the knowledge generated during the optimization process. The primary mechanism of the GK-SS conforms to the GK-SPSA in the search direction compensation. Using knowledge of historical equivalent gradient information generating during the iteration of the simplices, more accurate search directions could be obtained during the optimization. Thus, the optimization efficiency would be enhanced.

4. Results & Discussions

A simulation platform based on a weight prediction model of a kind of post insulator is adopted to demonstrate the GK-SS’s performance on the quality control of medium voltage insulators. Based on the platform, a series of deliberately designed tests are conducted to verify the GK-SS’s performance and characteristics.

4.1. Simulation Experimental Setups

As a typical type of medium voltage insulators, a post insulator widely used in electrical engineering was employed to verify the revised knowledge-informed strategy, GK-SS. The post insulators investigated are made up of epoxy resin, a widely used material for the insulators. In this study, an epoxy molding compound with the brand of Duresco NU 5680 V was used. The prototype of the post insulator is shown in Figure 11. Part weight, as a critical quality indicator, was controlled. The target weight of the insulator was set according to the product specification requirements. Three different types were supposed to be used in the test. The weight target of Type A post insulator was set to 635 g, while Type B 620 g and Type C 860 g. According to the process characteristics and control requirements, a qualified post insulator’s weight bias tolerance is 0.5 g. According to the literature review and field investigation, four process parameters, including melt temperature, injection time, packing time, and packing pressure, have the most significant influence on post insulators’ weight. Thus, these four parameters were chosen as process parameters to be optimized, represented as X = [x₁, x₂, x₃, x₄]^T. The descriptions and the ranges of the parameters are listed in

Table 1. Process Parameters for Weight Control of a Kind of Post Insulator.

Process Parameters	Process Variables	Range	Units
Melt Temperature	x1	120–255	°C
Injection Time	x2	0.2–0.88	s
Packing Time	x3	3–10	s
Packing Pressure	x4	75–90	MPa

.

With the simplex-search-based quality control framework, as seen in Figure 3, quality control of medium voltage insulators relies on online experiments. However, the verification for the revised method needs a certain amount of sampling experiments, which may be costly if these experiments are to be conducted on the actual process. The surrogate post insulator platform, which was constructed based on a back-propagation artificial neural network (BP-ANN), was employed to advance the efficiency of method verification. Two main reasons are considered, viz. (i) As the key of the paper mainly focused on the effectiveness of the quality control methodology, the utilization of surrogate models with enough accuracy would not affect the validity of the verification for the quality control methods; (ii) the availability of the simulation platform could shore up a systematic test for the revised method. Taking advantage of the platform, a systematic sampling methodology can be conducted on the simulation platform to provide thorough test results at a low cost. Sufficient experimental results can provide a relatively reliable performance evaluation in statistics. The detailed information for the platform can be referred to [4].

4.2. Results & Discussions

4.2.1. Effectiveness Test of the GK-SS on Different Application Scenarios

The traditional simplex search method (called Traditional Method), which is a typical model-free algorithm of the MFO, is effective on quality control of batch processes. The GK-SS, however, as a revised strategy, still needs to be verified for its effectiveness. The tests at three different scenarios were demonstrated.

Quality Control from Different Initial Points

Firstly, to show GK-SS’s effectiveness under any initial conditions, this methodology was tested on quality control of medium voltage insulators from different initial points. The post insulator of Type A was employed. The part weight specification of the insulator was 635 ± 0.5 g. Without loss of generality, two initial points were randomly selected for the demonstration. The first optimization test was conducted from the first initial point X₁ = [181.33, 0.2, 8.74, 77.66]^T with an initial weight of 642.3 g. The quality target was attained with 15 iterations. The optimization trajectory was shown in Figure 12a. It can be seen that the weight of the insulator was adjusted gradually to achieve quality control. Another test from a different initial point, X₁ = [202.73, 0.84, 7.82, 81.87]^T with an initial weight of 628.9 g, was conducted. The trajectory of quality control was shown in Figure 12b with 22 iterations. As the two tests were started from different initial points, they had different optimization trajectories and different iteration numbers. However, under both two cases, the target of part weight was achieved effectively in a limited iteration number.

To exhibit the optimization trajectory characteristics of the GK-SS further, the iterative trajectories of all the four process conditions were demonstrated in Figure 13. The results with the initial point, X₁ = [202.73, 0.84, 7.82, 81.87]^T, were showcased. As the process conditions have different scales in physical quantities, to observe their iteration tendencies at the same scale, all of them were normalized, respectively, to a scaling percentage quantity of 0–100% according to their physical bounds. It can be clearly seen that the first five iterations were used for the initial simplex construction due to that the process parameters were perturbed one by one in accordance with the perturbation strategy that we proposed before. From the 6th iteration, the simplices iterated with the knowledge-informed strategy, along with each iteration perturbed in every dimension simultaneously. As the optimization process progressed, the parameters converged to the optimal parameter settings iteratively. It was shown that the GK-SS was effective in quality control of medium voltage insulators from different feasible starting points.

Quality Target Tracking Test under Type Transition

As a typical batch process, the post-insulators’ product specifications might frequently change due to the market demand shifting. The weight control methodology should have the flexibility to cope with type transitions when the market demand shifts. A weight setpoint tracking test was conducted to further demonstrate GK-SS’s effectiveness on quality target tracking with type transition. Type A insulator was employed with a setpoint of 635 ± 0.5 g. An initial point X₁ = [229.51, 0.87, 3.92, 83.74]^T was selected. The initial weight was 625.9 g, which deviated from the target. GK-SS was conducted. It can be seen that the method attained the target weight speedily. GK-SS used 15 iterations to obtain the optimal settings X_{opt_Type_A} = [182.6, 0.86, 3.25, 85.8]^T with the part weight of 635.3 g, which meets the current quality specifications. It was supposed that the type B insulator was required to meet the new market needs at a certain moment. The target weight of type B was 620 g. At the moment, type transition occurred. To achieve the new target, GK-SS was implemented. Weight tracking results could be seen in Figure 14. Utilizing the GK-SS, the weight of the insulator reached the new steady-state within 27 iterations. Finally, the new optimal settings, X_{opt_Type_B} = [255, 0.88, 4.74, 87.68]^T with a weight of 620 g, were gained. It was indicated that GK-SS was effective in quality setpoint tracking.

Quality Control under Disturbance

In the industrial environment, the APG process for the insulators’ production is susceptible to disturbances. When the process is affected by any disturbance, the steady state of the product weight control might be interrupted. The process parameters should be tuned to revert to the target. To test GK-SS’s effectiveness on the robustness of quality control, a disturbance signal was artificially introduced to simulate the disturbance scenario. Type A insulator was used in the test. The target weight of the insulator was 635 g.

When the quality of epoxy resin fluctuates, the weight of the post under the same process conditions will drift from the target. Without loss of generality, the part weight model was added a weight deviation by 20 g intentionally to simulate the weight offset scene. Subsequently, the weight of the insulator under the previous process conditions deviated from the target. The weight deviation could be seen in Figure 15. When the disturbance occurred, GK-SS was reactivated. It could be seen from Figure 15 that GK-SS returned the weight to the target within only ten iterations. It was indicated that the robustness of GK-SS to cope with weight disturbance was effective.

4.2.2. Efficiency Test of the GK-SS

The GK-SS was showcased to be effective on the quality control of medium voltage insulators from the above tests. As this knowledge-informed methodology was proposed to improve quality control efficiency, the critical question is whether the revised method could achieve the expected goal: to reduce the cost of quality control.

GK-SS vs. GK-SPSA

To verify the efficiency of the GK-SS relative to the GK-SPSA, the Latin Hypercube Sampling (LHS), which was a popular statistical method for generating a near-random sample of process conditions from a multidimensional distribution, was utilized. A sample of 100 different initial points was generated. The GK-SS and GK-SPSA were conducted on the sample, respectively, and the experiment results were recorded and analyzed accordingly. In the test, Type C with a part weight target of 860 g was employed. A randomly selected optimization result was demonstrated in Figure 16, which illustrated the typical characteristics of the GK-SS and GK-SPSA. The result showed that GK-SS achieved the goal of quality optimization more quickly than GK-SPSA. The statistical results could verify the efficiency of the GK-SS relative to GK-SPSA. Figure 17a illustrates the average iteration number of the GK-SPSA and GK-SS. It can be observed that the average iteration number diminished significantly by the GK-SS with a reduction of 43.91%. Figure 17b demonstrates the performance statistics on the superior, inferior, and equal ratios that the GK-SS relative to the GK-SPSA. It can be seen that the GK-SS behaved better in 69% of cases, while the GK-SPSA behaved better only in 25% of cases. The statistical results indicated that the GK-SS acted better than the GK-SPSA on medium voltage insulators’ quality control.

GK-SS vs. Traditional Simplex Search Method

The efficiency improvement of the GK-SS relative to the traditional simplex search method should be verified further. The quality control methods’ relative performance efficiency should be a statistical efficiency under the initial points of the whole feasible region. To compare the GK-SS’s performance efficiency and the traditional method, appropriate experiments should be deliberately designed to provide a group of sufficient, randomly distributed, and representative initial points for quality control simulation tests. The performance should be evaluated by estimating their comprehensive performance indicators under these tests. Under all test samples, the average number of iterations of the two methods and the improved method’s performance improvement ratio can be utilized as the performance evaluation indices for the GK-SS method.

In this framework, the quality control process was treated as a system, as illustrated in Figure 18. The system’s input is the optimization method to be tested, and the output is the response of the performance efficiency index. The representative initial points generated by the design of experiments, which are the controllable variables in this test framework, represent the process conditions’ feasible region. The LHS was adopted for generating a sampling of initial points. In LHS, a Latin square is a square grid in which there is only one sample in each row and column. For a problem with N variables, with each variable be divided into M equally intervals, there will be M^N hypercubes. However, with the LHS, only M independent samples are generated. With the help of LHS, the initial points in experimental design could satisfy the requirements of the characteristics of randomly distributed and representative. As the number of sampling points in a single LHS is relatively limited, the initial points’ coverage cannot be comprehensively guaranteed. To ensure the comprehensive coverage of the feasible region on the basis of the LHS method, a sequential LHS scheme was proposed. A series of LHS batches constitute this strategy. A single LHS batch is utilized to generate a basic unit of independent initial points with a fixed number. The performance index could be evaluated at each LHS batch. A series of LHS batches are conducted iteratively in a sequential way. With this strategy, the tendency of the performance index could be observed. Subsequently, the gradual change of performance evaluation index with the increase of LHS batches could be evaluated and analyzed. The conclusion on the performance efficiency could be given based on the changing trend and the stability of the performance indices.

In this test, the dimension of the process conditions of the initial points is 4. Consider adopting a reasonable level for the process conditions, each process condition was equally divided into ten levels. For a single LHS, 10 independent sampling points would be generated randomly. To evaluate the performance of an optimization run objectively, the iteration number of the transient process on quality control was selected as a critical index to show the method’s performance on a single run. The average iteration number will be calculated for an LHS batch to reflect the two methods’ performance on this batch. Furthermore, the superior, inferior, and the equal number of the quality control runs, that the GK-SS relative to the traditional method on the performance, is recorded.

A certain number of LHS batches will be sequentially conducted to ensure enough coverage on the initial points. In this test, the batch number of the sequential design was set to 100. That is a 100 LHS batch with 1000 runs, which provides a group of sufficient, randomly distributed, and representative initial points.

All the experiments were carried out under identical conditions, with only the difference in the initial points. The results of every LHS batches were observed. The dot diagram of the averaged iteration number for each LHS batch was shown in Figure 19a. It can be seen from the figure that the average iteration number per LHS batch varied considerably. Even though the GK-SS behaved inferior in some batches, it costed relatively fewer iterations in most cases. To evaluate their relative costs on iterations clearly, the box plot of the averaged iteration number at the two methods was illustrated in Figure 19b. It can be seen that the averaged iteration number was significantly decreased with the revised GK-SS.

The sequential LHS design provided an angle of view to observe the change of the performance indices and the increase of independent initial points. Figure 20a showed the trajectories of the accumulated average iteration number of the GK-SS and the traditional method. Unlike the above Figure 19a, this figure showed a clear tendency of the accumulated averaged iteration number, which was developed with more initial points of the feasible region covered. It can be seen clearly that the GK-SS costed relatively fewer iteration numbers than the traditional method. Figure 20b showed the trajectory of GK-SS’s accumulated reduction ratio on the iteration number. The reduction ratio of the GK-SS on the iteration number finally converged to about 10.56%. The results have shown that the GK-SS could reduce the traditional method’s optimization cost by about 10% in a statistical sense. The cumulative effect eliminates the randomness of single LHS batches and presents a more smooth performance accurately. It can be seen that in the initial LHS batches, due to the limited coverage of the total samples, there are some fluctuations in the trajectories of the above two cumulative indexes. With the proceeding of LHS batches, the number of samples covered increased gradually. Finally, the cumulative performance indices converged to a stable value. Under the cumulative effect, the average performance of quality control in the whole feasible region exhibited convergence characteristics, and the variability was thus eliminated. Through the sequential LHS scheme, the changing trend of the performance evaluation indexes of the two methods could be monitored dynamically. The statistical results can provide the basis for the performance evaluation of the two methods.

To sum up, the performance efficiency indices were demonstrated in Figure 21. Downward trends in iteration number statistics of the GK-SS relative to the traditional simplex search method suggested that the revised method is relatively efficient. Figure 21a illustrates the average iteration number with the two methods. It can be observed that the average iteration number diminished significantly, with a reduction of 10.56%. Figure 21b demonstrates the performance statistics on the superior, inferior, and equal ratios that the GK-SS relative to the traditional method. It can be seen that the GK-SS behaved not inferior to the traditional method in almost 71% of cases. Drawing a contrast, GK-SS acted better in of 48% cases, and the traditional method behaved better only in 28% of cases. Therefore, the proportion of the GK-SS behaved superior is 1.71 times that of the traditional method. Accordingly, the results showed that the knowledge-informed mechanism had an appreciable effect on the quality control methods, and the conclusion can be drawn that the GK-SS behaved better than the traditional method. Hence, it was indicated that the knowledge-informed mechanism is effective for the quality control of medium voltage insulators.

5. Conclusions

In this study, the idea of a knowledge-informed optimization strategy was extended to another kind of gradient-free algorithm—the simplex search method. The knowledge that the simplex search method generated during the optimization process was investigated. A retrospect on the development course of the simplex search from a view of knowledge utilization, which provides some general acquaintance with the method’s essence, was proposed. Furthermore, the simplex search method was reviewed as a type of steepest descent methodology that a new mathematical quantity-quasi-gradient estimation was constructed. Based on the simplex search principle and the knowledge-informed idea, a revised strategy based on the utilization of the historical quasi-gradient estimations, GK-SS, was proposed and formulated with a feasible knowledge fusion mechanism. Typical illustrative tests were conducted on a post insulator’s weight control to confirm that the GK-SS was effective and efficient. The experimental results have shown that the GK-SS could significantly reduce the effort required to attain quality control. The knowledge-informed simplex search can be investigated deeply on its theoretical analysis and the construction of more efficient knowledge fusion mechanisms. As a general optimization methodology, the idea and the general framework of the GK-SS can be extended to other similar industrial applications.