Particle Swarm Optimization–Long Short-Term Memory-Based Dynamic Prediction Model of Single-Crystal Furnace Temperature and Heating Power

Abstract

Precise temperature and heating power control are crucial for crystal quality and production efficiency in the Czochralski single-crystal growth process. Existing sensor technologies can only monitor these parameters in real time, lacking the ability to predict future trends, which limits the ability to implement preventive control before issues arise. To address this, a temperature and heating power prediction model based on Long Short-Term Memory (LSTM) is proposed and developed using extensive production data. Spearman’s rank correlation coefficient is applied to identify the key parameters related to temperature and heating power. Hyperparameter optimization uses Particle Swarm Optimization (PSO) to improve prediction accuracy. The performance of the PSO-LSTM model is compared with two other widely used prediction models, demonstrating its superior predictive capability. The results show that the PSO-LSTM model achieves highly accurate temperature and heating power predictions in the crystal growth process, with a Mean Absolute Error (MAE) of 0.0295 for temperature and 0.0392 for heating power, further validating its effectiveness for real-time predictive control.

1. Introduction

Single-crystal materials are critical in the development of the semiconductor field. In recent years, single-crystal silicon, the primary material for photovoltaic cells, has captured a significant share of the photovoltaic market [1,2]. The Czochralski process is a single-crystal growth technique first proposed by Czochralski in 1918 [3]. The method involves immersing seed crystals in molten material and slowly pulling them out under a controlled temperature and pulling speed, causing the material to solidify on the surface of the seed crystals. This process results in the growth of large-sized, high-quality single crystals and is widely used in fields such as semiconductors and optoelectronics. The Czochralski silicon single-crystal growth process exhibits considerable delays and complex nonlinear characteristics involving multi-physical field coupling. The single-crystal furnace relies on numerous sensors to realize the real-time monitoring of crystal growth conditions. The crystal pulling process involves complex heat exchange, so the temperature gradient of the melt inside the crucible is large and the temperature distribution inside the furnace is complex. As a key factor in temperature regulation, heating power directly affects the temperature distribution and melt stability inside the stove and plays a crucial role in crystal growth. In the process of pulling silicon single crystals, the regulation of heating power usually relies on experimental data and extensive production experience. It is automatically adjusted through a preset control trajectory. This method is characterized by being “static”, and making dynamic adjustments based on real-time data is difficult. In actual production, the growth process is affected by various factors, and operators often need to rely on their own experience to adjust manually, resulting in insufficient control accuracy and adaptability, thus increasing the possibility of error.

The temperature gradient is the driving force of crystal growth, and regulating heating power is one of the most important methods for controlling the thermal field. In a thermal field environment with multiple flow fields and phase transitions, achieving the ideal crystal pulling state through the precise control of parameters such as temperature and pulling speed is difficult. The crystal manufacturing process demands accurate control of the thermal field, and the heat exchange inside the furnace is remarkably complex; the real-time monitoring and precise regulation of temperature and heating power have become one of the difficulties in crystalline silicon manufacturing. Komperod and Lie [4] developed a Hammerstein–Wiener model for the CZ crystal growth process, which reveals the dynamics of the actual heating element power concerning the TRIAC input signal. Nguyen and Chen [5] found that the impact of different double-side heater designs with varying power ratios between the lower and upper heaters on crystal quality in silicon crystal growth. Several scholars have also researched specific issues surrounding heaters, heat transfer, etc. [6,7,8,9]. In recent years, with improvements in the level of production automation and intelligence, production enterprises have seen an increasing demand for the prediction of key parameters in the production process of silicon single crystal, with the primary focus being on diameter prediction. Ren et al. [10] proposed a hybrid modeling approach combining the data-driven and mechanism models. It utilizes the improved LSTM–Hammerstein–Wiener model to improve the accuracy of the Czochralski silicon single-crystal growth process and verify its effectiveness in diameter control. Zhang et al. [11] established a silicon single-crystal diameter prediction model based on the BP neural network. Wan et al. [12] proposed a data-driven prediction and control method based on the HVW-SAE-RF soft sensing model to precisely control the quality during the growth of Czochralski silicon single crystals. Zhao et al. [13] proposed a hybrid deep learning model combining DBN, SVR, and ALO for silicon crystal diameter prediction in the Czochralski process. The prediction accuracy of accurate data is better than that of BPNN and SVR models. In predicting other key parameters of silicon single crystals, Feng et al. [14] used a bidirectional long- and short-term memory network (Bi-LSTM) method to establish a dynamic prediction model for predicting silicon single-crystal furnaces’ pulling speed and verified the model’s validity through actual plant data. Wang et al. [15] proposed a lumped-parameter model based on boundary layer theory to predict oxygen concentration. Kutsukake et al. [16] proposed a machine learning model for predicting the interstitial oxygen (Oi) concentration in Czochralski-grown silicon crystals, enabling the real-time prediction of Oi concentration. Most of the existing studies focus on the prediction of crystal diameter. However, in the actual production process, the accurate prediction of temperature and heating power is still a key challenge to optimize the control of the crystalline silicon growth process, so this study focuses on the precise prediction of two key parameters: temperature and heating power.

This paper proposes a data-driven temperature and heating power prediction method that avoids many problems associated with traditional monitoring methods. By predicting temperature and heating power, the trend of parameter changes can be identified in advance to optimize the control strategy, avoiding the delay and error effects of conventional monitoring methods. LSTM, a kind of neural network with strong sequence modeling capability, plays a significant role in many timing prediction problems [17,18]. PSO performs a global search of hyperparameters to identify a combination of hyperparameter combinations that make the best model predictions [19]. PSO-LSTM performs better in corresponding to long-time series and nonlinear modeling compared to methods such as GRU and Transformer. GRU, although simple in structure, does not perform as well as LSTM in dealing with long-term dependencies and complex tasks. Transformer consumes more resources and is more suitable for some large-scale datasets. Therefore, PSO-LSTM can better meet the needs of the research.

Therefore, this paper proposes a multi-input, single-output PSO-LSTM prediction model to predict temperature and heating power separately. By anticipating the trends of temperature and heating power, the thermal field environment can be more precisely controlled to facilitate high-quality crystal growth.

The main contributions of this study are summarized below:

• Identification and selection of key parameters: Spearman’s rank correlation coefficients are used to identify the key parameters most strongly correlated with temperature and heating power, which are then selected as inputs for the prediction model.
• Design of the PSO-LSTM prediction model: A temperature and heating power prediction model based on PSO and LSTM is proposed. PSO optimizes the hyperparameters of the LSTM model, significantly enhancing prediction accuracy and providing an effective solution for forecasting the future trends of temperature and heating power during the growth process of Czochralski silicon single crystals.
• Development of a real-time monitoring and control platform: A real-time monitoring platform for silicon single-crystal production is proposed. It can update key parameter changes in real-time and provide fault alerts for abnormal parameter fluctuations. The platform provides support for the monitoring and control of the production process, aiding in the enhancement of automation and intelligence in crystal growth.

2. Foundation for the Construction of the Prediction Model

The heater power must be regulated in real time according to the crystal growth state in the crystal-pulling process. A standard method to control the heater power is to adjust the current or voltage to form a suitable thermal field environment for crystal pulling. The prediction model is based on an actual silicon single-crystal growth control system to improve the study’s practical application value, as shown in Figure 1.

Figure 2 shows a schematic diagram of the heat exchange within the crystal growth facility, where different forms of heat exchange are intricate, directly leading to a large temperature gradient inside the crucible. The figure is a simplified schematic that roughly illustrates the temperature distribution inside the crucible. The intricate thermal field environment makes it challenging to analyze and obtain the temperature using mechanistic modeling methods accurately, and it is also challenging to provide a reference for the regulation of the heating power following this.

During the crystal growth process, the crystal length increases gradually. Heating power is among the most critical process parameters regulated by the thermal field. The energy source of single-crystal silicon growth is the energy difference between the solid–liquid transition. Assuming that the latent heat of the crystallization of the crystal material is L and the mass of the crystal growth is m per unit time, the energy that the solidification of the silicon solution can generate is $Q$, the heat transferred to the solid–liquid interface through the melt is $Q_l$, and the heat released from the solid–liquid interface to the outside world is $Q_s$.

$$Q=mL$$

(1)

$$Q=Q_s-Q_l$$

(2)

During crystal growth, the dissipation path of the latent heat of crystallization is constantly changing, meaning that $Q_s$ is continually evolving. In the case of crystal growth, to maintain a constant diameter, the unit time m has to be kept at a constant value, so, to ensure that the equation holds, $Q_l$ needs to be regulated. It can usually be considered that $Q_l$ is proportional to the heating power P.

$$Q_l=kP$$

(3)

Therefore, the heating power P can be adjusted in real time to ensure it is compatible with the crystals’ stable growth.

The experimental environment uses Windows 10, Intel (R) Core (TM) i7-9750H, 16 GB of RAM, and GeForce GTX 1660Ti. Python 3.11 is used. TensorFlow 2.16.1 is used for model development and training. During the study, the PSO process for hyperparameter optimization took a long time, but this process was only performed during the model development phase and therefore had no impact on the real-time requirements in production. Compared with the hyperparameter optimization process, the LSTM training speed is extremely fast, and in practical applications, the time for calling the pre-trained model to predict new data in a single prediction is so short that it is almost negligible. Therefore, this design not only meets the requirements of real-time prediction but also ensures the accuracy and reliability of the prediction, which enhances the value of practical applications.

3. Characteristic Parameter Identification and Processing

3.1. Spearman-Based Parametric Correlation Analysis

Spearman correlation analysis is a non-parametric method based on ranking and has broad applicability. In a nonlinear process, Spearman’s correlation coefficient performs better than other algorithms in assessing the correlation between variables [20]. In determining the relationship between the variables, the method of calculating the degree of inter-correlation between their rankings is used. The calculation formula is

$$\rho =1-{\displaystyle \frac{6\sum d_i^2}{n\left(n^2-1\right)}}$$

(4)

where $\rho$ is the correlation coefficient, $d_i$ is the difference in each pair of variable rankings, and n is the number of samples.

The correlation coefficient is calculated by collecting statistics on the data of 16 characteristic variables in the single-crystal furnace production data, converting them into rankings, and calculating their differences for each pair of variable rankings. The results of the correlation analysis, shown in Figure 3, illustrate the degree of correlation between different feature variables through the correlation coefficient matrix heatmap, where red represents positive correlation and blue represents negative correlation. The shade of the color indicates the strength of the correlation. The number in each cell indicates the correlation coefficient between the two variables. The correspondence between numbers and names in Figure 3 is shown in

Table 1. Correspondence between feature name and identifier.

Identifier	Parameter Name	Identifier	Parameter Name	Identifier	Parameter Name
A1	Temperature	A7	Crystal length	A13	Heating current
A2	Crystal lifting speed	A8	The whole bar pulling speed	A14	Heating voltage
A3	Main chamber pressure	A9	Crystal diameter	A15	Heating power
A4	Crucible lifting speed	A10	Fusion lumens	A16	Crystal weight
A5	Crystal lifting position	A11	Secondary chamber pressure
A6	Crucible lifting position	A12	Furnace wall thermometry

, and the data for each characterized variable are shown in

Table 2. Characterization variable data.

Time	A1	A2	A3	A4	A5	A6	A7	A8
11-09 22:01	1348	1.17	1530.14	0.158	1.8	−39.46	1.8	1.154
11-09 22:02	1347.6	1.07	1530.14	0.136	2.9	−39.3	2.9	1.154
11-09 22:03	1347.3	1.19	1529.07	0.155	4	−39.15	4	1.146
11-09 22:04	1347	1.18	1532.3	0.166	5.2	−38.98	5.2	1.159
11-09 22:05	1346.7	1.38	1529.07	0.181	6.5	−38.8	6.5	1.184
11-09 22:06	1346.5	1.36	1530.14	0.186	7.5	−38.67	7.5	1.193
11-09 22:07	1346.2	1.33	1531.22	0.18	9.1	−38.45	9.1	1.206
11-09 22:08	1345.9	1.39	1531.22	0.185	10.5	−38.26	10.5	1.232
…	…	…	…	…	…	…	…	…
Time	A9	A10	A11	A12	A13	A14	A15	A16
11-09 22:01	228.1	8.33	1391.53	1261.1	1.243	34.74	43.18	5.81
11-09 22:02	228.1	8.07	1415.94	1260.2	1.247	34.72	43.3	5.93
11-09 22:03	228.3	7.78	1403.74	1259.1	1.244	34.74	43.22	6.04
11-09 22:04	228.3	7.52	1407.8	1258.1	1.246	34.72	43.26	6.13
11-09 22:05	228.8	7.25	1415.94	1257.2	1.245	34.73	43.24	6.27
11-09 22:06	229	7.07	1420.01	1256.7	1.246	34.73	43.27	6.38
11-09 22:07	229.4	6.82	1399.67	1255.7	1.246	34.73	43.27	6.52
11-09 22:08	229.7	6.67	1391.53	1254.9	1.246	34.71	43.25	6.68
…	…	…	…	…	…	…	…	…

.

The characteristic parameters are ranked according to the correlation strength, redundant parameters are eliminated, and parameter selection is performed. For temperature prediction, the correlation coefficients of parameters such as crystal position, crystal length, and crystal weight are equal in value and exhibit the same distribution in the scatter plot, indicating a powerful linear relationship between these parameters. As shown in Figure 4, by plotting the scatter plots of crystal weight versus temperature and crystal length versus temperature separately, it can be observed that there is redundancy between the two parameters, crystal length, and crystal weight. Therefore, removing the redundant parameters and choosing one as a feature variable is necessary. Based on the ranking of the correlation strength between each characteristic parameter and the target variable (temperature or heating power), the parameters with stronger correlations are prioritized as model inputs. By comprehensively analyzing the efficiency and accuracy of the model prediction, six characteristic parameters with high correlation with temperature (fusion lumen, furnace wall thermometry, main chamber pressure, secondary chamber pressure, crystal rise position, and heating power) are selected as model inputs. The most relevant features for heating power prediction are the main chamber pressure, the secondary chamber pressure, the furnace wall thermometry, and the crucible lifting speed. By optimizing these parameters, accurate predictions of temperature and heating power can be achieved.

3.2. Data Standardization

Preprocessing the data information is necessary to reduce the impact of the magnitude and numerical differences between different features on the model training [21], while simultaneously enhancing the speed of model convergence. The min-max normalization method is used to deflate the data to within [0, 1], and the data are converted to a scalar form without units, which can make it easier to analyze and deal with the feature parameters between different units or magnitudes.

$$\overline{x_i}={\displaystyle \frac{x_i\left(k\right)-\min \left\{x_i\left(k\right)\right\}}{\max \left\{x_i\left(k\right)\right\}-\min \left\{x_i\left(k\right)\right\}}} i=\mathrm{0,1},\cdots ,m$$

(5)

where $\overline{x_i}$ is the normalized processing result, $x_i\left(k\right)$ denotes the original data, $\max \left\{x_i\left(k\right)\right\}$ denotes the maximum value of the feature parameter data, and $\min \left\{x_i\left(k\right)\right\}$ denotes the minimum value of the feature parameter data. In this paper, the standardized feature parameters are used as model inputs, and the predicted values are used as model outputs.

4. Theory of LSTM and PSO Optimization Methods

4.1. Theoretical Foundation of LSTM Models

LSTM is a deep learning algorithm based on a Recurrent Neural Network (RNN), and it has improved on this, making it more effective in processing sequence data [22,23,24]. Due to the innovation of the storage unit gating mechanism, LSTM is more efficient in capturing and processing long-term dependencies in sequences.

The structure of the LSTM model is shown in Figure 5.

The forget gate accomplishes partially targeted ignoring.

$$f_t=\sigma \left(W_f·\left[h_{t-1},x_t\right]+b_f\right)$$

(6)

where $f_t$ is the output of the forget gate, $\sigma$ denotes the sigmoid activation function, $h_{t-1}$ denotes the output of the previous time step, $x_t$ denotes the input at the current time step, $W_f$ denotes the weight matrix of the forget gate, and $b_f$ denotes the bias term of the forget gate.

The input gate is used to update the cell state.

$$\left\{\begin{array}{c}{i}_t=\sigma \left(W_i·\left[h_{t-1},x_t\right]+b_i\right)\\ {\tilde{c}}_t=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(W_c·[h_{t-1},x_t]+b_c)\end{array}\right.$$

(7)

where $i_t$ is the output of the input gate, ${\tilde{c}}_t$ is the candidate cell state, $W_i$ and $W_c$ are the weight matrices of the input gate and the candidate cell state, respectively, and $b_i$ and $b_c$ are the bias terms applied to the input gate and candidate cell state, respectively.

The cell state of the previous layer is multiplied element-wise with the forget vector.

$$c_t=f_t{\mathrm{☉}c}_{t-1}+i_t\mathrm{☉}{\tilde{c}}_t$$

(8)

where $c_t$ is the cell state at the time step and $c_{t-1}$ is the cell state at the previous time step.

The output gate determines the amount of content from the cell state that is passed to the hidden state at the current time step.

$$\left\{\begin{array}{c}{o}_t=\sigma \left(W_o·\left[h_{t-1},x_t\right]+b_o\right)\\ h_t=o_t\mathrm{☉}\tanh \left(c_t\right)\end{array}\right.$$

(9)

where $o_t$ represents the output of the output gate, $h_t$ denotes the output at time step t, and $W_o$ and $b_o$ are the weights and bias parameters of the output gate.

The sampling time of the single-crystal furnace sensor is roughly 60 s, i.e., the temperature and heating power values of the output are obtained once every minute on average. The empirical lag time, by choosing different lag times for testing, is found to have a better prediction effect when the lag time t = 8 min. The prediction accuracy and efficiency are considered comprehensively, and the data of the past eight time steps are used to predict the parameter values at the next moment. A sliding window is utilized to achieve this function, and the schematic is shown in Figure 6.

In LSTM algorithms, parameter selection plays a certain degree of influence on the training performance of the model. Manually selecting or setting the hyperparameters of the LSTM model is easily influenced by personal subjective judgment and entails a high degree of randomness, meaning that the selected optimal hyperparameters often lack general applicability. Through the application of PSO, the hyperparameters of the model can be dynamically adjusted, effectively avoiding the problem of reduced prediction performance caused by the improper selection of hyperparameters [25]. The minimization of the MSE is used as an evaluation index to find a set of optimal combinations of hyperparameter settings. In the process of building the temperature prediction model, the particle swarm algorithm is used to optimize the learning rate and the number of neurons in the hidden layer within the specified range.

To quantify the model’s prediction effect, the prediction results are analyzed using the MSE, MAE, and the coefficient of determination (R²). Both MSE and MAE measure the size of the model’s prediction error, whereas MSE penalizes larger errors more strongly. Therefore, MSE is more sensitive to outliers. MAE provides an intuitive understanding of the magnitude of errors. R² measures the model’s ability to explain the target variable’s explanatory ability and can assess the goodness of fit. The correlation formula is as follows:

$$\mathit{MSE}={\displaystyle \frac{1}{m}}\sum _{i=1}^m{(y_i-\widehat{y_i})}^2$$

(10)

$$MAE={\displaystyle \frac{1}{m}}\sum _{i=1}^m\left|(y_i-\widehat{y_i})\right|$$

(11)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^m{(\widehat{y_i}-y_i)}^2}{{\sum }_{i=1}^m{(\overline{y}-y_i)}^2}}$$

(12)

where $y_{\mathrm{i}}$ denotes the actual value, $\widehat{y_i}$ denotes the predicted value, $\overline{y}$ denotes the mean value, and m represents the sample size.

4.2. PSO-Based Hyperparameter Optimization

The PSO algorithm is a population intelligence optimization algorithm [26,27]. The PSO algorithm simulates the behavior of birds foraging [28]. The core concept is to find the best solution by fostering collaboration and information exchange among individuals in the group [29]. In each iteration, speed and position are adjusted according to the group’s best solution, and the process continues until the best result is achieved [30]. By dynamically adjusting the learning rate, the particle swarm algorithm can explore the search space flexibly, thus increasing the probability of converging to the global optimal solution. The flowchart of the particle swarm algorithm is shown in Figure 7.

Amid a search problem in D-dimensional space, N particles are introduced to perform this task. As an independent individual, each particle has its position and velocity. Among them, the information of its position can be represented by X_id = {x_i1, x_i2, …, x_iD} and the data of its velocity can be represented by V_id = {v_i1, v_i2, …, v_iD}. The motion of a particle is influenced by its historical optimal position and the group optimal position, where the historical optimal position of the ith particle is P_id,pbest = {p_i1, p_i2, …, p_iD} and the global optimal position jointly found by all particles is P_d,gbest = {p_1,gbest, p_2,gbest, …, p_D,gbest}. To facilitate the measurement of each particle’s performance, its optimal fitness value during the current search is usually recorded. The historical optimal fitness of an individual is denoted by f_p, while the optimal fitness of the whole population is denoted as f_g.

The algorithm progressively updates the velocity and position by evaluating the optimal solution states between particles. Updating the velocity is a key step in balancing global and local search. It tracks the historical optimal solution and converges towards the best solution for the population. The formula for updating the velocity is as follows:

$$v_{id}^{k+1}=\omega v_{id}^k+c_1{r}_1\left(P_{id,pbest}^k-x_{id}^k\right)+c_2{r}_2\left(P_{d,gbest}^k-x_{id}^k\right)$$

(13)

The position update formula is as follows:

$$x_{id}^{k+1}=x_{id}^k+v_{id}^{k+1}$$

(14)

where i represents the particle’s index, d denotes the dimensionality, and k refers to how many iterations have occurred. By performing a position update, each particle gradually moves toward the direction of the solution direction and seeks a better solution in the entire search space, where the inertia weight ω can measure the degree of the particle inheriting its original velocity. Meanwhile, c1 and c2, as individual and global learning factors, reflect the tendency of the particle to converge to its historical optimum and group optimum during the updating process. The introduced r1 and r2, with values in the interval [0, 1], can add randomness to the search, which is convenient for the particles to form a dynamic balance between the local optimum and the global optimum and prevent the algorithm from converging prematurely. $v_{id}^k$ denotes the velocity vector of particle i in the dth dimension in the kth iteration, $x_{id}^k$ denotes the position vector of particle i in the dth dimension in the kth iteration. $P_{id,pbest}^k$ denotes the historical optimal position of particle i in the dth dimension in the kth iteration, and $P_{d,gbest}^k$ denotes the historical optimal position of the population in the dth dimension in the kth iteration.

The range of the number of neurons and the range of the learning rate are based on empirical references derived from experiments while balancing the computational efficiency and predictive performance of the model. Compared with grid search, PSO has higher computational efficiency and does not need to traverse a large number of hyperparameter combinations, hence resulting in a better computational cost. Genetic algorithms have more complicated parameter settings and slower convergence speeds, and in some cases, the optimization process is unstable, so genetic algorithms are also not identified as a suitable hyperparameter optimization method in this study.

The pulling crystal production data are preprocessed and divided into datasets. The ratio of training, validation, and test sets is 8:1:1. By using the early stopping method, the performance of the validation set is monitored, and training is stopped in time to prevent overfitting during the training process.

In the PSO algorithm, the number of particles is set to 20, and the number of iterations is set to 100. Usually, larger inertia weights facilitate global search and smaller weights facilitate local search. To improve the efficiency and accuracy of the PSO algorithm, this paper makes improvements to the setting of inertia weights. During the iteration process, the ω for the particle speed update decreases linearly from 0.9 to 0.4. The values of the c1 and c2 are both assigned as 2. The search ranges of the settings are as follows: the number of neurons in the hidden layer ranges from 10 to 400, and the learning rate ranges from 0.00001 to 0.1. Each particle will keep updating its location according to the settings and maintain a certain amount of inertia. The population will explore the optimal hyperparameters within the defined search area. The process of hyperparameter optimization using the PSO algorithm is shown in Figure 8, where the degree of adaptation decreases continuously with the continuation of iterations, and the hyperparameter optimization for temperature and heating power converges 54 times and 47 times, respectively, to reach a steady state.

The optimal hyperparameter combination is evaluated using MSE as the performance optimization index during iteration. After 100 iterations, the final hyperparameter optimization results are as follows: the number of neurons in the temperature prediction model is 171, and the learning rate is 0.00399. For the heating power prediction model, after the same optimization process, the final hyperparameter combination consists of 112 neurons and a learning rate of 0.00001. After several tests, the results of hyperparameter optimization are always similar, which can further verify that the selected hyperparameters are reasonable. Accurate temperature and heating power predictions can be achieved through the hyperparameter optimization of both models.

5. PSO-LSTM Prediction, Verification, and Integration of the Monitoring System

5.1. PSO-LSTM-Based Prediction

The advantage of the PSO-LSTM algorithm is that it can quickly find the global optimal solution with high prediction accuracy by leveraging the PSO algorithm’s global search capability and LSTM’s time series prediction capability [31,32].

Figure 9 shows the integrated flowchart of PSO-LSTM, illustrating the processing method that combines the PSO algorithm with the LSTM network. The PSO algorithm enhances the model’s training efficiency and prediction performance by optimizing the LSTM network’s hyperparameters, making the LSTM more efficient and accurate in processing sequential data.

This paper uses the PSO-optimized LSTM model to predict the temperature and heating power of the single-crystal furnace in the crystal-pulling workshop. The prediction results are shown in Figure 10. Figure 10 shows the prediction results of the PSO-LSTM model for temperature (Figure 10a) and heating power (Figure 10b). The horizontal axis of the figure represents the time, and the vertical axis represents the temperature and heating power values. The actual values closely match the predicted values of the PSO-LSTM algorithm, and the PSO-LSTM model can better capture the temperature and heating power trend. The trends of the predicted and actual values are almost the same, and the error values and error distributions are small.

Table 3. Predicted performance on the test set.

Targets for Projections	MAE	MSE	R2
Temperature	0.0295	0.0013	0.9710
Heating power	0.0392	0.0024	0.9836

presents the results for each performance metric on the test set.

Figure 11a shows the error histogram of the PSO-LSTM model in temperature prediction, in which the absolute value errors of all the samples are controlled within 0.15 °C. The frequency of errors between −0.05 °C and 0.05 °C exceeds 82.4% of the total samples. This indicates that the difference between most samples’ predicted and actual values is relatively small, and the prediction errors are more concentrated. The histogram shows a roughly symmetrical distribution, indicating that the PSO-LSTM model has no significant systematic bias in the prediction process, the errors are well-controlled, and the prediction performance meets the expected requirements.

The results show that the error for heating power prediction is also well controlled, and the error distribution is shown in Figure 11b. The accuracy of the data is controlled to one decimal point, according to the requirements for heating power control in actual production. In the actual prediction results, the absolute value errors of all samples do not exceed 0.2 kW, and more than 99% of the data samples have prediction errors within 0.1 kW. Therefore, the PSO-LSTM model can accurately predict the heating power and meet the control requirements. The model can also guide the regulation of heating power in actual production and assist operators in decision-making control.

Figure 12 shows the prediction errors of temperature and heating power for both the training and test sets, presented in box plots. The prediction model performs well on the training and test sets, indicating that it is well trained. It is worth mentioning that, in the power prediction, the heating power regulation fluctuates significantly at the beginning. According to the standard convention in time series prediction tasks, the training and test sets are usually divided chronologically. Therefore, in the prediction results, the training set error is slightly larger than the error distribution of the test set. However, the overall performance is still excellent, meeting the practical production requirements. The difference between the errors of the model on the training and test sets is not significant and the errors are at a low level, so the model generalizes well and no overfitting occurs.

5.2. Verification of the Superiority of PSO-LSTM

In the field of time series forecasting, a variety of models have been developed by different experts, such as SVR, MLP, and so on, which can be applied to the time series prediction task. There are differences in the prediction performance of different models. Heating power prediction is used as an example to evaluate the advantages of PSO-LSTM in predicting single-crystal furnace temperature and heating power, thereby improving the reliability of model evaluation [33]. The superiority of the PSO-LSTM model is assessed by comparing the prediction performance of four models: PSO-LSTM, LSTM, SVR, and MLP.

A comparison of the prediction accuracies of the PSO-optimized LSTM model (PSO-LSTM) and the LSTM model reveals that both have excellent prediction results. At the same time, PSO-LSTM can more accurately predict the trend of heating power and have a better prediction effect than LSTM. Prediction also uses the SVR and MLP, and the related performance indexes are recorded. The predicted and actual value data are shown in Figure 13. Since PSO-LSTM, LSTM, and MLP errors are relatively similar on the graph, the SVR error is considerable and gradually deviates from the trend of the actual value in the test set. To show the differences more clearly in prediction performance among the models, we present scatter plots of the prediction results of each of the three models—PSO-LSTM, LSTM, and MLP—on the test set. These scatter plots clearly illustrate the different prediction accuracies of each model. In the scatter plot, it can be seen that the more concentrated the samples with error values between −0.1 kW and 0.1 kW, the better the prediction performance, while the prediction error of the PSO-LSTM model is more minor, with the vast majority of the samples being located between −0.1 kW and 0.1 kW; the error of the LSTM model is also more minor, but there are still some error points exceeding the set −0.1 kW to 0.1 kW. The prediction error of MLP has a significant difference compared with the first two models. The prediction error is more extensive and more scattered. This demonstrates that the PSO-LSTM model delivers the most accurate prediction performance.

Table 4. Analysis of the prediction effect of each model.

Prediction Model	MAE	MSE	R2
LSTM	0.0554	0.0050	0.966
MLP	0.0725	0.0072	0.951
PSO-LSTM	0.0392	0.0024	0.984

and Figure 14 show the comparison of the prediction performance of each model, combined with the analysis in Figure 13; among the three models of LSTM, SVR, and MLP, the LSTM has the best effect, with an MSE of 0.005, while the SVR has the largest MSE of 1.3224. The comparison of the prediction results between the PSO-optimized LSTM model (PSO-LSTM) and the LSTM model shows that both of them have terrific prediction performance. Comparative analysis indicates that PSO-LSTM can predict the heating power trend more accurately than LSTM and that the prediction effect is better. After the optimization of hyperparameters by PSO, the MSE of PSO-LSTM is 0.0024, which is reduced by 52% based on the MSE of the LSTM model’s predictions, which proves that the prediction performance has been significantly improved after PSO optimization.

5.3. Experimental Verification

To verify the PSO-LSTM model’s effectiveness in temperature and heating power prediction during the crystal pulling process, we predicted the data for one complete isodiameter stage in actual production. The pre-trained PSO-LSTM model is loaded using the Keras library of Python to predict the temperature and heating power, and the prediction outcomes are contrasted with the actual values. The prediction outcomes are displayed in Figure 15, and the related performance metrics are recorded in

Table 5. Performance evaluation metrics in experiments.

Targets for Projections	MAE	MSE	R2
Temperature	0.1292	0.0234	0.9427
Heating power	0.0801	0.0111	0.9074

.

As shown in Figure 15, the predicted values in the experiment fit well with the actual values, proving that the model’s generalization performance is excellent. Overall, the PSO-LSTM model performs well in the task of predicting the temperature and power in the isodiameter stage of the single-crystal furnace, and the model can accurately predict the temperature and power values in the future moments, which provides strong support for the optimal control of the single-crystal furnace process. In future research, data will be collected under various working conditions. The training set will be extended, and the generalization ability of the model will be further improved. Meanwhile, we will explore better model structures and training strategies to enhance prediction accuracy further.

5.4. Predictive Monitoring System Development and Integration

The growth and fabrication of silicon single crystals are developing in the direction of intelligence. Among the various areas of research, growth process monitoring has become a key focus in today’s crystal growth control system.

We are working to advance the integration of the research results with actual production processes to develop a new type of monitoring platform for silicon single-crystal growth and manufacturing. The platform can realize fault detection, real-time sensor data updating, and critical parameter prediction. For example, under normal conditions, the parameters vary within certain intervals. When the predicted value derived from the model exceeds that range, this abnormality can be recorded and displayed through the platform. Operators can take regulatory measures before the abnormal condition occurs, thus improving the product quality of monocrystalline silicon.

With further development, the platform can be enhanced to monitor, predict, and control the entire silicon single-crystal production process, supporting the automation and intelligent upgrading of the entire manufacturing system.

6. Summary

6.1. Conclusions

In this paper, a data-driven prediction methodology is proposed to address the limitations of existing production equipment in measuring key parameters such as temperature and heating power during the growth of Czochralski silicon crystals. Although current equipment can estimate these parameters in real time, it cannot predict their future trends, making it impossible to take corresponding control measures before abnormalities occur. To address this issue, this study establishes a multi-input, single-output (MISO) PSO-LSTM prediction model for temperature and heating power prediction. Through analysis and validation, the model is shown to have significant advantages in predicting these two parameters. The specific conclusions are as follows:

• The correlation between each characteristic parameter and temperature and heating power is analyzed using Spearman’s rank correlation, and the characteristic parameters with the highest correlation with temperature and heating power are determined;
• The PSO-LSTM model was established to predict temperature and heating power, and the model showed high accuracy in predicting both variables;
• Taking MSE minimization as an indicator, the PSO-LSTM model has been proven to have the highest prediction accuracy, which verifies the superiority of the PSO-LSTM model in predicting this task;
• A real-time silicon single-crystal production monitoring platform is proposed, which can dynamically update key parameters and provide fault alarms. In the future, this platform will realize the comprehensive monitoring, prediction, and control of the silicon single-crystal production process and promote upgrading production intelligence and automation.

In summary, by proposing the PSO-LSTM model and integrating it with the production real-time monitoring platform, this study successfully overcomes the limitations of the existing equipment regarding the lack of prediction and control optimization for key parameters. This method can optimize the production process, provide specific scientific guidance for workers’ parameter regulation, reduce the production defective rate, and support the further development of automation and intelligence in this industry.

6.2. Limitations and Future Work

Although the PSO-LSTM method was successfully applied in this study to predict the temperature and heating power during the growth of silicon single crystals and good prediction results were achieved, this study still has some limitations and areas for improvement, mainly concerning the generalization of the method.

This can be summarized in two ways: firstly, whether the method can be extended to predict other crystals or substances, and secondly, whether it is compatible with different heating methods. Different crystalline materials have distinct physical and chemical properties, and the production environment and conditions vary to some extent. Regarding the heating methods, resistive heating and inductive heating differ significantly in principle, and this paper is based on resistive heating. However, since the PSO-LSTM method is data-driven, it can be applied as long as the appropriate data are available by making some simple adjustments to the parameters and structure of the model. However, the prediction performance may vary to some extent, and its effectiveness needs to be verified in more scenarios.

This study will be continuously improved, validated, and promoted in more crystal materials and different heating methods. It will also be combined with actual production to realize the real-time monitoring, prediction, and control of the silicon single-crystal manufacturing process, further improving its practical application value.