Monitoring of Temperature Measurements for Different Flow Regimes in Water and Galinstan with Long Short-Term Memory Networks and Transfer Learning of Sensors

Abstract

Temperature sensing is one of the most common measurements of a nuclear reactor monitoring system. The coolant fluid flow in a reactor core depends on the reactor power state. We investigated the monitoring and estimation of the thermocouple time series using machine learning for a range of flow regimes. Measurement data were obtained, in two separate experiments, in a flow loop filled with water and with liquid metal Galinstan. We developed long short-term memory (LSTM) recurrent neural networks (RNNs) for sensor predictions by training on the sensor’s own prior history, and transfer learning LSTM (TL-LSTM) by training on a correlated sensor’s prior history. Sensor cross-correlations were identified by calculating the Pearson correlation coefficient of the time series. The accuracy of LSTM and TL-LSTM predictions of temperature was studied as a function of Reynolds number (Re). The root-mean-square error (RMSE) for the test segment of time series of each sensor was shown to linearly increase with Re for both water and Galinstan fluids. Using linear correlations, we estimated the range of values of Re for which RMSE is smaller than the thermocouple measurement uncertainty. For both water and Galinstan fluids, we showed that both LSTM and TL-LSTM provide reliable estimations of temperature for typical flow regimes in a nuclear reactor. The LSTM runtime was shown to be substantially smaller than the data acquisition rate, which allows for performing estimation and validation of sensor measurements in real time.

1. Introduction

The performance of nuclear reactors could be enhanced through automation of monitoring tasks for the early detection of incipient signs of failure in the reactor monitoring system data streams [1,2,3]. Temperature sensing is one of the most common types of measurements in a reactor monitoring system [4]. A nuclear system typically has hundreds of temperature sensors, the most common of which are thermocouples. When repeatedly exposed to high temperatures and ionizing radiation, thermocouples are prone to drift and de-calibration. The challenges in the detection of sensor fault include uncertainties and noise inherent in the physics of measurements, as well as fluctuations in sensor readings due to normal transients in the fluid. One approach to system monitoring is through development of a model, which typically involves the numerical solution of a nonlinear system of differential equations [5,6]. An example is the development of virtual sensors through a computational fluid dynamics solution of Navier–Stokes equations [7]. However, the model-based detection of sensor faults is difficult to accomplish because exact knowledge of a complex system, such as a nuclear power plant, is required [1,2]. On the other hand, data-driven methods learn directly from experimental observations without any prior knowledge of the system.

A number of supervised and unsupervised machine learning (ML) strategies for sensor fault detection in nuclear power plants (NPPs) have been proposed recently. A review of the literature can be found in [1,2,3]. Some specific methods include anomaly detection and discrimination with Hotelling’s T² test and interquartile range (IQR) method [8], principal component analysis (PCA) [9], enhanced singular value decomposition (ESVD) [10], and correlation analysis and deep belief networks [11]. In addition, there have been studies specifically focused on NPP thermocouples using deep learning [12] and SVD [13]. In particular, ML-based strategies have been proposed for the detection of thermocouple sensor failures from time series analysis [14,15], which include strategies based on recurrent neural networks (RNN) [16], and a special case of RNN, long short-term memory (LSTM) networks [17,18]. In general, LSTM networks are highly efficient in predicting the evolution of dynamic systems because they contain “feedback loops” within their structure [19]. LSTM’s analysis of time series has been benchmarked in the literature [20,21]. Recent studies have investigated LSTM’s performance in various applications, including nuclear reactor control [22], fiber-optic sensor monitoring [18,23,24], failure detection and remaining useful lifetime estimation [25,26], detection of wind turbine blade icing [27], short-term forecasting of solar energy systems [28,29], and water distillation [30,31].

In this paper, we develop thermocouple monitoring LSTM models trained on the same sensor, and through transfer learning (TL-LSTM) by training on a correlated sensor [11]. In transfer learning (TL), a pre-trained model is utilized for different applications [32]. This is advantageous for NPP sensor monitoring when limited data for model training are available, such as for the monitoring of newly installed sensors. Recent work has explored TL in conjunction with convolutional neural networks (CNNs) [33], and in conjunction with LSTM networks for time series forecasting [34], pandemic modeling [35], and hydrology [36].

We study the accuracy of LSTM and TL-LSTM prediction of a nuclear-grade type K thermocouple sensor in water and Galinstan fluids for different flow regimes. In the envisioned nuclear energy application, ML methods developed in this work are used for monitoring the same type K thermocouple sensors installed in high-temperature sodium, molten salt, or water fluids. Water and Galinstan fluids, which are used for LSTM model validation in this paper, have been shown to be good surrogates for sodium when performing thermal hydraulic experiments, possessing very similar density and viscosity to within an order of magnitude [37,38]. We investigate the sensor monitoring accuracy dependence on the flow rate because, in nuclear applications, depending on the reactor power state, the same thermocouple sensors will experience different types of coolant fluid flow regimes. During start-up and shut-down, when the reactor is subcritical, the coolant fluid velocity in reactor core channels is relatively low. During full-power operation when the reactor is critical, the fluid velocity in core channels is higher.

In this study, the accuracy of time series prediction is evaluated with the root-mean-square error (RMSE), which is compared to the uncertainty of thermocouple sensor fluid temperature measurements. For the development of TL-LSTM for water and Galinstan loops, we identify correlated sets of sensors by calculating the Pearson correlation coefficient for the sensor time series [11]. We show that LSTM and TL-LSTM perform similarly in one-step-ahead prediction of thermocouple time series. Flow regimes can be characterized by the nondimensional Reynolds number (Re) [39]. We show that the RMSE of LSTM prediction increases linearly with the value of Re for both water and Galinstan fluids. However, the values of RMSE as well as all errors in the time series in each test segment are smaller than the sensor measurement uncertainty. Deriving the correlation between RMSE and Re allows the estimation to be made of the range of flow regimes for which LSTM provides a reliable estimation of temperature. Benchmarking of the LSTM performance shows that one-step-ahead predictions can be performed in real time, i.e., faster than sensor data acquisition, which is limited by a sensor’s time constant.

This paper is organized as follows. Section 2 describes temperature and flow rate measurements in water and Galinstan flow loops. Section 3 discusses the development of LSTM and TL-LSTM networks using training data from the same sensor and training data from a correlated sensor. Section 4 and Section 5 discuss the results of LSTM and TL-LSTM monitoring of thermocouple time series in water and Galinstan loops, respectively. Section 6 contains a discussion of the main results. Section 7 concludes the paper and discusses future work.

2. Data Collection in Thermal Hydraulic Flow Loop

A thermal hydraulic test flow loop was assembled and instrumented with temperature and flow sensors [18,40]. The loop allows temperature sensor time series to be monitored for different flow regimes, with data from all sensors recorded with a data acquisition system with the LabVIEW^TM interface. A schematic diagram of the loop is shown in Figure 1. The loop was constructed with polycarbonate pipe sections, which have an operating temperature range between 0 °C and 121 °C. In one set of experiments, the loop was filled with water, and in another separate set of tests, the loop was filled with liquid metal Galinstan. Water and Galinstan are frequently used for proof-of-principle studies of fluid sensing at room temperature [18,40]. Galinstan (Ga 68.5%, In 21.5%, and Sn 10%) is a liquid metal at room temperature, which has a melting point of −19 °C. Galinstan has a higher density and thermal conductivity than water, but lower heat capacity.

Fluid in the loop was circulated clockwise against gravity with a variable-speed 1.5HP stainless circulation pump. A split in the main loop created “hot leg” and “cold leg” flows. Fluid pumped through the “hot leg“ was heated with a variable power Watlow FLC-16 heater, with a maximum power rating of 4 kWe. Fluid in the “hot leg” was eventually mixed with the “cold leg” fluid in a thermal Tee. The inner diameters of the “hot leg” and “cold leg” pipes were 0.75 in and 1.5 in, respectively. After mixing, fluid in the main loop was brought back to room temperature with a 130,000 BTU Shell & Tube standard heat exchanger, which was coupled to a 11,300 BTU/hr chiller.

Temperature measurements in the flow loop were performed using type K thermocouples, which are commonly found in nuclear applications because of their relative radiation hardness. This thermocouple, which is rated for operation at temperatures up to 1100 °C, consists of two dissimilar metallic contacts enclosed in a protective metallic sheath. The positive contact is Chromel^TM (nickel and chromium alloy) and the negative contact is Alumel^TM (nickel, manganese, silicon, and aluminum alloy) (Concept Alloys Inc., Whitmore Lake, MI, USA). The protective sheath, which is in contact with high-temperature fluid, is made of corrosion-resistant stainless steel or nickel alloy. The accuracy of measurements with the type K thermocouple is given as max (±1.1 °C, 0.4%). For fluid temperature ranges in this study, the thermocouple measurement uncertainty was ±1.1 °C. The flow was measured using Turbine Blancett B110-500-1/2 flowmeters, which operate at temperatures up to 177 °C. The accuracy of flow measurements is ±1.0% of the reading and the repeatability is ±0.1%.

Flow rate and temperature data were sampled using sensors at different points throughout the loop. Locations of the sensors are indicated in the diagram in Figure 1. Seven thermocouples are labeled as TC-1 through TC-7 and two flowmeters are labeled as FT-1 and FT-2. Correlated sensors (discussed in Section 3.1) are highlighted with the same color in Figure 1 (color online). Temperatures of the “cold leg” were measured with thermocouples TC-1 and TC-2, in the “hot leg” with thermocouples TC-6 and TC-7, and in the mixing zone and downstream with TC-3, TC-4, and TC-5. Thermocouples TC-6, TC-2, and TC-3 were located 14.75 in, 5.5 in, and 17.75 in, respectively, from the thermal mixing tee. The mass flow rate in the main loop and “cold leg” was measured with two volumetric flow meters FT-1 and FT-2, respectively. In this study, the ranges of operating temperatures in the “hot leg” (TC-6) and “cold leg” (TC-2) were approximately 30 °C and 50 °C, and 20 °C to 30 °C, respectively. The fluids were maintained at ambient pressure.

Temperature and flow rate transients were generated by varying the heater power and pump speed, respectively, from within the LabVIEW^TM interface. Transient temperature and flow rate data were collected in separate experiments with water and liquid metal Galinstan fluids, consisting of time series of 2100 and 1470 points, respectively, which were sampled every second. Measurements from all sensors were collected concurrently.

3. Development of LSTM Networks for Temperature Sensors Validation

3.1. Identification of Correlated Sensors

To identify redundancies in sensor measurements, we calculated the Pearson correlation coefficient to determine the pairwise similarity between different sensor time series. For two signals x, y, the Pearson correlation coefficient r is defined as [41]

$$r\left(x,y\right)=\frac{{{\displaystyle \sum }}_{i=1}^N(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{(x_i-\overline{x})}^2}\sqrt{{{\displaystyle \sum }}_{i=1}^N{(y_i-\overline{y})}^2}}$$

(1)

where x_i, y_i are the measurements of signals x and y at time i, respectively; N is the total number of samples; $\overline{x}$, $\overline{y}$ are the mean values for each signal. The coefficient r takes on a value in the range of −1 ≤ r ≤ 1. The similarity between two signals increases as the value of the coefficient approaches 1. Identical signals produce a Pearson correlation coefficient r = 1. Correlations were calculated between respective temperature sensors and flow sensors. Values of Pearson correlation coefficients calculated for sensors in water and Galinstan loops are shown in Figure 2a,b, respectively.

Our analysis showed that for both water and Galinstan loops, correlated sensor sets were {TC-1, TC-2}, {TC-3, TC-4, TC-5}, {TC-6, TC-H}, and {FT-1, FT-2}. Correlated sensors are indicated with the same colors in Figure 1. Thus, the basis set (linearly independent sensors) for this loop consisted of [TC-2, TC-3, TC-6, FT-1].

3.2. Development of LSTM Networks

In this work, we focused on the validation of temperature sensor TC-3, which was located downstream of the mixing zone. LSTM networks were developed with the MATLAB Deep Learning Toolbox. Sensor time series were subdivided into training (80% of data) and validation (10% of data) segments for the development of the LSTM, and test (10% of data) segments for which the LSTM was used to perform monitoring/validation function. This is a common data partitioning ratio for machine learning of time series [42]. The structure of the LSTM neural network used for this study is shown in Figure 3.

The network is comprised of an LSTM layer, a fully connected layer, and a regression layer. The one-step LSTM layer is able to learn the dependencies between the time steps in the sequence data. For both water and Galinstan fluids, we determined that the best performance of the LSTM network for all sensors was achieved for a number of hidden nodes in that layer equal to 15. The state activation function for the LSTM cell was specified as tanh and the gate activation function was specified as sigmoid. The solver selected for this problem was Adam and the number of epochs was 250. When training the LSTM, we used the loss function of the root-mean-square error (RMSE), which is defined as

$$\mathit{RMSE}= \sqrt{\frac{{{\displaystyle \sum }}_{i=1}^N{\left(x_n-{\widehat{x}}_n\right)}^2}{N}}$$

(2)

where x_n is the actual data, ${\widehat{x}}_n$ is the forecasted data, and N is the total number of points.

An LSTM unit cell is comprised of the forget, input, and output gates [19,20]. The structure of an LSTM cell is shown in Figure 3b. The forget gate (f) makes a decision whether to retain previous information or not. Equation (3a) gives the expression for the forget gate:

$$f_t{=\mathit{sigmoid}(x}_t·U_f{+H}_{t-1}·W_f)$$

(3a)

where $x_t$ is the current input, $U_f$ is the weight corresponding to that input, $H_{t-1}$ is the hidden state of the previous step, and $W_f$ is the weight matrix for that state.

The input gate (i) is described by Equation (3b):

$$i_t{=\mathit{sigmoid}(x}_t·U_i{+H}_{t-1}·W_i)$$

(3b)

The updated state of the LSTM cell is described by Equation (3c):

$$C_t=f_t·C_{t-1}{+i}_t·\mathit{tanh}(x_t·U_c+H_{t-1}·W_c)$$

(3c)

The activation function used to express the information that passes through the cell state is tanh. The subscripts i and c indicate relevance to the input gate and to the updated state, respectively.

The output gate (o) is described by Equation (3d):

$$o_t=\sigma \left(x_t·U_o+H_{t-1}·W_o\right)$$

(3d)

The updated hidden state of the LSTM cell is shown in Equation (3e):

$$H_t=o_t·\mathit{tanh}\left(C_t\right)$$

(3e)

The accuracy of sensor prediction with LSTM was evaluated with RMSE calculated over the test segment. In one approach, we studied the prediction of TC-3 using the LSTM developed with training and validation on TC-3 data in water and Galinstan. Because TC-3 and TC-4 were highly correlated in both water and Galinstan (r = 0.99 according to Figure 2), we investigated the transfer learning prediction of TC-3 with the TL-LSTM network trained and validated on TC-4 data.

3.3. Selection of Time Series Test Segments

Flow regimes in the pipes can be characterized by the nondimensional Reynolds number (Re), which can be calculated as

$$\mathit{Re}=\frac{\rho QD}{\mu A}$$

(4)

where ρ is the fluid density, Q is the volumetric flow rate, D is the hydraulic diameter (inner diameter of the pipe), A = πD²/4 is the pipe cross-sectional area, and µ is the dynamic viscosity of the fluid. Typical criteria are that the flow in a pipe is laminar when Re < 2300, transitional when 2300 < Re < 4000, and turbulent when Re > 4000 [39].

Table 1. Fluid thermophysical properties and pipe diameter.

Fluid	ρ (kg/m3)	μ (Pa·s)	c (J/kg·K)	k (W/m·K)	D (in)
Water	1000	8.9 × 10−4	4183	0.6	1.61
Galinstan	6440	2.4 × 10−3	296	16.5	1.61

shows the values of ρ and µ, as well as heat capacity c and thermal conductivity k, for water and Galinstan at room temperature and ambient pressure, and the diameter of the pipe containing the TC-3 thermocouple.

Depending on the reactor power state, the coolant flow velocity in coolant channels and, hence, Re numbers take on different values. For pressurized water reactors (PWRs), the range of values is approximately 10³ < Re < 10⁶. For liquid metal fast breeder reactors (LMFBRs), which use liquid sodium as a coolant, the range of values is approximately 10³ < Re < 10⁵ [39].

The graph of time-dependent values of Re for water flow in the pipe with TC-3 is shown in Figure 4. Values of Re for water flow were calculated using Equation (4) and parameters in Table 1, and values for Q were taken from FT-1 time series of water loop measurements. Five time segments with nearly constant flows are labeled 1 through 5, in order of increasing values of Re.

Note that in segment 1, the flow was laminar. For segment 2, the flow was in the transitional regime, while for segments 2 through 5, the flow was turbulent. Five time intervals, each 210 s long or 10% of the total times series of 2100 s, were selected as subsets of constant-flow time intervals. These 210 s long time intervals are used in Section 4 as TC-3 test segments for LSTM development. The numbers of segments, constant-flow time intervals, 210 s long time series test segments, and corresponding values of Re are listed in

Table 2. Time segments and corresponding Re values for water flow near TC-3.

Segment	Constant-Flow Time Interval (s)	Time Series Test Segments (s)	Re
1	1836–2100	1870–2079	1680
2	1444–1661	1444–1653	2600
3	1124–1393	1130–1339	5600
4	811–1027	811–1020	11,900
5	1–739	200–409	18,300

.

The graph of time-dependent values of Re for Galinstan flow in a pipe with TC-3 is shown in Figure 5. Values of Re for Galinstan flow were calculated using Equation (4) and parameters in Table 1. Values of Q were taken from FT-1 time series of measurements in the Galinstan flow loop. Four intervals of nearly constant flow are labeled in Figure 5 as segments 1 through 4, in order of increasing values of Re.

All four constant-flow segments in Figure 5 corresponded to the turbulent flow regime. Four time intervals for the temperature test, each 147 s long or 10% of the total times series of 1470 s, were selected as subsets of the constant-flow segments. These 147 s long time intervals are used in Section 5 as TC-3 test segments for LSTM development. Segment numbers, constant-flow time intervals, time series test segments, and the corresponding values of Re are listed in

Table 3. Time segments and corresponding Re values for Galinstan flow near TC-3.

Segment	Constant-Flow Time Interval (s)	Time Series Test Segments (s)	Re
1	276–791	300–446	5800
2	1170–1471	1200–1346	9100
3	1–216	50–196	14,500
4	918–1100	950–1096	18,200

.

4. Monitoring of Thermocouple Time Series in Water Flow Loop

4.1. LSTM Prediction of Thermocouple Time Series

Time series of TC-3 and TC-4 measurements in the water loop are shown in Figure 6a,b, respectively. Locations of each of the five 210 s long test segments of temperature measurements, numbered 1 through 5, are indicated in red (color online). The numbers of test segments in Figure 6a,b correspond to the numbers of time segments in Table 2.

Results of TC-3 time series prediction for each of the five test segments with LSTM developed using TC-3 data and TL-LSTM developed with transfer learning using TC-4 data are shown in Figure 7. Note that a separate LSTM network was developed for the prediction of each of the five test segments. In each case, the training and validation data sets excluded the corresponding test segment. The performances of LSTM and TL-LSTM predictions are quantified by listing RMSE values in

Table 4. Statistics of thermocouple prediction errors with LSTM and TL-LSTM in water.

Segment	RMSE (10−2) (°C)		μ(·10−2) (°C)		σ(·10−2) (°C)		Measurement Uncertainty (°C)
	LSTM	TL-LSTM	LSTM	TL-LSTM	LSTM	TL-LSTM
1	1.96	2.08	−1.49	0.84	1.28	1.91	±1.1
2	2.01	3.64	0.25	−0.01	2.00	3.65	±1.1
3	4.18	4.60	−1.32	−0.09	3.98	4.61	±1.1
4	6.72	7.85	5.25	2.82	4.20	7.34	±1.1
5	9.07	9.34	3.14	5.17	8.53	7.80	±1.1

. RMSE is a measure of the distance between LSTM-predicted and observed values. In addition, statistics of the LSTM performance are investigated by listing the mean values µ and standard deviations σ for each time segment in Table 4. Error mean value µ indicates if LSTM, on average, under-predicts or over-predicts the observed values. Standard deviation σ is an indicator of the spread of the instantaneous errors around the mean value µ.

The left column of Figure 7 labeled “Test Segment” shows the graphs of the observed (black), predicted-with-LSTM (blue), and predicted-with-TL-LSTM (orange) time series for each of the test segments 1 through 5 of TC-3. The middle column labeled “Error Time Series” shows the graphs of real-time errors for LSTM (blue) and TL-LSTM (orange). The errors at each time step are calculated as error = predicted – observed. The right column labeled “Error Histogram” shows the histogram of prediction errors of LSTM (blue) and TL-LSTM (orange). A Gaussian fit curve is added to each histogram.

The measurement uncertainty of TC-3 for all five test segments was max (±1.1 °C, 0.4%) = ±1.1 °C. For reference, this number is listed in Table 4. The trend for the LSTM prediction errors listed in Table 4 showed that RMSE and σ increased with the increase in Re value (the time segments are numbered in the order of increasing values of Re number). However, the RMSE values as well instantaneous errors (middle column in Figure 7) for each test segment were smaller than the TC-3 measurement uncertainty error by at least an order of magnitude. The RMSE, µ, and σ in predictions with LSTM and TL-LSTM had comparable values for all five time segments. The values of RMSE and σ for LSTM were smaller than the corresponding values for TL-LSTM for all test segments. The exception was segment 5, for which TL-LSTM had a smaller σ value. From the error histogram displayed in the last column in Figure 7, prediction errors for both LSTM and TL-LSTM for time segment 5 had more outliers compared to other time segments. These outliers occurred mostly in the beginning of test segment 5. Error mean values µ indicated that both LSMT and TL-LSTM over-predicted the actual time series values for time segments 4 and 5.

4.2. Relationship between RMSE, σ, and Re

To investigate the performance of the LSTM for different flow regimes in the water loop, we plot Table 4 values of RMSE as a function of Re in Figure 8. We obtained linear fits for the LSTM (dashed blue line) and the TL-LSTM (dashed red line), with R² = 0.99 and R² = 0.96, respectively. High values of R² indicate strong linearity in the data.

Linear correlations for RMSE as a function of Re are listed in

Table 5. Correlation between RMSE and Re in water.

Model	RMSE (°C)	RMSE < 1.1 °C
LSTM	4.38 × 10−6Re + 0.013	Re < 2.48 × 105
TL-LSTM	4.21 × 10−6Re + 0.022	Re < 2.56 × 105

. Note that the slopes of the two linear correlations were almost the same, with the difference appearing in the second significant digit. Using linear correlations, we estimated the range of applicability of LSTM-based monitoring by setting the condition that RMSE < 1.1 °C (thermocouple measurement uncertainty). Solving for Re numbers using linear correlations, we obtained the upper bound values, which are listed in the last column of Table 5. The upper bounds for Re numbers were almost the same for both LSTMs, with a difference appearing in the second significant digit.

As an additional consideration, for the data in Table 4, we investigated the correlation between σ and Re. The graphs of σ vs. Re for LSTM (blue color) and TL-LSTM (red color) LSTMs are shown in Figure 9. Linear fits were obtained with R² = 0.91 for LSTM and R² = 0.89 for TL-LSTM. This indicated a relatively weak linearity of the data, compared to the data in Figure 8.

4.3. Benchmarking LSTM Runtime

The runtime of the LSTM network was benchmarked on an Intel© Xeon© CPU E5-2687W v3 @ 3.10 GHz with 64 GB RAM. A representative benchmark LSTM runtime in the water loop is shown in Figure 10. The graph displays the runtime of LSTM prediction of each sample in the 210 s long test segment 3 of the TC-3 time series. For the data in Figure 10, we obtained the LSTM per sample runtime mean value µ = 3.4 ms with standard deviation σ = 0.85 ms. Note that the LSTM runtime for the prediction of each sample was at least an order of magnitude smaller than the acquisition time of 1 s. Similar runtime characteristics were observed in benchmarking of the LSTM prediction of other TC-3 test segments. Therefore, in the online monitoring scenario, validation of the measurements with LSTM can be, in principle, performed in real time.

5. Monitoring of Thermocouple Time Series in Galinstan Flow Loop

5.1. LSTM Prediction of Thermocouple Time Series

Time series of TC-3 and TC-4 in Galinstan loop are shown in Figure 11a,b, respectively. The location of each of the five 147 s long test segments, numbered 1 through 4, is indicated in red (color online). The numbers of test segments in Figure 11a,b correspond to the numbers of time segments in Table 3.

The results of TC-3 time series prediction for each of the four test segments with the LSTM developed using TC-3 data and TL-LSTM developed using TC-4 data are shown in Figure 12. Note that a separate LSTM network was developed for the prediction of each of the four test segments. In each case, the training and validation datasets excluded the corresponding test segment. Similar to the case of the water loop considered in Section 4, we considered the RMSE over each test segment, as well as mean value of the error µ and standard deviation σ of errors from the mean. Statistics of LSTM and TL-LSTM predictions were quantified by listing RMSE, µ, and σ values for each time segment in

Table 6. Statistics of thermocouple prediction errors with LSTM and TL-LSTM in Galinstan.

Segment	RMSE (10−2) (°C)		μ(·10−2) (°C)		σ(·10−2) (°C)		Measurement Uncertainty (°C)
	LSTM	TL-LSTM	LSTM	TL-LSTM	LSTM	TL-LSTM
1	3.47	4.51	1.64	−2.93	3.07	3.44	±1.1
2	6.01	8.44	3.81	−0.89	4.66	8.42	±1.1
3	10.55	11.97	3.58	−1.03	9.95	11.97	±1.1
4	15.39	16.18	0.96	−5.51	15.42	15.27	±1.1

.

The left column of Figure 12 labeled “Test Segment” shows the graphs of the observed (black), predicted-with-LSTM (blue), and predicted-with-TL-LSTM (orange) time series for each of the test segments 1 through 4 of TC-3. The middle column labeled “Error Time Series” shows the graphs of real-time errors for the LSTM (blue) and TL-LSTM (orange). The errors at each time step were calculated as error = predicted − observed. The right column labeled “Error Histogram” shows the histogram of prediction errors of the LSTM (blue) and TL-LSTM (orange). A Gaussian fit curve is added to each histogram.

The measurement uncertainty of TC-3 for all four test segments was max (±1.1 °C, 0.4%) = ±1.1 °C. For reference, this number is listed in Table 6. The trend for the LSTM prediction errors listed in Table 6 showed that RMSE and σ increased with the increase in Re value (the time segments are numbered in the order of increasing values of Re number). However, the RMSE values as well instantaneous errors (middle column in Figure 12) for each test segment were smaller than the TC-3 measurement uncertainty error by at least an order of magnitude. The RMSE, µ, and σ in predictions with the LSTM and TL-LSTM had comparable values for all four time segments. The values of RMSE and σ for LSTM were smaller than the corresponding values for TL-LSTM for all test segments. From the error histogram displayed in the last column in Figure 12, prediction errors for both LSTM and TL-LSTM for all time segments had a Gaussian-like shape for all time segments. Error mean values µ indicated that the LSTM over-predicted while the TL-LSTM under-predicted the actual time series values for all time segments.

5.2. Relationship between RMSE, σ, and Re

To investigate the performance of the LSTM for different flow regimes in the water loop, we plot Table 6 values of RMSE as a function of Re in Figure 13. We obtained linear fits for the LSTM (dashed blue line) and TL-LSTM (dashed red line), for both of which R² = 0.99. High values of R² indicate strong linearity in the data. Linear correlations for RMSE as a function of Re are listed in

Table 7. Correlation between RMSE and Re in Galinstan.

Model	RMSE (°C)	RMSE < 1.1 °C
LSTM	9.44 × 10−6 Re − 0.024	Re < 1.19 × 105
TL-LSTM	8.95 × 10−6 Re − 0.004	Re < 1.23 × 105

. Note that the slopes of the two linear correlations were close in value.

Solving for Re numbers using linear correlations, we obtained the upper bound values, which are listed in the last column of Table 7. The upper bounds for Re numbers were almost the same for both LSTMs, with the difference appearing in the second significant digit.

As an additional consideration, for the data in Table 6, we investigated the correlation between σ and Re. The graphs of σ vs. Re for LSTM (blue color) and TL-LSTM (red color) are shown in Figure 14. Linear fits were obtained with R² = 0.97 for both the LSTM and TL-LSTM. Unlike the case of the water loop, we observed a strong linearity in the dependence between σ and Re in the Galinstan loop.

5.3. Benchmarking LSTM Runtime

A representative benchmark LSTM runtime in the Galinstan loop is shown in Figure 15. The graph displays the runtime of the TC-3/TC-3 LSTM prediction of each sample in the 147 s long test segment 1 of the TC-3 time series. For the data in Figure 15, we obtained the LSTM per sample runtime mean value µ = 3.5 ms with standard deviation σ = 1.1 ms. These values were comparable to those of the water loop benchmark in Figure 10. Note that the LSTM runtime for prediction of each sample was at least an order of magnitude smaller than the acquisition time of one second. Similar runtime characteristics were observed in the benchmarking of LSTM prediction of other TC-3 test segments.

6. Discussion

In Section 4 and Section 5, we investigated monitoring of the thermocouple times series with the LSTM in water and Galinstan fluids. In both fluids, RMSE increased linearly with Re. Comparing the LSTM performance in different fluids (Figure 7 and Figure 12) over the common range of Re values (approximately 6000 < Re< 18,000), RMSE of the same sensor’s prediction in the Galinstan was consistently larger than RMSE in the water loop. This can be potentially attributed to higher noise in the temperature measurements in Galinstan. Using the MATLAB function, we calculated the 99% occupancy bandwidth (BW) for each test segment of TC-3 in water to be BW = 2.4 mHz. On the other hand, for each test segment of TC-3 in Galinstan, we obtained BW = 3.4 mHz.

For developing a qualitative explanation for the observed increase in RMSE with increasing Re, we considered the dependence of the thermocouple time constant τ on Re. We showed that τ decreases with increasing Re (sensor responds faster). At the same time, as Re increases, the fluid flow becomes more turbulent [39]. Thus, our hypothesis is that the LSTM prediction accuracy decreases because the thermocouple becomes more sensitive to temperature fluctuations in an increasingly turbulent flow. In prior work, we showed that τ of the cylindrical resistance temperature detector (RTD) decreases with the increase in the fluid-to-RTD heat transfer coefficient [4]. For the cylindrical-shaped type K thermocouple probe, the heat transfer coefficient h in forced convective heat transfer for a fluid flow normal to a circular cylinder is [4]

$$h=\frac{k·\mathit{Nu}}{d}$$

(5a)

where k is the fluid thermal conductivity, d is the cylinder diameter, and Nu is the Nusselt number:

$$Nu=0.43+C·{Re}^m·{Pr}^{0.31}$$

(5b)

Here, Pr is the Prandtl number given as

$$Pr=\frac{c·\mu }{k}$$

(5c)

where c is the specific heat capacity, µ is the dynamic viscosity, and C and m are fitting constants. Values of C and m for different ranges of Re number are given in

Table 8. Values of constants for heat transfer coefficient correlation.

Re	C	m
35–5 × 103	0.583	0.471
5 × 103–5 × 104	0.148	0.633
5 ×104–5 × 105	0.0208	0.814

.

Using thermophysical property values for water and Galinstan fluids listed in Table 1, and d = 5 mm as the type K thermocouple diameter, we calculated h as a function of Re. Results are plotted in Figure 16 for the range of 10³ < Re < 2 × 10⁴, which spans the range of Re numbers in the experiment. Note that h increased monotonically with Re for both fluids, with a steeper slope for the case of Galinstan. Therefore, thermocouple time constant τ decreased with increasing Re.

Note that for the same value of Re, the value of h was higher for Galinstan. This indicated that thermocouple τ in Galinstan was smaller than that in water. From the discussion above, RMSE for Galinstan was higher than that for water. This is consistent with the hypothesis that a larger value of RMSE is correlated with a smaller value of thermocouple τ.

7. Conclusions

We investigated the performance of LSTM and TL-LSTM in monitoring and estimating the thermocouple time series in water and Galinstan fluids at room temperature and ambient pressure. LSTMs for predicting the time series of a thermocouple in each loop were constructed by training on prior history of the same sensor, while TL-LSTMs were constructed by training on the history of a correlated sensor. For each fluid, a similar performance was observed for both LSTM and TL-LSTM models. The benchmarking LSTM performance indicated that monitoring can be performed in real time.

Using linear correlations, we estimated that RMSE in LSTM prediction is smaller than the uncertainty in temperature measurement within the range, with the upper bound of Re~10⁵. During startup and shutdown (low power operation), typical values for fluid flow in core coolant channels PWR and LMFBR are Re~10³ and Re~10², respectively [39]. During full-power operation, the values for fluid flow in the same core coolant channels in PWR and LMFBR are Re~10⁵ and Re~10⁴, respectively [39].

Based on the findings of this paper, LSTM-based monitoring can provide reliable real-time estimates of a thermocouple for typical operational ranges of a nuclear reactor. However, further studies are needed to validate these predictions. In particular, LSTM analysis should be performed on data obtained from measurements in water and liquid sodium at temperatures, pressures, and flow rates similar to those in nuclear reactors. Future work will also benchmark the performance of LSTMs (prediction accuracy and runtime) against other ML algorithms, such as Auto Regressive Integrated Moving Average (ARIMA), Generative Adversarial Networks (GAN), and Bayesian Networks. We will also investigate transfer learning (TL) for a high-temperature thermal hydraulic system by pre-training the model on a room-temperature fluid flow. As the operation of a room-temperature thermal hydraulic facility is considerably less expensive than the operation of a high-temperature facility, TL offers the possibility of cost saving for the development of models for monitoring nuclear systems.