Plastic Constitutive Training Method for Steel Based on a Recurrent Neural Network

Abstract

The deep learning steel plastic constitutive model training method was studied based on the recurrent neural network (RNN) model to improve the allocative efficiency of the deep learning steel plastic constitutive model and promote its application in practical engineering. Two linear hardening constitutive datasets of steel were constructed using the Gaussian stochastic process. The RNN, long short-term memory (LSTM), and gated recurrent unit (GRU) were used as models for training. The effects of the data pre-processing method, neural network structure, and training method on the model training were analyzed. The prediction ability of the model for different scale series and the corresponding data demand were evaluated. The results show that LSTM and the GRU are more suitable for stress–strain prediction. The marginal effect of the stacked neural network depth and number gradually decreases, and the hysteresis curve can be accurately predicted by a two-layer RNN. The optimal structure of the two models is A50-100 and B150-150. The prediction accuracy of the models increased with the decrease in batch size and the increase in training batch, and the training time also increased significantly. The decay learning rate method could balance the prediction accuracy and training time, and the optimal initial learning rate, batch size, and training batch were 0.001, 60, and 100, respectively. The deep learning plastic constitutive model based on the optimal parameters can accurately predict the hysteresis curve of steel, and the prediction abilities of the GRU are 6.13, 6.7, and 3.3 times those of LSTM in short, medium, and long sequences, respectively.

1. Introduction

In the process of engineering structural design, a high-precision finite element method (FEM) is needed to ensure the reliability of steel structures [1,2,3]. An accurate material constitutive model forms the basis of ensuring the reliability of the simulation results. At present, elastic constitutive model calculation is often used in the design of steel structures to achieve a fast design and save on computing resources, without considering the plastic stage. Elastic constitutive properties are different from the real mechanical properties of steel, and so a design based on this model calculation is inaccurate and too conservative [4,5]. The use of a plasticity constitutive model in the FEM calculation process of existing steel structures inevitably uses complex integration algorithms, and the process will go through complex iterations, which are time-consuming and have uncontrollable convergence [6,7]. Therefore, determining how to improve the efficiency of structural calculation while using the plastic constitutive model is a key objective in the industry [8,9].

With the rapid development of artificial intelligence (AI) technology, this problem can be solved. AI is essentially a neural network structure formed by using deep learning algorithms to fit and summarize a large number of existing data [10]. The structure of these neural networks is highly complex, allowing for rapid prediction and the generation of results as new data are input [11]. Accordingly, if the deep learning algorithm is used to learn the stress–strain variation law of steel in the plastic stage, then a neural network can be trained to replace the existing plastic constitutive model. The mapping method of inputting a batch of strain end to end to obtain a batch of stress can be realized. This can solve the existing plastic constitutive model’s problems of repeated iterations, a slow calculation speed, and poor convergence [12,13]. Similar work began with the use of artificial neural networks instead of material plasticity. This approach takes the strain increment and the stress and strain of the previous or first n incremental steps as inputs and the stress increment as outputs [14,15]. More recently, researchers proposed the application of an RNN to predict the hysteresis curve of metal materials and were able to predict the hysteresis curve under a cyclic load [16,17,18].

While the use of deep learning models to replace the plasticity principal structure of metallic materials is certainly efficient, its application and diffusion have been slow to advance. This is largely because the established research is mainly devoted to the innovation of deep learning algorithms, proposing models with a higher training efficiency and more accurate prediction. Issues have been detailed such as the form of the neural network structure, the mode of raw data processing, and the means of data training, but these have not been studied in depth. This makes it difficult for deep learning frameworks to be applied to FEM computational problems. Detailed research on the parameter setting and training methods required to apply the model is needed to ensure the reliability of the model. Engineers must be given parameter setting suggestions to avoid a trial-and-error cost of multiple engineers repeatedly adjusting the parameters in order to solve the same problem. The corresponding prediction ability and prediction effect of different models need to be understood, to clarify their prediction capacity for different lengths of hysteresis curves and whether they meet the demands for specific data quantities. These topics have been ignored by researchers in the field for a long time, hindering the development and application of a deep learning plastic constitutive model.

Focusing on the above problems, this paper studies the plastic constitutive model training method for steel based on an RNN. Linear isotropic hardening and linear kinematic hardening models were used for numerical experiments. A method of randomly generating stress–strain sequence datasets is proposed, which strictly follows the principles of plasticity mechanics and can create larger datasets for neural network training than traditional mechanical tests. Three recurrent neural networks suitable for time-series prediction were used to train the steel plasticity constitutive model. The effects of the dataset and neural network model parameters on model training are discussed, and an optimal training method for neural networks is given for the hysteresis curve prediction problem. This study provides a more refined method of parameterization and training of RNNs for stress–strain sequence prediction, which avoids the trial-and-error cost of repeated parameterization for this type of problem.

2. Basic Principles

2.1. The Computational Procedure of the Constitutive Model in the FEM

The traditional plastic constitutive model of materials is based on phenomenological theory. The mechanical properties of materials are identified through material tests and described by a set of mathematical expressions that can reasonably express these characteristics. With the most basic metal plastic constitutive model taken as an example, the stress–strain curve (i.e., constitutive relation) is obtained by a uniaxial tensile test and low-cycle tensile and compressive test [19,20,21].

The total strain $\epsilon$ of the material can be divided into elastic strain ${\epsilon }^e$ and plastic strain ${\epsilon }^p$, as shown in Equations (1) and (2). The yield criterion can be used to determine whether the material is yielding, as shown in Equation (3). The flow criterion is used to determine the flow direction of the yield surface, as shown in Equation (4). The yield stress is adjusted by the hardening criterion, and the internal variable is updated, as shown in Equations (5) and (6). The elastic–plastic model should satisfy the discrete and complementary conditions required [22], as shown in Equation (7).

$$\epsilon ={\epsilon }^e+{\epsilon }^p$$

(1)

$$\sigma =E{\epsilon }^e$$

(2)

$$\mathsf{\Phi }\left(\sigma ,{\sigma }_y\right)=\left|\sigma \right|-{\sigma }_y$$

(3)

$${\dot{\epsilon }}^p=\dot{\gamma }sign\left(\sigma \right)$$

(4)

$${\sigma }_y={\sigma }_y\left({\overline{\epsilon }}^p\right)$$

(5)

$${\dot{\epsilon }}^p=\dot{\gamma }$$

(6)

$$\mathsf{\Phi }\le 0,\hspace{1em}\hspace{1em}\dot{\gamma }\ge 0, \dot{\gamma }\mathsf{\Phi }=0$$

(7)

where $E$ is Young’s modulus; $\sigma$ is the total stress; $\epsilon$ is the total strain; $\mathsf{\Phi }$ is the yield function; ${\sigma }_y$ is the yield stress; ${\dot{\epsilon }}^p$ is the plastic strain rate; $\dot{\gamma }$ is the plastic multiplier; ${\overline{\epsilon }}^p$ is the equivalent plastic strain; and $sign\left(\right)$ is the sign function, whose specific form is $sign\left(a\right)=\left\{\begin{array}{c}+1 if a\ge 0\\ -1 if a<0\end{array}\right.$.

In FEM calculation, the material constitutive model needs to be used when updating the stress and internal variable state and calculating the tangential stiffness matrix. When the state of stress and the internal variable is updated, the stress ${\sigma }_{n+1}$ and internal variable ${\alpha }_{n+1}$ of $t_{n+1}$ are updated by Equations (8) and (9) in an incremental time step [$t_n,t_{n+1}$] and then applied to the internal force assembly of the element, as shown in Equation (10). When the tangential stiffness matrix is calculated, the correlated uniform tangential modulus is updated by Equation (11) and used in the next iteration of the FEM balance equation, as shown in Equation (12).

$${\sigma }_{n+1}=\widehat{\sigma }({\alpha }_n,{\epsilon }_{n+1})$$

(8)

$${\alpha }_{n+1}=\widehat{\alpha }({\alpha }_n,{\epsilon }_{n+1})$$

(9)

$${\mathbf{f}}_e^{int}=\sum _{i=1}^{n_{gausp}}{j}_i{w}_i{\mathbf{B}}_i^T{\left.{\sigma }_{n+1}\right|}_i$$

(10)

$$\mathbf{D}\equiv {\displaystyle \frac{\partial \widehat{\sigma }}{\partial {\epsilon }_{n+1}}}$$

(11)

$${{\mathbf{K}}_T}^{\left(e\right)}=\sum _{i=1}^{n_{gausp}}{w}_i{j}_i{\mathbf{B}}_i^T{\mathbf{D}}_i{\mathbf{B}}_i$$

(12)

where $j_i$ is the Jacobian matrix; $w_i$ is the Gaussian orthogonal weight; ${\mathbf{f}}_e^{int}$ is the internal force of the element; ${\mathbf{D}}_i$ is the correlated uniform tangent modulus; ${\mathbf{B}}_i$ is the strain displacement matrix; and ${\mathbf{K}}_T$ is the tangent stiffness matrix of the element.

The Euler discrete scheme is needed to solve the above problems. In an incremental time step [$t_n,t_{n+1}$], given the elastic strain ${\epsilon }_n^e$ and internal variable ${\alpha }_n$ at time $t_n$, and with the strain increment set as $∆\epsilon$, the elastic strain ${\epsilon }_{n+1}^e$ and internal variable ${\alpha }_{n+1}$ at time $t_{n+1}$ are solved, as shown in Equations (13) and (14). Then, the stress ${\sigma }_{n+1}$, internal variable $A_{n+1},$ and plastic strain ${\epsilon }_{n+1}^p$ at time $t_{n+1}$ are obtained, as shown in Equations (15) and (16).

$${\epsilon }_{n+1}^e={\epsilon }_n^e+∆\epsilon -∆\gamma N({\sigma }_{n+1},A_{n+1})$$

(13)

$${\alpha }_{n+1}={\alpha }_n+∆\gamma N({\sigma }_{n+1},A_{n+1})$$

(14)

$${\sigma }_{n+1}={\left.\overline{\rho }{\displaystyle \frac{\partial \psi }{\partial {\epsilon }^e}}\right|}_{n+1}, A_{n+1}={\left.\overline{\rho }{\displaystyle \frac{\partial \psi }{\partial \alpha }}\right|}_{n+1}$$

(15)

$${\epsilon }_{n+1}^p={\epsilon }_n^p+∆\epsilon -∆{\epsilon }^e$$

(16)

Generally, the solution algorithm is the fully implicit elastic predictor/return-mapping algorithm. After the material enters the plastic stage, the Newton iteration method needs to be used for several iterations, and the calculation time is long. The plastic constitutive model of steel based on a neural network can bypass the complex iterative steps and directly map the stress sequence from the input strain sequence, thus greatly reducing the calculation time of numerical simulation. The basic process is divided into three steps: dataset production, data pre-processing, and model training.

The specific process is shown in Figure 1. The establishment method does not require users to master more prior knowledge and only needs to place the stress and strain data obtained by experiments or numerical simulation into the RNN model. The neural network that can master the constitutive relationship of steel can be used in structural calculation after training.

2.2. Linear Hardening Constitutive Model for Metals

In order for the trained RNN model to predict the complex mechanical properties of steel under cyclic loading, two classical linear hardening constitutive models are selected, namely, the linear isotropic hardening model and linear kinematic hardening model [22]. The elastic parts of the two models are shown in Equations (1) and (2), and the plastic part adopts the von Mises yield criterion, shown in Equation (17), and the associated flow rule, shown in Equation (18). The difference between the two models lies in the difference in hardening criteria. The isotropic hardening criterion defines that the yield surface center remains unchanged, while the yield surface radius expands with the accumulation of equivalent plastic strain. The cyclic hardening behavior of steel can be adequately reflected. Its formula when the hardening function is linear is shown in Equation (19). The kinematic hardening criterion defines that the radius of the yield surface does not change, but the center of the yield surface moves with the accumulation of internal variables. Then, Equation (17) becomes Equation (20), and the flow rule is also transformed into Equation (21). This can reflect both the cyclic hardening behavior and the Bauschinger effect. The two-dimensional state of the two models is shown in Figure 2.

$$\mathsf{\Phi }\left(\sigma ,{\sigma }_y\right)=\left|\sigma \right|-{\sigma }_y={\left({\displaystyle \frac{2}{3}}{\sigma }^{\prime }:{\sigma }^{\prime }\right)}^{{\displaystyle \frac{1}{2}}}-{\sigma }_y$$

(17)

$$d{\epsilon }_p=d\lambda {\displaystyle \frac{\partial \mathsf{\Phi }}{\partial \sigma }}={\displaystyle \frac{3}{2}}d\lambda {\displaystyle \frac{{\sigma }^{\prime }}{\left|\sigma \right|}}$$

(18)

$${\sigma }_y\left({\overline{\epsilon }}^p\right)={\sigma }_{y0}+H{\overline{\epsilon }}^p$$

(19)

$$\mathsf{\Phi }\left(\sigma ,{\sigma }_y\right)={\left({\displaystyle \frac{2}{3}}({\sigma }^{\prime }-x^{\prime }):({\sigma }^{\prime }-x^{\prime })\right)}^{{\displaystyle \frac{1}{2}}}-{\sigma }_y$$

(20)

$$d{\epsilon }_p={\displaystyle \frac{3}{2}}d\lambda {\displaystyle \frac{{\sigma }^{\prime }-x^{\prime }}{J\left|\sigma -x^{\prime }\right|}}$$

(21)

where $\sigma$ is the von Mises equivalent stress; ${\sigma }^{\prime }$ is the deviator stress tensor; ${\sigma }_y$ is the yield stress; $H$ is the slope of the linear hardening function; $d\lambda$ is the plastic multiplier; and $x^{\prime }$ is the tensor of the kinematic hardening back stress.

The metal material used in this paper is Q235 steel, which has an elastic modulus $E$ of 200 GPa, a Poisson’s ratio $v$ of 0.3, an initial yield stress ${\sigma }_{y0}$ of 250 MPa, and a hardening parameter $H$ of 50,000.

3. Basic Structure and Characteristics of the Recurrent Neural Network

3.1. Original Recurrent Neural Network

RNNs are applied to the establishment of metal plastic constitutive models to consider the influence of historical time data [23]. By introducing memory $h$, RNNs can associate the output $y_t$ of the later time with the input information $h_{t-1}$ of the previous time so that the network has a memory effect, as shown in Equations (22) and (23). In the forward propagation of RNNs, the information $x_t$ of each moment is input into the network, the memory $h_t$ of the moment is updated by Equation (22) and passed to the next moment, and the output $y_t$ of the moment $t$ is calculated at the same time, as shown in Figure 3.

$$y_t=g\left(V{h}_t\right)$$

(22)

$$h_t=f(U{x}_t+W{h}_{t-1})$$

(23)

where $y_t$ is the output value at time $t$; $g$ is the activation function from the hidden layer to the output layer; $f$ is the activation function from the input layer to the hidden layer; $V$ is the weight matrix from the hidden layer to the output layer; $U$ is the weight matrix from the input layer to the hidden layer; and $h_t$ is the memory.

In the above approach, the RNN can build a certain short-term memory effect, and the output $y_t$ at time $t$ is related to the memory of the previous time $t-1$ so that Equation (22) can be expanded into the form of Equation (24). As can be seen from Equation (24), when the input sequence is long, the data far away from time $t$ in the process of backpropagation will hardly affect the update of weight matrix $W$, resulting in gradient disappearance and gradient explosion and forgetting the distant historical inputs. As a result, the RNN has a poor long-term memory effect.

$$y_t=g\left(V{h}_t\right)=g\left(Vf(U{x}_t+Wf(U{x}_{t-1}+Wf(U{x}_{t-2}+Wf(U{x}_{t-3}+\dots \left)\right)\left)\right)\right)$$

(24)

3.2. Long Short-Term Memory

To compensate for the shortcomings of the RNN, researchers have proposed an RNN with long-term memory effect properties, known as LSTM [24]. A dual-line parallel memory structure is formed by adding a state quantity $c$ to save the part of the historical input that is needed for long-term memory, as shown in Figure 4. LSTM controls long-term memory state $c$ through three gate structures: the forgetting gate, input gate, and output gate. The forgetting gate determines the proportion of the long-term memory state $c_{t-1}$ retained in the current long-term memory state $c_t$, as shown in Equation (25).

$$f_t=\mathsf{\sigma }\left(W_f·\left[h_{t-1},x_t\right]+b_f\right)$$

(25)

$$i_t=\mathsf{\sigma }(W_i·\left[h_{t-1},x_t\right]+b_i)$$

(26)

$$\widetilde{c_t}=\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}(W_c·\left[h_{t-1},x_t\right]+b_c)$$

(27)

$$c_t=f_t·c_{t-1}+i_t·\widetilde{c_t}$$

(28)

$$o_t=\mathsf{\sigma }(W_o·\left[h_{t-1},x_t\right]+b_o)$$

(29)

$$h_t=o_t·\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}{c}_t$$

(30)

where $f_t$ is the output value of the forgetting gate; $\widetilde{c_t}$ is the information formed after the induction of current information and historical information, which is generally called the candidate state; $i_t$ represents the proportion of the candidate state in the quantity of long-term memory states; $o_t$ represents the output value of the output gate; and so on. $W_f$, $W_i$, $W_c$, and $W_o$ are the weight matrices of the forgetting gate, input gate, memory state, and output gate, respectively. $b_f$, $b_i$, $b_c$, and $b_o$ are the bias terms of the forgetting gate, input gate, memory state, and output gate, respectively. $\mathsf{\sigma }$ and $\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}$ are activation functions, whose specific forms are $\mathsf{\sigma }\left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$ and $\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}(x)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$, respectively.

3.3. Gated Recurrent Unit

The structure of LSTM is complex, the number of parameters in training is large, and the internal structure of some gate elements is too weak to improve the training ability of the model. Therefore, some researchers simplified it and proposed a GRU, which includes only two gate structures, namely, the update gate and reset gate [25], as shown in Figure 5. The GRU has one less state output than LSTM, so the number of training parameters is greatly reduced, and it is a lightweight time-series neural network model. The memory update to the GRU is determined jointly by the update gate and the reset gate, as shown in Equations (31) and (32), and the update formula of the update gate and the reset gate are shown in Equations (33) and (34).

$$h_t=\left(1-z_t\right)\ast h_{t-1}+z_t\ast \widetilde{h_t}$$

(31)

$$\widetilde{h_t}=\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}(W·\left[r_t\ast h_{t-1},x_t\right]+b_h)$$

(32)

$$z_t=\mathsf{\sigma }(W_z·\left[h_{t-1},x_t\right]+b_z)$$

(33)

$$r_t=\mathsf{\sigma }(W_r·\left[h_{t-1},x_t\right]+b_r)$$

(34)

where $z_t$ and $r_t$ are the output of the update gate and reset gate, respectively; $\widetilde{h_t}$ is the candidate state, which functions to summarize the current input information and historical information; $W$, $W_z$, and $W_r$ are the weight matrices of the candidate state, update gate, and reset gate, respectively; and $b_h$, $b_z$, and $b_r$ are the bias terms of the candidate state, update gate, and reset gate, respectively.

The three kinds of RNNs have their own characteristics and different applicability to different data. Therefore, this paper will comprehensively discuss the performance of the three kinds of RNNs with different data, give their respective application ranges, and study their training methods.

4. Data Generation and Model Training

4.1. Data Preparation

4.1.1. Data Generation

On the basis of the two linear hardening constitutive models described in Section 2.2, a script was written in Python to randomly generate stress–strain sequences of the two constitutive models. Its main tasks were divided into two parts: one was to generate a batch of random strain loading paths through a random process; and the other was to input the random strain loading paths into the theoretical model in Section 2.2 to calculate the stress paths. The process was divided into four steps: (1) n data points with a Gaussian random distribution are generated randomly; (2) 50 loading steps are inserted into these points; (3) the generated data are multiplied by 0.01 to limit the strain data to a reasonable range [−0.03,0.03]; and (4) the stress path is calculated using theoretical formulas. The calculation method uses Equations (17)–(21). Using this method, 283,500 random curves were generated with curve lengths ranging from 200 to 2000, with the specific parameters shown in

Table 1. Deep learning model parameters.

Parameter Class	Parameter Setting Situation
Type of model	RNN/LSTM/GRU
Sequence length	200/400/600/1000/1200/1400/1600/1800/2000
Training data size	500/1000/2000/4000/6000/8000/10,000
Network layer number	1/2/3/4/5
Number of neurons	50/100/150/200/250/300
Learning rate	Constant learning rate/Decay learning rate
Training batch	0–100
Batch size	10/20/40/60/80/100/120/140/160/180/200

. Of these, 141,750 were used as the training set, 70,875 as the validation set, and 70,875 as the test set. The code for generating the database is shown in Appendix A.

4.1.2. Data Pre-Processing

Generally, the data used for training are difficult to distribute evenly. If the data are not pre-processed before training, then the gradient gap between the data will be large, which will lead to a poor training efficiency, gradient explosion, and gradient disappearance. The traditional deep learning standardization methods are mainly min–max standardization and mean variance standardization, and the calculation methods are shown in Equations (35) and (36), respectively. However, the sequence data processed in this paper are strain and stress, which have a large difference in order of magnitude, and the effect of the above method is poor, as shown in Figure 6. The stress–strain data obtained by mean variance standardization differ greatly in order of magnitude. As a result, the deep learning model is unable to extract features, thus making subsequent training impossible. Although min–max standardization can reduce the stress and strain data to an order of magnitude, it deletes all the negative parts of the data, and metal materials cannot be free from pressure. Thus, this method has serious defects. Therefore, this paper adopts the nonlinear reduction method as the data pre-processing method [26], and the calculation method is shown in Equation (37). After the same data are processed, the stress and strain data can be reduced to an order of magnitude and the negative part of the data can be considered at the same time.

$$\widehat{x_i}={\displaystyle \frac{x_i-\mathrm{m}\mathrm{i}\mathrm{n}\hspace{0.17em}\left(X\right)}{\max \hspace{0.17em}\left(X\right)-\mathrm{m}\mathrm{i}\mathrm{n}\hspace{0.17em}\left(X\right)}}$$

(35)

$$\widehat{x_i}={\displaystyle \frac{x_i-\mathsf{\mu }}{\mathsf{\sigma }}}$$

(36)

$$\widehat{x_i}=\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}({\displaystyle \frac{x_i}{X_{ref}}})$$

(37)

where $x_i$ is the input feature, $X$ = ($x_1,x_2,x_3,\cdots {,x}_n$); $\mathrm{m}\mathrm{i}\mathrm{n}\hspace{0.17em}\left(\right)$ and $\mathrm{m}\mathrm{a}\mathrm{x}\hspace{0.17em}\left(\right)$ are the minimum and maximum values; $\mathsf{\mu }$ and $\mathsf{\sigma }$ are the mean and variance of $X$, respectively; $\widehat{x_i}$ is the normalized feature; $\mathbf{t}\mathbf{a}\mathbf{n}\mathbf{h}(x)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$; and $X_{ref}$ is the nonlinear shrinkage coefficient, which takes a value that can shrink the data to an order of magnitude.

4.2. Model Training Method

The equipment used to train the deep learning model in this study is a laboratory workstation equipped with an Intel(R) Core(TM) i9-10980XE processor (Produced by Intel Corporation, Santa Clara, CA, USA) with 16 cores and a GTX2070 SUPER graphics card (Produced by NVIDIA Corporation, Santa Clara, CA, USA) with 8 GB video memory. The deep learning framework used is TensorFlow2.0, which has all the basic methods needed for training deep learning, is optimized in the calling method, and has the characteristics of convenient deployment and a fast training speed. Three kinds of RNNs (RNN, LSTM, and GRU) were trained to predict the plastic cyclic mechanical response for steel. Stress–strain sequences of different lengths were generated as training data to study the predictive ability of the three models. Different network structure forms, data entry forms, and learning rate setting methods were set up to study the influence of model parameters on the training effect, and the specific parameters are shown in Table 1. The neural network optimizer selected the Adam optimizer, and the model evaluation method selected the mean square error (MSE), whose calculation method is shown in Equation (38). A low MSE corresponds to a small gap between the predicted value and the true value, and thus a better prediction accuracy of the model.

$$MSE={\displaystyle \frac{1}{N}}\sum _{n=1}^N(y-\widehat{y})$$

(38)

where $N$ is the total number of samples, $y$ is the real value of the data, and $\widehat{y}$ is the predicted value.

5. Influence Effect and Prediction Results of Model Parameters

5.1. Model Parameter Influence Effect

5.1.1. Influence of Recurrent Neural Network Structure

Stress–strain datasets generated randomly by a linear hardening constitutive model were used in the study of the effects of model parameters. The performance of the neural network was evaluated in terms of the number of neurons and the number of layers. In order to discuss the effect of the network structure, the sequence length and the size of the dataset were controlled to be medium, with the sequence length selected as 1000 and the training data size as 2000 groups. The data were trained by maintaining a fixed learning rate of 0.001, training 50 epochs, and feeding 50 sets of data each time. The effects of the number of neurons on the model performance and training time in the single-layer network are shown in Figure 7. The influence of the number of hidden layers on the model performance and training time is shown in Figure 8.

As can be seen from Figure 7, when only a single-layer neural network is used, the type of neural network essentially determines the model performance. Two kinds of neural networks that introduce the forgetting mechanism—LSTM and the GRU—improve the model accuracy more than traditional RNNs, and their training speed is more than 10 times that of traditional RNNs. However, increasing the number of neurons cannot improve the performance of the model and will greatly increase the training time of the model. When the number of neurons exceeds 100, the accuracy of the model will increase almost to zero, and the training time will almost double.

As can be seen from Figure 8, when multi-layer neural networks are introduced, the model accuracy gap of the three kinds of neural networks is further widened, with GRU being the best, followed by LSTM and the RNN. Moreover, a number of deep layers corresponds to a great improvement in model accuracy. The three models all perform at their best with a two-layer neural network. However, stacking the number of layers of the neural network will not improve the accuracy of the model and will lead to a significant increase in training time. Figure 8b shows that the training time of traditional RNNs is about 18 to 300 times that of the other two neural network structures. The findings indicate that the RNN is not suitable as a neural network model for medium-length sequences (sequence length is about 1000).

After studying the single-layer neural network model and the multi-layer neural network model, we analyzed the influence of the two-layer neural network model with a non-equal number of neurons to further analyze the influence of neural network topology on model training. The cyclic neural network model was combined in sequential and reverse orders to explore whether the quality of the model could be further improved. The topology structure is represented by Ai-j and Bi-j, where A and B represent LSTM and the GRU, respectively; i represents the number of neurons corresponding to the first layer; and j represents the number of neurons corresponding to the second layer, as shown in Figure 9.

As can be seen from Figure 9, no matter how the topology of the neural network changes, the model trained by the GRU is more accurate than that trained by LSTM. In addition, the training time of the GRU is generally more than 50 s shorter than that of LSTM when the number of layers and neurons are the same, thereby indicating that the GRU has advantages in addressing the problem of a medium-scale stress–strain sequence.

A comparison among neural network models with different topologies showed that the accuracy of the double-layer model with an equal number of neurons in the GRU was better. When the gap between the number of neurons in the hidden layer was too large, the prediction effect of the GRU was poor. For example, when the number of neurons in the first layer was 50, the prediction accuracy of the model decreased rapidly when the number of neurons in the second layer reached 200.

However, in LSTM, the structure of equal neurons is not optimal, and when the number of neurons in adjacent hidden layers has a gradient, this can improve the generalization ability of the model. Moreover, a sequential-structure RNN has the strongest generalization ability after the completion of training, but this effect decreases with the increase in the gap in the number of neurons in adjacent hidden layers, thereby indicating that the proper arrangement of the neural network structure with a gradient is appropriate, and the order of the network structure arrangement does not affect the training time. However, when the number of neurons between the two layers is too large, this will not only make the training time too long but also weaken the lifting effect and even produce negative effects.

The arrangement of the number of neurons in the two layers also satisfies a general rule, that is, the model trained when the number of neurons in the second layer is an even multiple of the number of neurons in the first layer has a better prediction accuracy than that trained when the number of neurons in the first layer is an odd multiple, and the structure of the sequential arrangement is more conducive to the improvement of the model performance.

In summary, the optimal structure of the GRU is B150-150, which is an equal neuron structure, and each hidden layer contains 150 neurons. The optimal structure of LSTM is A50-100, which is a non-equal neuron structure, and the first and second layers contain 50 and 100 neurons, respectively.

5.1.2. Training Frequency and Training Batch

The data training frequency and training batch are two important parameters of model training. The memory of the computer is limited, which is why the dataset is often sent to the model for training several times in the process of deep learning training to save memory and improve the training efficiency. The quantity training frequency is controlled by the number of feed-in data per batch, which in this model refers to the number of training inputs per stress–strain sequence, while the training batch refers to the total number of times the dataset is trained. In this case, the optimal structure of two kinds of RNNs (LSTM: A50-100, GRU: B150-150) was used to train the medium-scale stress–strain sequence, and the two parameters were studied by controlling the training frequency and training batches. The maximum training batch was set as 100, and the results are shown in Figure 10.

As can be seen from Figure 10, the MSE of the two models decreases continuously with the increase in training batches during the training process, which can be divided into three parts: rapid decline segment, transition segment, and stable decline segment. The decline rate is initially fast and then slow and gradually approaches 0. LSTM can essentially enter the stable decline stage after 10 training batches, while the GRU can enter the stable decline stage after only 5 training batches, indicating that the convergence speed of the GRU is faster. In addition, during the training process, the MSE descent curve of the GRU is smooth and the training process is more stable, while the LSTM keeps jumping in the stable descent stage, which indicates that the gradient descent of the model is less stable.

With the increase in each batch of input data, the training time will be greatly reduced, but the accuracy of the model will be decreased. Overall, when the number of batch sizes is the same, the accuracy of the GRU is always higher than that of LSTM, but when the batch size exceeds 140, the accuracy of the GRU is similar to or even higher than that of LSTM. When the training time and training accuracy are considered together, the model has higher accuracy and a shorter training time when the batch size is 60. Therefore, 60 is the preferred batch size of the model.

5.1.3. Learning Rate of the Optimizer

The deep learning optimizer selected in this paper is the Adam optimizer, and the calculation method of its updated parameters is shown in Equation (39). The learning rate $l_r$ in the formula is an important factor that affects the update of model parameters. In this paper, the influence of this parameter on the model is discussed by comparing the method of setting the constant learning rate and the method of setting the decay learning rate, as shown in Figure 11.

$$w_{t+1}=w_t-l_r·{\displaystyle \frac{m_t}{1-{\beta }_1^t}}/\sqrt{{\displaystyle \frac{V_t}{1-{\beta }_2^t}}}$$

(39)

where $w_{t+1}$ and $w_t$ are the parameters before and after the update; $l_r$ is the learning rate; $m_t$ and $V_t$ are the first-order momentum and second-order momentum, respectively; and ${\beta }_1^t$ and ${\beta }_2^t$ are the correction coefficients of first-order momentum and second-order momentum, respectively.

As can be seen from Figure 11, the MSE training curve of the model with a small learning rate is relatively smooth, but it cannot achieve a more accurate prediction effect after training 100 epochs. When the learning rate is large, although the curve can converge quickly, the curve oscillation is more intense, especially when the fixed learning rate is greater than 0.001. Whether with LSTM or the GRU, this phenomenon is more significant, which shows that the generalization ability of the neural network trained by the model is weak, showing false accuracy. The use of the attenuation learning rate setting method can alleviate the oscillation phenomenon of the MSE curve to a certain extent and achieve rapid convergence of the model while ensuring the reliability of the model because the characteristics of the attenuation learning rate method ensure that the learning rate gradually decreases with the increase in training batches. This feature ensures the stability of the model when a high learning rate is used in the early stage and a low learning rate is used in the later stage.

See Figure 12 for the results after extracting the number of epochs when the MSE is first reduced below 2.5 × 10⁻⁵ under different learning rate schemes. It can be found that the higher the initial learning rate of LSTM, the slower the convergence, while the opposite is true for the GRU. The use of a decaying learning rate enables the model to achieve the desired prediction accuracy in a lesser number of iterations. Since the two RNN structures chosen are already optimal, the neural networks trained to accurately predict the stress–strain curves often have no more than 70 iterations.

5.2. Forecast Result

The optimal neural network structure, training frequency, and optimizer learning rate of LSTM and the GRU were obtained by analyzing the effects of model parameters with a medium-scale dataset. The training ability and training speed of the original RNN in medium-scale datasets were no longer satisfactory. Thus, this paper offer analyses only of the prediction results of LSTM and the GRU.

In this section, we demonstrate how we applied the conclusions of Section 3 to train datasets of different sizes to analyze the predictive power of LSTM and the GRU. The structures of the two RNNs were selected as A50-100 and B150-150, the batch size was 60, the initial learning rate of the Adam optimizer was 0.001, the attenuated learning rate method was enabled, and the training batch was 100. The lengths of the stress–strain sequence were divided into short (200/400/600), medium (800/1000/1200), and long (1600/1800/2000). The dataset used in the analysis of model prediction ability was obtained through the calculation of the linear isotropic hardening constitutive model, and the dataset used in the analysis of model prediction results was based on the linear kinematic hardening constitutive dataset.

5.2.1. Model Prediction Ability

After the model was trained in the above steps, 10% of the samples in each group were selected as verification sets to verify the prediction results. In the process of verification, we found that the size of the training set had a great influence on the predictive ability of the model. With a dataset with a stress–strain sequence length of 1000 taken as an example, the model prediction results obtained by training datasets of different sizes are shown in Figure 13.

As can be seen from Figure 13, the model trained by LSTM with few data lacks generalization power. When the size of the training set is less than 6000, the model can essentially predict the trend of the hysteresis curve, but its deviation from the real values is large. When the number of training data reaches or exceeds 6000, the model shows a better prediction accuracy, but some mutation points are still present in the prediction results. These mutations occur in a small range and can be automatically corrected quickly. The GRU has a small demand for data, so it can predict hysteresis curves with a high accuracy when the number of data reaches 2000, and no data point mutation occurs, unlike in LSTM. The prediction results of the two models under different scales of stress–strain sequences are shown in

Table 2. Forecast results of LSTM.

Forecast Result		Training Set Size
		500	1000	2000	4000	6000	8000	10,000
Stress–strain scale	200	×	—	—	—	√	√	√
	400	×	×	—	—	√	√	√
	600	×	×	×	—	√	√	√
	1000	×	×	×	—	—	√	√
	1200	×	×	×	—	—	—	√
	1400	×	×	×	—	—	—	√
	1600	×	×	×	—	—	—	√
	1800	×	×	×	×	—	—	√
	2000	×	×	×	×	—	—	√

and

Table 3. Forecast results of GRU.

Forecast Result		Training Set Size
		500	1000	2000	4000	6000	8000	10,000
Stress–strain scale	200	—	—	√	√	√	√	√
	400	—	—	—	√	√	√	√
	600	×	—	—	√	√	√	√
	1000	×	—	—	√	√	√	√
	1200	×	—	—	√	√	√	√
	1400	×	—	—	√	√	√	√
	1600	×	—	—	√	√	√	√
	1800	×	×	—	√	√	√	√
	2000	×	×	—	√	√	√	√

, respectively. In the table, √ indicates an accurate prediction, — indicates that the trend can be predicted but has a small error, and × indicates that the trend cannot be predicted or the trend can be predicted but the error is large.

Table 2 and Table 3 comprehensively reflect the ability of the two models to predict stress–strain curves of different scales from a qualitative point of view. For small-scale series (length less than or equal to 600), both models have a strong prediction ability, but LSTM cannot make accurate predictions when there are fewer than 6000 data points, while the GRU can make accurate predictions when the number of data points reaches 4000. For medium-scale sequences (1000 to 1400 in length), LSTM requires 8000 or more data points for an accurate prediction, while the GRU still requires only 4000. For large-scale sequences (length greater than or equal to 1600), the LSTM’s predictive power is weak, requiring a data volume of more than 10,000 to ensure basic accuracy, while the GRU still requires 4000.

The prediction ability of the two models for stress–strain sequences of different sizes under different training set sizes was further demonstrated quantitatively by calculating the MSE value predicted by each model, which can reflect the difference between the predicted value and the real value, as shown in Figure 14.

As can be seen from Figure 14, the model’s prediction ability is positively correlated with the size of the training dataset and inversely correlated with the sequence length. When LSTM is used for training, and when the number of data points is less than 6000, increasing the amount of data can rapidly reduce the MSE of the predicted value; when the number of data is greater than 6000, increasing the amount of data can still reduce the MSE to a certain extent, but the curve tends to be flat and the decline rate low. This finding indicates that the data quantity cut-off point for LSTM prediction of the stress–strain curve problem is 6000, after which the prediction ability stops improving. However, when the GRU is used for training, the data quantity cut-off point is 4000, and the curve still has a large negative slope after 4000. This finding shows that although the GRU can accurately predict hysteresis curves after the number of data reaches 4000, the accuracy can still be improved to a certain extent with a further increase in the amount of data, and the training and prediction potential of the GRU model is greater than that of LSTM.

Figure 14 shows the prediction abilities of the two models for sequences of different lengths. For LSTM, when the number of data points is less than 6000, predicting sequences of various lengths is basically impossible. When the number of data reaches 6000, only short sequences can be predicted, and when the number of data points exceeds 8000, only medium-long sequences can be predicted. For GRUs, short sequences can be predicted after the number of data reaches 2000, and medium-long sequences can be predicted after 4000. The mean value of MSE in each length of series is taken as the predictive ability of the model for the length of series. In short, medium, and long sequences, the prediction abilities of the GRU were 6.13, 6.7, and 3.3 times those of LSTM, respectively.

5.2.2. Model Prediction Effect

After the prediction abilities of the two models were analyzed, three long sequences were selected for analysis to show the prediction effect of the models more clearly and assess the difference between the predicted value and the real value. All the models used were GRUs, as shown in Figure 15.

Figure 14 shows that the optimal parameter setting and training methods summarized in this paper produced relatively accurate prediction effects for the two commonly used steel structures. Driven by data, the deep learning model could better learn the two constitutive model features and quickly predict the stress sequence after the input strain sequence. The total time of this process did not exceed 0.01 s, thus having strong engineering application value.

In summary, the metal neural network model trained in this study can be well-applied to mechanical response prediction problems and has the potential to be integrated with the FEM. The computational framework of existing FEM program is largely the same, which is mainly based on the displacement field driven by integrating the global stiffness matrix and nodal force matrix and solving the equilibrium equations iteratively. The local iteration at the material level involves stress updating and consistent elastic–plastic tangent modulus (i.e., material Jacobian matrix) updating, where the constitutive model comes into play. Generally customized constitutive models can be implemented through material subroutine interfaces where the neural network model trained in this paper replaces the traditional constitutive model.

6. Conclusions

In this paper, deep learning model training methods for steel hysteresis curve prediction have been studied. On the basis of the linear isotropic hardening constitutive model and linear kinematic hardening constitutive model, random stress–strain curves were generated as datasets, and three kinds of RNNs were trained as models. Various model parameters and model training methods were discussed, and the following conclusions were drawn:

(1)

During data generation and processing, the data pre-processing method of the stress–strain curve prediction problem seriously affects the accuracy of the model. Using the nonlinear square reduction method to process the data to the same order of magnitude can better ensure that gradient explosion and gradient disappearance do not occur during the training of the model. The method of randomly generating a strain sequence and calculating a stress sequence based on a Gaussian random distribution is reasonable, and the generated curve can reflect the corresponding constitutive model.

(2)

In terms of model type and structure, RNNs are weak in stress–strain prediction, with a long training time and poor accuracy. The GRU and LSTM are suitable for this kind of problem. The marginal effect of the depth and number of simple stacked neural networks decreases, which is not only unable to effectively improve the model performance but also increases the training time. A two-layer structure can predict such problems. The optimal structures of LSTM and the GRU are A50-100 and B150-150, respectively.

(3)

In terms of model parameters and training methods, the accuracy of the model will increase with the decrease in batch size and the increase in training batches, but the training time will also increase. The optimal batch size and training batches are 60 and 100, respectively. The influence of the learning rate of the model was discussed. Setting the initial learning rate of 0.001 and setting the decay learning rate can improve the accuracy of the model.

(4)

The prediction ability of the model is positively correlated with the number of training set data and negatively correlated with the length of the required prediction curve. The GRU is less dependent on datasets than LSTM is, and both models are more accurate for short series’ prediction. In the case of short, medium, and long sequences, the prediction abilities of the GRU are 6.13, 6.7, and 3.3 times those of LSTM, respectively. Moreover, the proposed method can learn the law of two kinds of linear hardening constitutive models well and has a good hysteretic curve prediction ability.

(5)

The method proposed in this paper is mainly applicable to the prediction of the linear hardening principal relationship for steel and can predict the lengths of data series with up to 2000 entries. The trained constitutive neural network needs to be combined with a customized material interface if it is to be applied to existing commercial or open-source numerical simulation software, which will be further investigated in future research.