A Hard-Constraint Wide-Body Physics-Informed Neural Network Model for Solving Multiple Cases in Forward Problems for Partial Differential Equations

Abstract

In the fields of physics and engineering, it is crucial to understand phase transition dynamics. This field involves fundamental partial differential equations (PDEs) such as the Allen–Cahn, Burgers, and two-dimensional (2D) wave equations. In alloys, the evolution of the phase transition interface is described by the Allen–Cahn equation. Vibrational and wave phenomena during phase transitions are modeled using the Burgers and 2D wave equations. The combination of these equations gives comprehensive information about the dynamic behavior during a phase transition. Numerical modeling methods such as finite difference method (FDM), finite volume method (FVM) and finite element method (FEM) are often applied to solve phase transition problems that involve many partial differential equations (PDEs). However, physical problems can lead to computational complexity, increasing computational costs dramatically. Physics-informed neural networks (PINNs), as new neural network algorithms, can integrate physical law constraints with neural network algorithms to solve partial differential equations (PDEs), providing a new way to solve PDEs in addition to the traditional numerical modeling methods. In this paper, a hard-constraint wide-body PINN (HWPINN) model based on PINN is proposed. This model improves the effectiveness of the approximation by adding a wide-body structure to the approximation neural network part of the PINN architecture. A hard constraint is used in the physically driven part instead of the traditional practice of PINN constituting a residual network with boundary or initial conditions. The high accuracy of HWPINN for solving PDEs is verified through numerical experiments. One-dimensional (1D) Allen–Cahn, one-dimensional Burgers, and two-dimensional wave equation cases are set up for numerical experiments. The properties of the HWPINN model are inferred from the experimental data. The solution predicted by the model is compared with the FDM solution for evaluating the experimental error in the numerical experiments. HWPINN shows great potential for solving the PDE forward problem and provides a new approach for solving PDEs.

1. Introduction

Numerical modeling methods based on the finite element method (FEM) [1], finite difference method (FDM) [2], and finite volume method (FVM) [3] have made remarkable progress in solving physical problems. These methods are commonly used to describe partial differential equations (PDEs) for natural phenomena, relying on polynomials, segmented polynomials, and other elementary functions for modeling. Although traditional methods are considered to provide efficient and reliable solutions, their applicability is limited as the dimensionality of the problem increases, a phenomenon known as the “curse of dimensionality” (CoD) [4]. The CoD means that computational operations and resource requirements grow exponentially as the problem dimension increases, limiting the scope of application of these methods. In addition, grid-based methods suffer from discretization errors that fail to capture the required resolution of the modeling system when the grid size is not small enough, leading to inaccurate results. These challenges make it more difficult to solve PDEs in high-dimensional problems. When some physical problems with special properties are involved, such as chemical phase transition coupled porous media flow [5], lattice Boltzmann method coupled temperature field prediction [6], and microscopic fluid flow [7]. Complex chemical phase transitions and physical variability between the macroscopic and microscopic levels can lead to a dramatic increase in the computational cost of traditional numerical modeling methods. Faced with these physical problems with phase transitions and the obstacles in solving PDEs, researchers are actively looking for any potential solutions.

The rapid development of machine learning in recent decades has introduced new opportunities in various fields, having a significant impact on science and engineering and providing an opportunity for a shift in the solution paradigm. Machine learning offers brand new possibilities for numerical modeling, prediction, and forward problem solving for the study of physics problems.

Deep learning is a new and highly emphasized technique in the field of machine learning, one of the most important branches, and deep learning algorithms have been extremely effective in the fields of engineering and science, such as computer vision, speech recognition, natural language processing, assisted driving [8,9,10,11], etc. They can process large-scale data based on the special structure of their networks, extract high-level features, and achieve excellent performance and accuracy in various tasks such as image classification, target detection, semantic segmentation, machine translation [12,13,14,15], etc. Typical DNN models include multilayer perceptron (MLP) [16], convolutional neural network [17], and recurrent neural network (RNN) [18].

Physics-informed neural networks (PINNs), as new deep learning algorithms, can integrate physics law constraints with neural network algorithms for PDEs. They provide new methods for solving PDEs in addition to the traditional numerical modeling methods. PINNs predict the results of PDEs without constructing a detailed grid, thus overcoming the CoD problem experienced by traditional numerical modeling methods. Instead of using a grid for spatiotemporal stepping, PINNs collect points irregularly from the defined domain through different collection distributions [19]. The PDEs are embedded in PINNs in the form of a loss function, rather than an algebraic matrix as in traditional numerical modeling methods. The gradient optimizer is a residual minimizer in PINNs [20], which is quite different from the linear solver in traditional numerical modeling methods.

The paradigm shift of PINNs provides a new approach to solving PDEs for practical physical applications where they are needed. Raissi et al. [21,22] proposed and used PINNs to solve several cases of PDEs. They applied PINNs to solve NS equations to simulate the flow around the circular cylinder case. The SRT-LBM-PINN model was proposed by Liu et al. [23], who used PINNs in combination with a single relaxation time lattice Boltzmann method to solve inverse problems in fluid mechanics. They used PINNs to solve PDEs with unclear boundary conditions and initial conditions. Lou et al. [24] solved the forward–backward problem in fluid dynamics by using PINNs in conjunction with the BGK equation. In addition, the performance of PINNs is extremely promising for applications in the engineering sciences. The stiff-PINN method developed by Ji et al. [25] utilizes QSSA to enable PINNs to solve for stiffening chemical kinetics, which opens up the possibility of applying PINNs to a wide variety of reaction–diffusion systems involving stiffening kinetics. Huang et al. [26] applied PINNs in a power system. Zhong et al. [27] proposed two generalized artificial intelligence frameworks for low-temperature plasma simulations, the coefficient-subnetwork (CS-PINN) physics-informed neural network and the Runge–Kutta physics-informed neural network (RK-PINN). PINNs are also prominent in coastal storm surge prediction [28] and geophysics [29], where they provide functions including state analysis, parameter estimation, dynamic analysis, macroscopic physical quantity computation, anomaly detection and localization, and data synthesis for real-world problems by integrating specific physical information into state-of-the-art deep learning methods.

Although PINNs have been widely used in solving PDEs, they still face many challenges, such as poor interpretability, difficult debugging of parameters, training difficulties, and unclear optimization strategies. Especially when dealing with higher-dimensional physical problems, PINNs consume a lot of time for training and debugging. For problems with nonunique solutions, traditional PINNs may not appear smooth enough in design. The hard-constraint physics-informed neural network (HPINN) was proposed by Lu et al. [30] for solving topology optimization problems. Lu et al. found that compared with traditional constrained optimization methods for PDEs based on the adjoint method and numerical PDEs solvers, HPINN is able to achieve similar optimization objectives and the resulting designs are smoother when dealing with problems with nonunique solutions.

HPINN takes advantage of the recent advances in PINNs for solving PDEs without the need for large-scale datasets (generated by numerical PDEs solvers) for training and imposes hard constraints using penalization and augmented Lagrange methods. The training and debugging of HPINN re still extremely challenging. HPINN has advantages in terms of fast convergence and accuracy as well as solution smoothness. HPINN abandons the flexible boundary conditions and initial condition settings of traditional PINNs, making it extremely difficult to formulate an optimization strategy for HPINN.

In order to improve the performance of HPINN and develop an optimization strategy, we propose a hard-constraint wide-body network PINN model (HWPINN). HWPINN introduces a wider neural network structure, which enhances the approximation ability of neural networks. By combining the highly constrained nature of HPINN, HWPINN further improves the accuracy of PINNs in solving PDEs while reducing the time and number of iterations required for training. This model provides a new optimization idea for optimizing HPINNs from the point of view of changing the network model, which offers the possibility of solving PDEs in positive problems with higher accuracy and speed.

In this manuscript, we focus on three cases of PDEs: the 1D Allen–Cahn (AC) equation, the 1D Burgers equation, and the 2D wave equation. Among them, the 1D AC equation and the 1D Burgers equation are common PDEs in phase transition dynamics and computational fluid dynamics, and the code for their tensor-flow versions is provided in reference [22]. To compare the performance of HWPINN in different dimensions, the 2D wave equation is provided in this paper. HWPINN is able to provide higher PDE solution accuracy and faster training time than the traditional soft-constraint PINN. A series of numerical experiments were set up for the case of PDEs of interest to compare the performance of HWPINN with respect to soft-constraint PINN (SPINN) and HPINN, while appropriate test hyperparameters were applied for each experiment. It is worth mentioning that SPINN can also provide better prediction owing to its wide-body structure, but the constraints (residual function) of SPINN can be flexibly adjusted; the soft-constrained wide-body PINN (SWPINN) was used as a comparison test in this study.

The rest of the manuscript is set up as follows: Section 2 describes the experimental methodology and the design approach for PDEs combined with PINNs, as well as the network structure. Section 3 shows the experimental results obtained with the four models, SPINN, HPINN, SWPINN, and HWPINN, to solve the three PDEs cases of interest. Section 4 presents our conclusions and future work.

2. Materials and Methods

2.1. Partial Differential Equation (PDE) Cases

This manuscript focuses on three cases of PDEs: the 1D Allen–Cahn equation (AC equation), the 1D Burgers equation, and the 2D wave equation.

The AC equation is a PDE that describes the phenomenon of phase transitions, and it is widely used in materials science [31]. The AC equation is used to describe the evolution of interfaces between different phases (e.g., solids and liquids) in a material. An important characteristic of the AC equation is that it is capable of modeling fuzzy regions of phase transition interfaces, rather than just well-defined boundaries. The standard form of the Allen–Cahn equation can be written as:

$$\frac{\partial u}{\partial t}={ϵ}^2{\nabla }^2-W’\left(u\right)$$

(1)

where $u(x, t)$ is the function that describes the state of the material, $t$ represents time, $x$ is the spatial coordinate and $ϵ$ is a small positive number referred to as the interface width parameter. The $W\left(u\right)$ is potential energy function, $W’\left(u\right)$ is the derivative of $W\left(u\right)$ with respect to u. The first term ${ϵ}^2{\nabla }^2$ of this equation represents the diffusion term, which describes the ambiguity of the phase transition interface. The second term $-W’\left(u\right)$ represents the potential energy term, which describes the difference in energy of the material between phases, driving the system to tend to reach an energy minimum.

The 1D AC equation is a special case of the Allen–Cahn equation in which only one spatial variable $x$ is involved and can be expressed as follows:

$$\frac{\partial u(x,t)}{\partial t}={ϵ}^2\frac{{\partial }^{2}u(x,t)}{{\partial x}^2}-W^{’}\left(u\right),$$

(2)

The Burgers equation integrates a nonlinear convection term and a viscous diffusion term and can be used to model a variety of complex fluid dynamics phenomena [32]. The Burgers equation is a 1D nonlinear PDE, which is expressed as follows:

$$\frac{\partial u(x,t)}{\partial t}+u\left(x,t\right)·\frac{\partial u\left(x,t\right)}{\partial x}=v\frac{{\partial }^{2}u\left(x,t\right)}{{\partial x}^2},$$

(3)

where $u(x,t)$ represents the velocity field as a function of space $x$ and time $t$, and $v$ is a positive constant and represents the viscous coefficient.

The wave equation is a partial differential equation that describes the propagation and interaction of waves in space. It is widely used in physics, engineering, and mathematics to study the propagation behavior of water, sound, and light waves in a plane [29]. Equation (4) denotes the lossless two-dimensional wave equation:

$$\frac{\partial u(x,y,z)}{\partial t^2}=c^2\left(\frac{\partial u\left(x,y,t\right)}{\partial x^2}+\frac{{\partial }^{2}u\left(x,y,t\right)}{{\partial y}^2}\right),$$

(4)

where $u(x,y,t)$ denotes the displacement of point $(x,y)$ at time instance $t$ for the wave in the plane, and $c$ is the velocity of the 2D wave.

2.2. Research Program

PINNs leverage deep neural networks (DNNs) to approximate the intermediate quantities within partial differential equations (PDEs). The automatic differentiation capabilities of Pytorch or Tensorflow are then employed to solve these PDEs. To incorporate physical insights into the neural network training process, PINNs establish a residual network. This network combines physical information, typically represented using PDEs, with an automatic differentiation functionality, imposing constraints on the DNN. This approach integrates the principles of physics directly into the neural network training procedure, enhancing the model’s ability to capture and simulate complex physical phenomena.

Figure 1 depicts a network architecture of a traditional PINN architecture when solving PDEs. The traditional PINN architecture takes the form of a soft constraint. The NN architecture in Figure 1 represents the structure of the deep neural network, where the input part $x$ represents the spatial coordinates, and $t$ represents the time. The red network represents an approximator network that estimates the solution $u(x,t)$ of the PDE, where a is the activation function. The approximator network is formed by a neural network with $k$ layers. In the physics-informed part, $\frac{\partial N}{\partial x^N}$ represents the spatial derivative, and $\frac{\partial M}{\partial t^M}$ represents the time derivative, which comprise the residuals of the PDEs. For the IC/BC part, $u^{*}(x,t)$ is the boundary condition. $W^{\mathrm{*}}$ and $b^{*}$ represent the optimized network parameters. N_c and N_IC/BC represent the training and boundary points, respectively. The variable $\epsilon$ is the loss threshold.

Figure 2 illustrates the HWPINN architecture. In contrast to the PINN structure shown in Figure 1, a wide-body structure is added in the approximator network part in the HWPINN illustrated in Figure 2. The neural network in the blue section is the added wide-body model, which is a separate neural network and participates in every feedforward process of the main neural network. For the IC/BC part, the hard-constraint method is used to add constraints. It is worth noting that the hard-constraint method is imposed using the penalty method and the augmented Lagrange method (ALM) [33]. The method of imposing hard constraints is explained in detail in Section 3.

2.3. Loss Function

The goal of PINN is to minimize the loss function in Equation (5). The total loss function value is given by Equation (5), which is composed of the residuals for the PDEs and the mean square error of the initial and boundary conditions. In the loss function (Equation (5)), we introduce a set of weighting factors $w=({\beta }_1,{\beta }_2,{\beta }_3)$ to address the issue of disparate error magnitudes among different terms. These weighting factors ensure that the different components of the loss function have equal importance in the optimization process.

$$L={\beta }_1·L_{pde}+{\beta }_2·L_{IC}+{\beta }_3·L_{BC},$$

(5)

$$L_{pde}=\frac{1}{N_c}\sum _{n=1}^{N_c}|R_{pde}(x_n,t)-0{|}^2,$$

(6)

$$L_{IC}=\frac{1}{N_{IC}}\sum _{n=1}^{N_{IC}}|u(x_n,t)-u^{*}(x_n,t){|}^2,$$

(7)

$$L_{BC}=\frac{1}{N_{BC}}\sum _{n=1}^{N_{BC}}|f_{bc}(u(x_n,t)-u^{*}(x_n,t){)|}^2,$$

(8)

Equation (6) displays the loss function for the residual values of the PDEs. The residual values of the PDE $R_{pde}$ in Equation (6) can be obtained by shifting the governing equation of the original PDE. The following is an example of the 1D AC equation expressed in Equation (1):

$$R_{pde}\left(x,t\right)={ϵ}^2{\nabla }^2-W^{’}\left(u\right)-\frac{\partial u}{\partial t}.$$

(9)

In Equation (6), N_c represents the points generated for training in the computational domain. Equations (7) and (8) represent the loss values for the initial conditions and boundary conditions, respectively. N_BC and N_IC represent the sampled points that meet the initial conditions and the points on the boundary conditions, respectively. In Equation (7), $u(x,t)$ represents the result predicted by the neural network, and $u^{*}(x,t)$ represents the exact value from the initial conditions. In Equation (8), $f_{bc}$ represents the type of boundary condition. Periodic boundary conditions [34] are used for all cases in this study.

For hard-constraint PINNs, the weight coefficients in Equation (5) need to be set to $w=(\mathrm{1,0},0)$. And, a hard constraint function needs to be implemented in the program. The hard-constraint functions need to be set based on concrete boundary conditions and initial condition information, so they are described in Section 3.

2.4. Optimization Strategies

In order to obtain more accurate results for the HWPINN proposed in this paper, we preprocessed the data and adopted multiple optimization strategies when training the network.

In the application of PINN to solve PDE problems, utilizing min–max normalization holds significant importance [35]. This normalization method ensures that input and output data fall within similar numerical ranges, enhancing the numerical stability of the neural network. When the numerical disparity between input and output data is significant, unnormalized data might lead to numerical instability during the training process, affecting the model’s convergence and accuracy. Min–max normalization helps to speed up the training process. After normalization, the optimization algorithms for neural networks converge more easily. Methods such as gradient descent can find appropriate weights and biases faster, thus improving training efficiency. Using the normalization scheme not only improves the numerical stability of the model but also speeds up the training process. This approach provides an effective numerical treatment for solving complex PDE problems in scientific research.

The equation for min–max normalization is as follows:

$$Y_{normalized}=\frac{Y-min\left(Y\right)}{\mathit{max}\left(Y\right)-min\left(Y\right)}$$

(10)

where $Y$ represents the original data; $Y$_normalized is the normalized data; $max\left(Y\right)$ and $min\left(Y\right)$ are the minimum and maximum values of the original data, respectively. Subtracting the minimum value from the original data and then dividing by the difference between the maximum and minimum values yields the normalized value. This process scales the data range to [0, 1].

For initializing the neural network weights and biases, we used Xavier initialization. The hyperbolic tangent function (Tanh) was chosen for the activation function. Xavier initialization, also known as Glorot initialization [36], is a method specifically designed for initializing neural network weights, especially when using S-shaped activation functions like Tanh. The advantage of Xavier initialization lies in its ability to mitigate the issues of vanishing and exploding gradients associated with these activation functions. When employing Tanh as the activation function, its output range is limited to between −1 and 1. Initializing weights too large results in Tanh saturating to 1 or −1, causing gradients to approach zero and leading to the vanishing gradient problem. Initializing weights too small makes the activations close to zero, again resulting in very small gradients and the same vanishing gradient problem.

Xavier initialization intelligently adjusts the initial weight range based on the number of input and output units in the preceding layer. By initializing weights from a uniform or Gaussian distribution with a standard deviation related to the input and output unit numbers, Xavier initialization ensures that activations are in a suitable range, preventing vanishing and exploding gradients. Therefore, Xavier initialization, when applied with Tanh or similar activation functions, accelerates network convergence, enhances training efficiency, and effectively addresses the challenges associated with gradient-related problems, making neural networks more trainable.

In our cases, training is achieved by minimizing the error using a variant of the stochastic gradient descent (SGD) method of adaptive moment estimation weight-decay (AdamW) [37] and Broyden–Fletcher–Goldfarb–Shanno (BFGS) [38]. Training begins by applying the AdamW optimizer to the constructed model. This training phase is stopped according to a predetermined number of training calendar elements. Subsequently, another training phase is performed using the L-BFGS-B optimization method. This optimization sequence set up compensates for the limited amount of training data by reducing computational loss and is expected to achieve faster convergence [20].

3. Results

In the results in this section, for computational convenience, all physical parameters were nondimensionalized, ensuring consistency in the flow and heat transfer criteria before and after nondimensionalization. In the experimental part, all the models were trained using the same convergence conditions, and the training was stopped at 10,000 iterations. All modeling experiments were performed on machines equipped with NVIDIA TITANX GPUs and Windows operating systems in Shanghai, China. The back end of all PINN designs was implemented using Pytorch, where the code for the 1D AC equation and the 1D Burgers equation was converted from the Tensorflow code provided in reference [22].

3.1. One-Dimensional Allen–Cahn Equation

The Allen–Cahn equation, a prominent mathematical model in the realm of material science and physics, captures the evolution of phase boundaries in various materials [39]. This PDE delineates the gradual transition between different phases within a material. The 1D AC equation is commonly used to study phase transition problems in linear structures, such as the movement of interfaces and the propagation of phase transitions [40].

The 1D AC equation case in this subsection is set up as follows:

$$u_t-0.0001u_{xx}+5u^3-5u=0$$

(11)

The boundary conditions and initial conditions are set as follows:

$$\begin{gathered} u\left(x,0\right)=x^{2}cos\left(\pi x\right) \\ u\left(-1,t\right)=u\left(1,t\right) \\ {u}_x\left(-1,t\right)=u_x(1,t) \end{gathered}$$

(12)

The diffusion term ${ϵ}^2{\nabla }^2$ is set to 0.0001, where $x\in [-1, 1]$ and $t\in [0, 1]$. The 1D AC equation case is a PDE with periodic boundary conditions.

Hard constraints are implemented by adding a hard constraint function to the physics-informed part of the network architecture, which is set as follows:

$$u_h\left(x,t\right)=x^2\mathit{cos}\left(\pi x\right)+t·u\left(x,t\right),$$

(13)

where $u_h(x,t)$ denotes the hard-constraint function, and $u(x,t)$ is the output of the approximator neural network (in Figure 1). The hard-constraint function mandatorily constrains the output of the neural network in terms of boundary and initial conditions, making it linearly smoother and closer to the exact solution of the PDE.

In the case of the 1D AC equation, we designed a series of experiments to verify the performance of the HWPINN proposed in this paper. There were four neural network architectures used in the numerical experiments: HWPINN, hard-constraint PINN (HPINN), traditional PINN with soft constraint (SPINN), and a wide-body soft-constraint PINN (SWPINN).

As shown in

Table 1. Hyperparameters of HWPINN, HPINN, SPINN, and SWPINN models.

Model	Layers	Neurons	Iterations
HPINN	8	64	10,000
HWPINN	8	32 × 2	10,000
SPINN	8	64	10,000
SWPINN	8	32 × 2	10,000

, for the parameter settings of the neural network part, both HWPINN and SWPINN used the wide-body model of the neural network architecture. The neural network part was set to 8 layers and 32 neurons in the hidden layers; the wide-body network part was set to 8 layers and 32 neurons. For HPINN and SPINN, the parameters in the neural network were set to 8 layers of 64 neurons in the hidden layers, and the maximum training number was set to 10,000. To verify that the wide-body model was not simply widening the network itself, the total number of settings for the number of neurons in the hidden layers for all four PINN models was 64.

Benefiting from the properties of PINN in solving forward problems in PDEs, no specific dataset is required for training, and PINN discretely collects training points randomly as training data within the space–time of the computational domain. In this case, the density of the collection points is set to $N_c=31$, and the density of the collection points on the boundary was similarly set to $N_{IC/BC}=31$. These collection points contain only spatial coordinate information $x$, and discrete temporal information $t$.

From the training loss curves shown in Figure 3, it can be concluded that the HWPINN and HPINN models converge quickly under the set conditions, and due to the very low density of the collection points, the training loss curves of the SWPINN and SPINN models show overfitting, which needs to be solved by changing the training parameters. A comparison of the training loss curves showed that HWPINN can extract abstract features from the solution space of PDEs faster than other models and is more effective in solving high-dimensional problems [30].

Figure 4 illustrates the FDM solution of the 1D AC equation (Equation (11)). The results predicted by the four neural network architectures are presented in Figure 5. The results in Figure 5 show that HPINN and HWPINN outperform SPINN and SWPINN for a small number of iterations (maximum number of iterations = 10,000).

For a more intuitive comparison of the effects of HPINN, HWPINN, SPINN, SWPINN, a comparison chart of the absolute error visualization is provided in Figure 5. The absolute error $E_{abs}$ is defined as follows:

$$E_{abs}=\left|u_{FDM}-u_{predit}\right|,$$

(14)

where $u_{FDM}$ denotes the results of FDM, and $u_{predict}$ represents the results predicted using the PINN models.

Figure 6 visualizes the absolute errors of the four model predictions with respect to the FDM results, more intuitively demonstrating the advantages of HWPINN over SWPINN, SPINN, and HPINN, especially when the collection point density was low and fewer iterative loops were set up in the experiments. The results in Figure 6a,b present the absolute error of HPINN and HWPINN; their comparison clearly shows the strength of HWPINN. The results in Figure 6c,d show that the PINN with soft constraint performed badly with the very low density of collection points and number of iterations designed in this experiment and that the loss functions of all four architectures converged when the maximum number of iterations (10,000) was reached.

Table 2. Error evaluation and computational time of the HWPINN, HPINN, SPINN, and SWPINN models for the prediction of 1D AC equations.

Model	MSE (%)	RMSE (%)	R2 (%)	Relative L2 Error (%)	Computational Time (s)
HPINN	0.36%	6.01%	99.21%	8.49%	1641
HWPINN	0.25%	5.03%	99.45%	7.10%	1654
SPINN	14.18%	37.66%	68.86%	53.10%	1752
SWPINN	14.00%	36.43%	69.23%	52.78%	1791

lists the mean square error (MSE), root mean square error (RMSE), R-squared (R2), and relative L₂ error of the prediction results of the four models: HPINN, HWPINN, SPINN, and SWPINN. The relative L₂ error is defined as follows:

$${err}_{L_2}=\frac{\sqrt{\sum _{i=1}^N|{\gamma }_i-{\gamma }_i^{*}{|}^2}}{\sqrt{\sum _{i=1}^N|{\gamma }_i^{*}{|}^2}},$$

(15)

where $N$ represents the number of predicted points, $Y_i$ denotes the results predicted by the neural networks, and $Y_i^{*}$ represents the results of FDM.

By analyzing the data in Table 2, we conclude that HWPINN demonstrated significantly better performance than the other PINN architectures under the experimental conditions we set. Due to architectural advantages, the computation time of HWPINN and HPINN was better than that of SPINN and SWPINN. HPINN already has many advantages in solving the 1D AC equation, and HWPINN is a complementary model to the HPINN model, which is characterized by faster convergence and slightly better predictions than HPINN when working with small datasets.

3.2. One-Dimensional Burgers Equation

The 1D Burgers equation is typically used to study phenomena in the fields of fluid dynamics, nonlinear fluctuation theory, and phase transition dynamics. This equation describes the nonlinear fluctuation behavior in a nonviscous, incompressible fluid. Although its mathematical form is relatively simple, its analytical solution is often difficult to obtain due to the inclusion of nonlinear terms, causing researchers to focus more on numerical simulations, approximation methods, and mathematical analysis.

The numerical experimental case set up for the 1D Burgers equation is as follows:

$$\begin{gathered} u_t+{uu}_x-\left(0.01/\pi \right)u_{xx}=0, \\ u\left(x,0\right)=-\sin \left(\pi x\right), \\ u\left(\pm 1,t\right)=0 \end{gathered}$$

(16)

Equation (16) denotes the 1D Burgers equation with the coefficient of viscosity $v=0.01/\pi$, and both boundary and initial condition settings are included. Equation (16) can be obtained via shifting the terms of Equation (2). The range is set to $x\in \left[-1, 1\right],t\in [0, 1]$.

$$u_h\left(x,t\right)=-\mathit{sin}\left(\pi x\right)+\left(x+1\right)\left(x-1\right)·t·u(x,t)$$

(17)

The hard-constraint function of the 1D Burgers equation is shown in Equation (17). The hard constraint is implemented in the same way as in the case of the AC equation, imposed according to the initial and boundary conditions. $u\left(x,t\right)$ is the output of the approximator neural networks. $u_h\left(x,t\right)$ is output of the neural network part of PINNs with hard constraints imposed using the penalty method and the ALM.

The input of the neural network is $\left(x,t\right),$ and the output is $u(x,t)$. HWPINN and SWPINN use the same neural network parameter settings. The neural network part was set to 8 layers and 32 neurons in the hidden layers, the wide-body network part was set to 8 layers and 32 neurons. For HPINN and SPINN, the parameters in the neural network were set to 8 layers of 64 neurons in the hidden layers, and the maximum number of iterations was set to 10,000 iterations. The density of the collection points was set to $N_C=51$, and the density of the collection points on the boundary was similarly set to $N_{IC/BC}=51$. These collection points contained only spatial coordinate information $x$ and discrete temporal information $t$.

Figure 7 illustrates the training loss curves for HWPINN, HPINN, SPINN, and SWPINN. It can be inferred that the PINN models (HWPINN and SWPINN) that incorporate wide-body structures performed more smoothly in terms of the training loss curves. It is worth mentioning that the residual function was discarded when hard constraints were used, resulting in a loss function value that does not match the traditional soft-constrained PINNs (SPINN and SWPINN) in terms of scale, and the results presented in Figure 7 are only used as a reference for approximating the smoothness. The accuracy of the prediction results was compared in terms of percentage error.

Figure 8 shows the FDM solution of the 1D Burgers equation set up in the numerical experiments described in this subsection. Figure 9a–d display the results of the 1D Burgers equation predicted using HPINN, HWPINN, SPINN, and SWPINN; a comparison revealed that HWPINN, HPINN, and SWPINN all perform relatively well. SPINN performs poorly due to the fact that very few sampling points were used for training, resulting in a residual neural network constrained by the residual function that fails to capture the features of the solution space of the 1D Burgers equation. In order to more intuitively demonstrate the prediction accuracy of the four models, Figure 10 provides a visualization of the absolute error.

The maximum absolute error of HWPINN is not more than 0.4 in the absolute error distribution plot of HWPINN shown in Figure 10b, which shows the superior advantage of the model proposed in this paper in terms of fast convergence to provide high accuracy in the case of sparse collection point density. It is worth noting that as the collection point density increases, the SWPINN model with the addition of the wide-body structure is also able to capture the abstract features of the solution of the 1D Burgers equation with relatively low-density collection points. This confirms that the inclusion of the wide-body structure has the effect of enhancing the performance of neural networks in the PINN architecture.

The error evaluation of HWPINN, HPINN, SPINN, and SWPINN are presented in

Table 3. Error evaluation and computational time of the HWPINN, HPINN, SPINN, and SWPINN models for the prediction of 1D Burgers equation.

Model	MSE (%)	RMSE (%)	R2 (%)	Relative L2 Error (%)	Computational Time(s)
HPINN	0.34%	5.91%	99.07%	9.59%	2144
HWPINN	0.23%	5.01%	99.45%	6.10%	2145
SPINN	19.98%	44.64%	47.86%	72.67%	2245
SWPINN	1.22%	11.07%	96.74%	18.03%	2290

, where it can be observed that HWPINN has a significant advantage over the other models in solving the 1D Burgers equation for the experimental set up. The SWPINN model also obtained predictions with some accuracy due to the increase in the number of collection points, which demonstrates the enhancement of the wide-body structure for the Approximator neural network in the PINN architecture. In terms of computational time, HWPINN still has an advantage over traditional PINNs.

3.3. Two-Dimensional Wave Equation

In this subsection of numerical experiments, we extended the PDE case from 1D to 2D, with the aim of validating the performance of the proposed HWPINN model in 2D problems. Having extensibility across dimensions is one of the advantages provided by the PINN architecture in combination with strict physical laws.

The wave equation finds diverse applications in oceanography, including simulating waves, tides, and tsunamis; analyzing marine structures’ and ships’ responses to waves and studying underwater acoustics, providing essential insights for marine engineering, safety, and environmental monitoring [41].

The 1D wave equation is a mathematical model describing fluctuations propagating along a straight line, and the 2D wave equation is its generalization in 2 dimensions, introducing a variable $y$. For the definition of a 2D wave equation, see Equation (3). The experimental set up for the 2D wave equation is as follows:

$$\begin{gathered} u_{tt}=c^2(u_{xx}+u_{yy}), \\ u\left(x,y,t\right){|}_{t=0}=e^{{-[(x-3)}^2{+(y-3)}^2]} \\ {u}_t{|}_{t=0}=0. \end{gathered}$$

(18)

where $(x,y,t)$ is the input, $u(x,y,t)$ is the output of the neural network and the velocity of wave $c$ = 1. The range is set to $x\in \left[0,6\right],y\in \left[0,6\right],t\in [0,1]$. The physical definition of this can be understood as applying a peak initial velocity to the membrane $\Omega =\left[\mathrm{0,6}\right]\times \left[\mathrm{0,6}\right]$ in a 2D region at moment t = 0. The membrane vibrates back and forth in time, and, at the same time, the resulting wave propagates in all directions. The initialization of the 2D wave equation is shown in Figure 11. The 2D wave field is formed in a two-dimensional plane. The solution to the 2D wave equation is a function of the wave field.

The implementation of the hard-constraint function is defined by the following:

$$u_h\left(x,t\right)=e^{{-[(x-3)}^2{+(y-3)}^2]}+\frac{1}{36}·t·u(x,t)$$

(19)

where $u_h\left(x,t\right)$ is the output constrained by the hard-constraint function. $u(x,t)$ represents the output of the approximator neural networks.

As for the network parameters, HWPINN and SWPINN used the same neural network parameter settings. The neural network part was set to 8 layers and 32 neurons in the hidden layers, the wide-body network part was set to 8 layers and 32 neurons. For HPINN and SPINN, the parameters in the neural network were set to 8 layers of 64 neurons in the hidden layers, and the maximum number of iterations was set to 10,000 iterations. The density of the collection points was set to $N_C=31$, and the density of the collection points on the boundary was similarly set to $N_{IC/BC}=31$. These collection points contained only spatial coordinate information $(x,y)$ and discrete temporal information $t$.

As shown in Figure 12, the convergence intervals of the training loss curves of HWPINN, HPINN, SPINN, and SWPINN are very close to each other in the case of the 2D wave equation, and it can be noticed that the training curves of HWPINN and HPINN are much smoother. It can be concluded that the network structure of HWPINN is more numerically stable than those of SPINN, SWPINN, and HPINN. It is worth noting that since HWPINN adds physical constraints in a different way than traditional residual neural networks, the loss function has a certain scale difference, which explains the different intervals of the loss curve values. The loss curves reveal the fast convergence property of HWPINN.

Figure 13 illustrates the FDM results for the 2D wave equation, presented in the form of three time-slices in the computational time domain. Figure 14 shows the predictions for HWPINN, HPINN, SPINN, and SWPINN at time slices $t=0$, $t=0.1$, and $t=1$. The absolute errors of the prediction results for these four models are shown in Figure 15 to give a more visual indication of the performance when solving the 2D wave equation. The comparison shows that HWPINN has the best performance in terms of absolute error, which does not exceed 0.02 over the entire time space of the computational domain. SPINN predicts the worst absolute error, which demonstrates the improvement in the prediction stability provided by HWPINN with respect to the conventional PINN.

Figure 15a–c show the absolute errors of the HPINN prediction results, and Figure 15d–f show the absolute errors of the HWPINN prediction results. We conclude from the comparison that the HWPINN provides more accurate predictions and the stability is improved relative to those of the HPINN.

The error evaluation is provided in

Table 4. Error evaluation for HWPINN, HPINN, SPINN, and SWPINN models for the prediction results of 2D wave equation.

Model	t (s)	MSE (%)	RMSE (%)	R2 (%)	Relative L2 Error (%)	Computational Time (s)
	0	0.00%	0.00%	99.99%	0.00%
HPINN	0.1	0.00%	0.03%	99.99%	0.14%	2587
	1.0	0.03%	0.55%	99.63%	4.41%
	0	0.00%	0.00%	99.99%	0.00%
HWPINN	0.1	0.00%	0.00%	99.99%	0.00%	2593
	1	0.01%	0.22%	99.93%	1.81%
	0	0.00%	0.03%	99.96%	1.67%
SPINN	0.1	0.43%	0.66%	99.87%	3.22%	2723
	1	31.36%	56.43%	79.23%	65.78%
	0	0.00%	0.34%	99.96%	1.67%
SWPINN	0.1	0.00%	0.66%	99.87%	3.22%	2811
	1	0.01%	0.44%	99.76%	3.56%

. The data in Table 4 show that HWPINN performs optimally and SPINN performs the worst under the set conditions, which reflects the fact that SPINN is difficult to debug although it can flexibly adapt to various boundary conditions. The comparison of the relative L₂ errors of HWPINN and HPINN demonstrates that HWPINN is a more accurate predictor than HPINN, improving the accuracy by a factor of about three. Changing from 1D PDEs to 2D PDEs significantly increases the complexity of the computation and the computation time. In particular, in this case, HWPINN and HPINN have a significant advantage in computation time over conventional PINN due to the hard-constraint-specific properties, which can increase the speed.

4. Conclusions

In this paper, we proposed the HWPINN neural network architecture based on a wide-body structure and hard-constraint method, and we set up a series of numerical experiments using PINN to solve PDEs. Numerical experiments were designed using sparser PINN collection point densities in conjunction with defined boundary conditions, and the results show that HWPINN converges faster and makes more accurate predictions than SPINN.

HWPINN is better than the PINN architecture with soft constraints in terms of the accuracy of the results and the speed of convergence. HWPINN still has disadvantages because it needs to be modeled with hard-constraint functions. Hard constraints also have the disadvantage of being inflexible and difficult to debug. Additionally, HWPINN does not perform consistently in dealing with some flexible boundary conditions. For example, when solving problems with Neumann boundaries, a residual function still needs to be construct to allow HWPINN to participate in fitting the constrained part of the process using traditional PINN technologies to increase its stability. In solving real physical problems, HWPINN needs a more refined design when facing complex boundary conditions. The flexibility and powerful approximation ability of traditional PINN enable it to handle all kinds of PDEs, whether homogeneous or nonhomogeneous, linear or nonlinear, systematic or monolithic; HWPINN discards the flexibility of PINN in boundary conditions to make the approximation smoother, converges more rapidly, is convenient for debugging, and provides better overall performance. When solving higher-dimensional problems, such as porous media flow, phase transition dynamics, and supercritical airfoil optimization, PINN is often confronted with the problem of being unable to debug and difficult to compute; then, the advantages of HWPINN, including ease of computation and smoothness of the fit, allow the modeling to be carried out normally.

For the part of solving PDEs with which we were concerned, we set up several cases” 1D AC equation, 1D Burgers equation, and 2D wave equation. The experimental results for different dimensional PDEs showed that HWPINN has better numerical stability; due to the more efficient network structure design, HWPINN can also converge to a suitable solution when the number of training iterations is small. For the traditional PINN, SPINN, and SWPINN with an added wide-body structure, the performance is not advantageous under the experimental conditions we set. The reason for this is that although PINN with soft constraints can more flexibly adapt to different problems, it requires more training iterations and more complex optimization strategies due to the complexity and instability of its constraints.

In future work, we envision utilizing HWPINN to solve a wider range of PDE problems, with a special emphasis on utilizing HWPINN to solve positive and negative PDEs. In addition, we are keen to explore the synergies between HWPINN and physical models, with the intention of adopting this combined approach to efficiently solve PDEs in real-world problems. The potential applications in phase transition dynamics and CFD offer exciting prospects for pushing the limits of HWPINN’s capabilities and refining its performance in a variety of complex physical scenarios. As our research progresses, our focus will extend beyond the traditional PDE cases explored in current experiments, and we will explore ways to improve the stability of HWPINN in solving real-world physics problems. We anticipate that incorporating HWPINN into the realm of phase transition dynamics and CFD will not only enhance its versatility and provide solutions for high-dimensional PDEs but will also provide groundbreaking solutions to real-world problems involving complex physical processes.