Physics-informed neural networks (PINNs) are known to suffer from optimization difficulty. In this work, we reveal the connection between the optimization difficulty of PINNs and activation functions. Specifically, we show that PINNs exhibit high sensitivity to activation functions when solving PDEs with distinct properties. Existing works usually choose activation functions by inefficient trial-and-error. To avoid the inefficient manual selection and to alleviate the optimization difficulty of PINNs, we introduce adaptive activation functions to search for the optimal function when solving different problems. We compare different adaptive activation functions and discuss their limitations in the context of PINNs. Furthermore, we propose to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization, which has a higher requirement for the smoothness and diversity on learned functions. This is achieved by removing activation functions which cannot provide higher-order derivatives from the candidate set and incorporating elementary functions with different properties according to our prior knowledge about the PDE at hand. We further enhance the search space with adaptive slopes. The proposed adaptive activation function can be used to solve different PDE systems in an interpretable way. Its effectiveness is demonstrated on a series of benchmarks. Code is available at https://github.com/LeapLabTHU/AdaAFforPINNs.

In our prior work [10], neural networks with local converging inputs (NNLCI) were introduced for solving one-dimensional conservation equations. Two solutions of a conservation law in a converging sequence, computed from low-cost numerical schemes, and in a local domain of dependence of the space-time location, were used as the input to a neural network in order to predict a high-fidelity solution at a given space-time location. In the present work, we extend the method to two-dimensional conservation systems and introduce different solution techniques. Numerical results demonstrate the validity and effectiveness of the NNLCI method for application to multi-dimensional problems. In spite of low-cost smeared input data, the NNLCI method is capable of accurately predicting shocks, contact discontinuities, and the smooth region of the entire field. The NNLCI method is relatively easy to train because of the use of local solvers. The computing time saving is between one and two orders of magnitude compared with the corresponding high-fidelity schemes for two-dimensional Riemann problems. The relative efficiency of the NNLCI method is expected to be substantially greater for problems with higher spatial dimensions or smooth solutions.

We propose a mesh-free method to solve the full Stokes equation for modeling the glacier dynamics with nonlinear rheology. Inspired by the Deep-Ritz method proposed in [13], we first formulate the solution to the non-Newtonian Stokes equation as the minimizer of a variational problem with boundary constraints. Then, we approximate its solution space by a deep neural network. The loss function for training the neural network is a relaxed version of the variational form, in which penalty terms are used to present soft constraints due to mixed boundary conditions. Instead of introducing mesh grids or basis functions to evaluate the loss function, our method only requires uniform sampling from the physical domain and boundaries. Furthermore, we introduce a re-normalization technique in the neural network to address the significant variation in the scaling of real-world problems. Finally, we illustrate the performance of our method by several numerical experiments, including a 2D model with the analytical solution, the Arolla glacier model with realistic scaling and a 3D model with periodic boundary conditions. Numerical results show that our proposed method is efficient in solving the non-Newtonian mechanics arising from glacier modeling with nonlinear rheology.

This paper develops the genuinely multidimensional HLL Riemann solver for the two-dimensional special relativistic hydrodynamic equations on Cartesian meshes and studies its physical-constraint-preserving (PCP) property. Based on the resulting HLL solver, the first- and high-order accurate PCP finite volume schemes are proposed. In the high-order scheme, the WENO reconstruction, the third-order accurate strong-stability-preserving time discretizations and the PCP flux limiter are used. Several numerical results are given to demonstrate the accuracy, performance and resolution of the shock waves and the genuinely multi-dimensional wave structures etc. of our PCP finite volume schemes.

We present an extension of the flux globalization based well-balanced path-conservative central-upwind scheme to the one- and two-dimensional thermal rotating shallow water equations. The scheme is well-balanced in the sense that it can exactly preserve a variety of physically relevant steady states. In the one-dimensional case, it can preserve different “lake-at-rest” equilibria, thermo-geostrophic equilibria, as well as general moving-water steady states. In the two-dimensional case, preserving general moving-water steady states is difficult, and to the best of our knowledge, none of existing schemes can achieve this ultimate goal. The proposed scheme can exactly preserve the $x$- and $y$-directional jets in the rotational frame as well as certain genuinely two-dimensional equilibria. Furthermore, our approach employs a path-conservative technique for discretizing nonconservative product terms, which are incorporated into the global fluxes. This allows the developed scheme to exactly preserve some of the discontinuous steady states as well. We provide a number of numerical examples to demonstrate the advantages of the proposed scheme over some alternative finite-volume methods.

While the recently proposed TENO (targeted essentially non-oscillatory) schemes [Fu et al., Journal of Computational Physics 305 (2016): 333-359] exhibit better performance than the classical WENO (weighted essentially non-oscillatory) schemes with the same accuracy order, there is still a room for further improvement, e.g., the physical discontinuities may be significantly smeared by the excessive numerical dissipation due to the enforcement of the ENO property after a long-time advection. More recently, a new fifth-order TENO5-THINC scheme is proposed by coupling the TENO5 scheme with a non-polynomial THINC (tangent of hyperbola for interface capturing) scheme based on a parameter-free discontinuity indicator. The novelty originates from the fact that the new strategy locates the discontinuities accurately and deploys the jump-like THINC reconstruction scheme for resolving the discontinuities with a sub-cell resolution, instead of enforcing the ENO property. The new scheme successfully leverages the excellent wave-resolution property of standard TENO schemes for smooth and under-resolved continuous scales and the discontinuity-resolving capability of THINC for reconstructing genuine discontinuities. In this work, we further develop the low-dissipation discontinuity-resolving very-high-order TENO-THINC reconstruction schemes for hyperbolic conservation laws by proposing tailored coupling strategies. Without loss of generality, the six- and eight-point TENO-THINC schemes are developed, and the explicit formulas are given as well as the built-in parameters. Based on a set of critical benchmark simulations, the newly proposed schemes show significantly lower numerical dissipation when compared to the counterpart TENO schemes without sacrificing numerical robustness. The presented numerical results represent the state-of-the-art in the literature and can serve as references for future algorithm development.

Particle-resolved direct numerical flow solvers predominantly use a projection method to decouple the non-linear mass and momentum conservation equations. The computing performance of such solvers often decays beyond $\mathcal{O}(1000)$ cores due to the cost of solving at least one large three-dimensional pressure Poisson problem per time step. The parallelization may perform moderately well only or even poorly sometimes despite using an efficient algebraic multigrid preconditioner [38]. We present an accurate and scalable solver using a direction splitting algorithm [12] to transform all three-dimensional parabolic/elliptic problems (and in particular the elliptic pressure Poisson problem) into a sequence of three one-dimensional parabolic sub-problems, thus improving its scalability up to multiple thousands of cores. We employ this algorithm to solve mass and momentum conservation equations in flows laden with fixed non-spherical rigid bodies. We consider the presence of rigid bodies on the (uniform or non-uniform) fixed Cartesian fluid grid by modifying the diffusion and divergence stencils on the impacted grid node near the rigid body boundary. Compared to [12], we use a higher-order interpolation scheme for the velocity field to maintain a second-order stress estimation on the particle boundary, resulting in more accurate dimensionless coefficients such as drag $C_d$ and lift $C_l.$ We also correct the interpolation scheme due to the presence of any nearby particle to maintain an acceptable accuracy, making the solver robust even when particles are densely packed in a sub-region of the computational domain. We present classical validation tests involving a single or multiple (up to $\mathcal{O}(1000))$ rigid bodies and assess the robustness, accuracy and computing speed of the solver. We further show that the Direction Splitting solver is ∼5 times faster on 5120 cores than our solver [38] based on a classical projection method [5].

Fractional partial differential equations (FPDEs) can effectively represent anomalous transport and nonlocal interactions. However, inherent uncertainties arise naturally in real applications due to random forcing or unknown material properties. Mathematical models considering nonlocal interactions with uncertainty quantification can be formulated as stochastic fractional partial differential equations (SFPDEs). There are many challenges in solving SFPDEs numerically, especially for long-time integration since such problems are high-dimensional and nonlocal. Here, we combine the bi-orthogonal (BO) method for representing stochastic processes with physics-informed neural networks (PINNs) for solving partial differential equations to formulate the bi-orthogonal PINN method (BO-fPINN) for solving time-dependent SFPDEs. Specifically, we introduce a deep neural network for the stochastic solution of the time-dependent SFPDEs, and include the BO constraints in the loss function following a weak formulation. Since automatic differentiation is not currently applicable to fractional derivatives, we employ discretization on a grid to compute the fractional derivatives of the neural network output. The weak formulation loss function of the BO-fPINN method can overcome some drawbacks of the BO methods and thus can be used to solve SFPDEs with eigenvalue crossings. Moreover, the BO-fPINN method can be used for inverse SFPDEs with the same framework and same computational complexity as for forward problems. We demonstrate the effectiveness of the BO-fPINN method for different benchmark problems. Specifically, we first consider an SFPDE with eigenvalue crossing and obtain good results while the original BO method fails. We then solve several forward and inverse problems governed by SFPDEs, including problems with noisy initial conditions. We study the effect of the fractional order as well as the number of the BO modes on the accuracy of the BO-fPINN method. The results demonstrate the flexibility and efficiency of the proposed method, especially for inverse problems. We also present a simple example of transfer learning (for the fractional order) that can help in accelerating the training of BO-fPINN for SFPDEs. Taken together, the simulation results show that the BO-fPINN method can be employed to effectively solve time-dependent SFPDEs and may provide a reliable computational strategy for real applications exhibiting anomalous transport.