  This paper presents the double-activation neural network (DANN), a novel
network architecture designed for solving parabolic equations with time delay.
In DANN, each neuron is equipped with two activation functions to augment the
network's nonlinear expressive capacity. Additionally, a new parameter is
introduced for the construction of the quadratic terms in one of two activation
functions, which further enhances the network's ability to capture complex
nonlinear relationships. To address the issue of low fitting accuracy caused by
the discontinuity of solution's derivative, a piecewise fitting approach is
proposed by dividing the global solving domain into several subdomains. The
convergence of the loss function is proven. Numerical results are presented to
demonstrate the superior accuracy and faster convergence of DANN compared to
the traditional physics-informed neural network (PINN).
