  This paper proposes an actor-critic algorithm for controlling the temperature
of a battery pack using a cooling fluid. This is modeled by a coupled 1D
partial differential equation (PDE) with a controlled advection term that
determines the speed of the cooling fluid. The Hamilton-Jacobi-Bellman (HJB)
equation is a PDE that evaluates the optimality of the value function and
determines an optimal controller. We propose an algorithm that treats the value
network as a Physics-Informed Neural Network (PINN) to solve for the
continuous-time HJB equation rather than a discrete-time Bellman optimality
equation, and we derive an optimal controller for the environment that we
exploit to achieve optimal control. Our experiments show that a hybrid-policy
method that updates the value network using the HJB equation and updates the
policy network identically to PPO achieves the best results in the control of
this PDE system.
