  Model discovery aims at autonomously discovering differential equations
underlying a dataset. Approaches based on Physics Informed Neural Networks
(PINNs) have shown great promise, but a fully-differentiable model which
explicitly learns the equation has remained elusive. In this paper we propose
such an approach by integrating neural network-based surrogates with Sparse
Bayesian Learning (SBL). This combination yields a robust model discovery
algorithm, which we showcase on various datasets. We then identify a connection
with multitask learning, and build on it to construct a Physics Informed
Normalizing Flow (PINF). We present a proof-of-concept using a PINF to directly
learn a density model from single particle data. Our work expands PINNs to
various types of neural network architectures, and connects neural
network-based surrogates to the rich field of Bayesian parameter inference.
