  We introduce a novel generative formulation of deep probabilistic models
implementing "soft" constraints on their function dynamics. In particular, we
develop a flexible methodological framework where the modeled functions and
derivatives of a given order are subject to inequality or equality constraints.
We then characterize the posterior distribution over model and constraint
parameters through stochastic variational inference. As a result, the proposed
approach allows for accurate and scalable uncertainty quantification on the
predictions and on all parameters. We demonstrate the application of equality
constraints in the challenging problem of parameter inference in ordinary
differential equation models, while we showcase the application of inequality
constraints on the problem of monotonic regression of count data. The proposed
approach is extensively tested in several experimental settings, leading to
highly competitive results in challenging modeling applications, while offering
high expressiveness, flexibility and scalability.
