  Deep learning is typically performed by learning a neural network solely from
data in the form of input-output pairs ignoring available domain knowledge. In
this work, the Constraint Guided Gradient Descent (CGGD) framework is proposed
that enables the injection of domain knowledge into the training procedure. The
domain knowledge is assumed to be described as a conjunction of hard inequality
constraints which appears to be a natural choice for several applications.
Compared to other neuro-symbolic approaches, the proposed method converges to a
model that satisfies any inequality constraint on the training data and does
not require to first transform the constraints into some ad-hoc term that is
added to the learning (optimisation) objective. Under certain conditions, it is
shown that CGGD can converges to a model that satisfies the constraints on the
training set, while prior work does not necessarily converge to such a model.
It is empirically shown on two independent and small data sets that CGGD makes
training less dependent on the initialisation of the network and improves the
constraint satisfiability on all data.
