  In this paper, the extension of the framework of Learning from Constraints
(LfC) to a distributed setting where multiple parties, connected over the
network, contribute to the learning process is studied. LfC relies on the
generic notion of "constraint" to inject knowledge into the learning problem
and, due to its generality, it deals with possibly nonconvex constraints,
enforced either in a hard or soft way. Motivated by recent progresses in the
field of distributed and constrained nonconvex optimization, we apply the
(distributed) Asynchronous Method of Multipliers (ASYMM) to LfC. The study
shows that such a method allows us to support scenarios where selected
constraints (i.e., knowledge), data, and outcomes of the learning process can
be locally stored in each computational node without being shared with the rest
of the network, opening the road to further investigations into
privacy-preserving LfC. Constraints act as a bridge between what is shared over
the net and what is private to each node and no central authority is required.
We demonstrate the applicability of these ideas in two distributed real-world
settings in the context of digit recognition and document classification.
