  Often in language and other areas of cognition, whether two components of an
object are identical or not determines if it is well formed. We call such
constraints identity effects. When developing a system to learn well-formedness
from examples, it is easy enough to build in an identify effect. But can
identity effects be learned from the data without explicit guidance? We provide
a framework in which we can rigorously prove that algorithms satisfying simple
criteria cannot make the correct inference. We then show that a broad class of
learning algorithms including deep feedforward neural networks trained via
gradient-based algorithms (such as stochastic gradient descent or the Adam
method) satisfy our criteria, dependent on the encoding of inputs. In some
broader circumstances we are able to provide adversarial examples that the
network necessarily classifies incorrectly. Finally, we demonstrate our theory
with computational experiments in which we explore the effect of different
input encodings on the ability of algorithms to generalize to novel inputs.
This allows us to show similar effects to those predicted by theory for more
realistic methods that violate some of the conditions of our theoretical
results.
