  Symmetry, a central concept in understanding the laws of nature, has been
used for centuries in physics, mathematics, and chemistry, to help make
mathematical models tractable. Yet, despite its power, symmetry has not been
used extensively in machine learning, until rather recently. In this article we
show a general way to incorporate symmetries into machine learning models. We
demonstrate this with a detailed analysis on a rather simple real world machine
learning system - a neural network for classifying handwritten digits, lacking
bias terms for every neuron. We demonstrate that ignoring symmetries can have
dire over-fitting consequences, and that incorporating symmetry into the model
reduces over-fitting, while at the same time reducing complexity, ultimately
requiring less training data, and taking less time and resources to train.
