  We propose a diagrammatic notation for matrix differentiation. Our new
notation enables us to derive formulas for matrix differentiation more easily
than the usual matrix (or index) notation. We demonstrate the effectiveness of
our notation through several examples.
