  We propose a new generalization of neural network parameter spaces with
noncommutative $C^*$-algebra, which possesses a rich noncommutative structure
of products. We show that this noncommutative structure induces powerful
effects in learning neural networks. Our framework has a wide range of
applications, such as learning multiple related neural networks simultaneously
with interactions and learning equivariant features with respect to group
actions. Numerical experiments illustrate the validity of our framework and its
potential power.
