  We find that simple neural networks with ReLU activation generate polytopes
as an approximation of a unit sphere in various dimensions. The species of
polytopes are regulated by the network architecture, such as the number of
units and layers. For a variety of activation functions, generalization of
polytopes is obtained, which we call neural polytopes. They are a smooth
analogue of polytopes, exhibiting geometric duality. This finding initiates
research of generative discrete geometry to approximate surfaces by machine
learning.
