  We show that for neural network functions that have width less or equal to
the input dimension all connected components of decision regions are unbounded.
The result holds for continuous and strictly monotonic activation functions as
well as for the ReLU activation function. This complements recent results on
approximation capabilities by [Hanin 2017 Approximating] and connectivity of
decision regions by [Nguyen 2018 Neural] for such narrow neural networks. Our
results are illustrated by means of numerical experiments.
