  The number of linear regions is one of the distinct properties of the neural
networks using piecewise linear activation functions such as ReLU, comparing
with those conventional ones using other activation functions. Previous studies
showed this property reflected the expressivity of a neural network family
([14]); as a result, it can be used to characterize how the structural
complexity of a neural network model affects the function it aims to compute.
Nonetheless, it is challenging to directly compute the number of linear
regions; therefore, many researchers focus on estimating the bounds (in
particular the upper bound) of the number of linear regions for deep neural
networks using ReLU. These methods, however, attempted to estimate the upper
bound in the entire input space. The theoretical methods are still lacking to
estimate the number of linear regions within a specific area of the input
space, e.g., a sphere centered at a training data point such as an adversarial
example or a backdoor trigger. In this paper, we present the first method to
estimate the upper bound of the number of linear regions in any sphere in the
input space of a given ReLU neural network. We implemented the method, and
computed the bounds in deep neural networks using the piece-wise linear active
function. Our experiments showed that, while training a neural network, the
boundaries of the linear regions tend to move away from the training data
points. In addition, we observe that the spheres centered at the training data
points tend to contain more linear regions than any arbitrary points in the
input space. To the best of our knowledge, this is the first study of bounding
linear regions around a specific data point. We consider our work as a first
step toward the investigation of the structural complexity of deep neural
networks in a specific input area.
