  We study the approximation properties of convolutional architectures applied
to time series modelling, which can be formulated mathematically as a
functional approximation problem. In the recurrent setting, recent results
reveal an intricate connection between approximation efficiency and memory
structures in the data generation process. In this paper, we derive parallel
results for convolutional architectures, with WaveNet being a prime example.
Our results reveal that in this new setting, approximation efficiency is not
only characterised by memory, but also additional fine structures in the target
relationship. This leads to a novel definition of spectrum-based regularity
that measures the complexity of temporal relationships under the convolutional
approximation scheme. These analyses provide a foundation to understand the
differences between architectural choices for time series modelling and can
give theoretically grounded guidance for practical applications.
