  Implicit Neural Representations (INRs) have gained popularity for encoding
signals as compact, differentiable entities. While commonly using techniques
like Fourier positional encodings or non-traditional activation functions
(e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content,
their properties lack exploration within a unified theoretical framework.
Addressing this gap, we conduct a comprehensive analysis of these activations
from a sampling theory perspective. Our investigation reveals that sinc
activations, previously unused in conjunction with INRs, are theoretically
optimal for signal encoding. Additionally, we establish a connection between
dynamical systems and INRs, leveraging sampling theory to bridge these two
paradigms.
