  We introduce a new training algorithm for variety of deep neural networks
that utilize random complex exponential activation functions. Our approach
employs a Markov Chain Monte Carlo sampling procedure to iteratively train
network layers, avoiding global and gradient-based optimization while
maintaining error control. It consistently attains the theoretical
approximation rate for residual networks with complex exponential activation
functions, determined by network complexity. Additionally, it enables efficient
learning of multiscale and high-frequency features, producing interpretable
parameter distributions. Despite using sinusoidal basis functions, we do not
observe Gibbs phenomena in approximating discontinuous target functions.
