  Using a statistical model-based data generation, we develop an experimental
setup for the evaluation of neural networks (NNs). The setup helps to benchmark
a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds.
This allows us to test the effects of training data size, data dimension, data
geometry, noise, and mismatch between training and testing conditions. In the
proposed setup, we use a Gaussian mixture distribution to generate data for
training and testing a set of competing NNs. Our experiments show the
importance of understanding the type and statistical conditions of data for
appropriate application and design of NNs
