  The data used during training in any given application space is directly tied
to the performance of the system once deployed. While there are many other
factors that go into producing high performance models within machine learning,
there is no doubt that the data used to train a system provides the foundation
from which to build. One of the underlying rule of thumb heuristics used within
the machine learning space is that more data leads to better models, but there
is no easy answer for the question, "How much data is needed?" This work
examines a modulation classification problem in the Radio Frequency domain
space, attempting to answer the question of how much training data is required
to achieve a desired level of performance, but the procedure readily applies to
classification problems across modalities. The ultimate goal is determining an
approach that requires the least amount of data collection to better inform a
more thorough collection effort to achieve the desired performance metric.
While this approach will require an initial dataset that is germane to the
problem space to act as a \textit{target} dataset on which metrics are
extracted, the goal is to allow for the initial data to be orders of magnitude
smaller than what is required for delivering a system that achieves the desired
performance. An additional benefit of the techniques presented here is that the
quality of different datasets can be numerically evaluated and tied together
with the quantity of data, and ultimately, the performance of the architecture
in the problem domain.
