  The inverse problem methodology is a commonly-used framework in the sciences
for parameter estimation and inference. It is typically performed by fitting a
mathematical model to noisy experimental data. There are two significant
sources of error in the process: 1.\ Noise from the measurement and collection
of experimental data and 2.\ numerical error in approximating the true solution
to the mathematical model. Little attention has been paid to how this second
source of error alters the results of an inverse problem. As a first step
towards a better understanding of this problem, we present a modeling and
simulation study using a simple advection-driven PDE model. We present both
analytical and computational results concerning how the different sources of
error impact the least squares cost function as well as parameter estimation
and uncertainty quantification. We investigate residual patterns to derive an
autocorrelative statistical model that can improve parameter estimation and
confidence interval computation for first order methods. Building on the
results of our investigation, we provide guidelines for practitioners to
determine when numerical or experimental error is the main source of error in
their inference, along with suggestions of how to efficiently improve their
results.
