  Rapid advances in computation, combined with latest advances in computer
graphics simulations have facilitated the development of vision systems and
training them in virtual environments. One major stumbling block is in
certification of the designs and tuned parameters of these systems to work in
real world. In this paper, we begin to explore the fundamental question: Which
type of information transfer is more analogous to real world? Inspired from the
performance characterization methodology outlined in the 90's, we note that
insights derived from simulations can be qualitative or quantitative depending
on the degree of the fidelity of models used in simulations and the nature of
the questions posed by the experimenter. We adapt the methodology in the
context of current graphics simulation tools for modeling data generation
processes and, for systematic performance characterization and trade-off
analysis for vision system design leading to qualitative and quantitative
insights. In concrete, we examine invariance assumptions used in vision
algorithms for video surveillance settings as a case study and assess the
degree to which those invariance assumptions deviate as a function of
contextual variables on both graphics simulations and in real data. As computer
graphics rendering quality improves, we believe teasing apart the degree to
which model assumptions are valid via systematic graphics simulation can be a
significant aid to assisting more principled ways of approaching vision system
design and performance modeling.
