Satisfaction of Assumptions is a Weak Predictor of Performance
  This paper demonstrates a methodology for examining the accuracy of uncertain
inference systems (UIS), after their parameters have been optimized, and does
so for several common UIS's. This methodology may be used to test the accuracy
when either the prior assumptions or updating formulae are not exactly
satisfied. Surprisingly, these UIS's were revealed to be no more accurate on
the average than a simple linear regression. Moreover, even on prior
distributions which were deliberately biased so as give very good accuracy,
they were less accurate than the simple probabilistic model which assumes
marginal independence between inputs. This demonstrates that the importance of
updating formulae can outweigh that of prior assumptions. Thus, when UIS's are
judged by their final accuracy after optimization, we get completely different
results than when they are judged by whether or not their prior assumptions are
perfectly satisfied.
