  The nonlinear and stochastic relationship between noise covariance parameter
values and state estimator performance makes optimal filter tuning a very
challenging problem. Popular optimization-based tuning approaches can easily
get trapped in local minima, leading to poor noise parameter identification and
suboptimal state estimation. Recently, black box techniques based on Bayesian
optimization with Gaussian processes (GPBO) have been shown to overcome many of
these issues, using normalized estimation error squared (NEES) and normalized
innovation error (NIS) statistics to derive cost functions for Kalman filter
auto-tuning. While reliable noise parameter estimates are obtained in many
cases, GPBO solutions obtained with these conventional cost functions do not
always converge to optimal filter noise parameters and lack robustness to
parameter ambiguities in time-discretized system models. This paper addresses
these issues by making two main contributions. First, we show that NIS and NEES
errors are only chi-squared distributed for tuned estimators. As a result,
chi-square tests are not sufficient to ensure that an estimator has been
correctly tuned. We use this to extend the familiar consistency tests for NIS
and NEES to penalize if the distribution is not chi-squared distributed.
Second, this cost measure is applied within a Student-t processes Bayesian
Optimization (TPBO) to achieve robust estimator performance for time
discretized state space models. The robustness, accuracy, and reliability of
our approach are illustrated on classical state estimation problems.
