  We study the ensemble Kalman filter (EnKF) algorithm for sequential data
assimilation in a general situation, that is, for nonlinear forecast and
measurement models with non-additive and non-Gaussian noises. Such applications
traditionally force us to choose between inaccurate Gaussian assumptions that
permit efficient algorithms (e.g., EnKF), or more accurate direct sampling
methods which scale poorly with dimension (e.g., particle filters, or PF). We
introduce a trimmed ensemble Kalman filter (TEnKF) which can interpolate
between the limiting distributions of the EnKF and PF to facilitate adaptive
control over both accuracy and efficiency. This is achieved by introducing a
trimming function that removes non-Gaussian outliers that introduce errors in
the correlation between the model and observed forecast, which otherwise
prevent the EnKF from proposing accurate forecast updates. We show for specific
trimming functions that the TEnKF exactly reproduces the limiting distributions
of the EnKF and PF. We also develop an adaptive implementation which provides
control of the effective sample size and allows the filter to overcome periods
of increased model nonlinearity. This algorithm allow us to demonstrate
substantial improvements over the traditional EnKF in convergence and
robustness for the nonlinear Lorenz-63 and Lorenz-96 models.
