  On-line data assimilation techniques such as ensemble Kalman filters and
particle filters lose accuracy dramatically when presented with an unlikely
observation. Such an observation may be caused by an unusually large
measurement error or reflect a rare fluctuation in the dynamics of the system.
Over a long enough span of time it becomes likely that one or several of these
events will occur. Often they are signatures of the most interesting features
of the underlying system and their prediction becomes the primary focus of the
data assimilation procedure. The Kuroshio or Black Current that runs along the
eastern coast of Japan is an example of such a system. It undergoes infrequent
but dramatic changes of state between a small meander during which the current
remains close to the coast of Japan, and a large meander during which it bulges
away from the coast. Because of the important role that the Kuroshio plays in
distributing heat and salinity in the surrounding region, prediction of these
transitions is of acute interest. Here we focus on a regime in which both the
stochastic forcing on the system and the observational noise are small. In this
setting large deviation theory can be used to understand why standard filtering
methods fail and guide the design of the more effective data assimilation
techniques. Motivated by our analysis we propose several data assimilation
strategies capable of efficiently handling rare events such as the transitions
of the Kuroshio. These techniques are tested on a model of the Kuroshio and
shown to perform much better than standard filtering methods.
