  Adaptive filters (AFs) are vital for enhancing the performance of downstream
tasks, such as speech recognition, sound event detection, and keyword spotting.
However, traditional AF design prioritizes isolated signal-level objectives,
often overlooking downstream task performance. This can lead to suboptimal
performance. Recent research has leveraged meta-learning to automatically learn
AF update rules from data, alleviating the need for manual tuning when using
simple signal-level objectives. This paper improves the Meta-AF framework by
expanding it to support end-to-end training for arbitrary downstream tasks. We
focus on classification tasks, where we introduce a novel training methodology
that harnesses self-supervision and classifier feedback. We evaluate our
approach on the combined task of acoustic echo cancellation and keyword
spotting. Our findings demonstrate consistent performance improvements with
both pre-trained and joint-trained keyword spotting models across synthetic and
real playback. Notably, these improvements come without requiring additional
tuning, increased inference-time complexity, or reliance on oracle signal-level
training data.
