  Anderson acceleration (AA) is a well-known method for accelerating the
convergence of iterative algorithms, with applications in various fields
including deep learning and optimization. Despite its popularity in these
areas, the effectiveness of AA in classical machine learning classifiers has
not been thoroughly studied. Tabular data, in particular, presents a unique
challenge for deep learning models, and classical machine learning models are
known to perform better in these scenarios. However, the convergence analysis
of these models has received limited attention. To address this gap in
research, we implement a support vector machine (SVM) classifier variant that
incorporates AA to speed up convergence. We evaluate the performance of our SVM
with and without Anderson acceleration on several datasets from the biology
domain and demonstrate that the use of AA significantly improves convergence
and reduces the training loss as the number of iterations increases. Our
findings provide a promising perspective on the potential of Anderson
acceleration in the training of simple machine learning classifiers and
underscore the importance of further research in this area. By showing the
effectiveness of AA in this setting, we aim to inspire more studies that
explore the applications of AA in classical machine learning.
