  Pre-trained large language models (LLM) have emerged as a powerful tool for
simulating various scenarios and generating output given specific instructions
and multimodal input. In this work, we analyze the specific use of LLM to
enhance a classical supervised machine learning method for classification
problems. We propose a few approaches to integrate LLM into a classical machine
learning estimator to further enhance the prediction performance. We examine
the performance of the proposed approaches through both standard supervised
learning binary classification tasks, and a transfer learning task where the
test data observe distribution changes compared to the training data. Numerical
experiments using four publicly available datasets are conducted and suggest
that using LLM to enhance classical machine learning estimators can provide
significant improvement on prediction performance.
