  Textual information of data is of vital importance for data mining and
feature engineering. However, existing methods focus on learning the data
structures and overlook the textual information along with the data.
Consequently, they waste this valuable resource and miss out on the deeper data
relationships embedded within the texts. In this paper, we introduce
Text-Informed Feature Generation (TIFG), a novel LLM-based text-informed
feature generation framework. TIFG utilizes the textual information to generate
features by retrieving possible relevant features within external knowledge
with Retrieval Augmented Generation (RAG) technology. In this approach, the
TIFG can generate new explainable features to enrich the feature space and
further mine feature relationships. We design the TIFG to be an automated
framework that continuously optimizes the feature generation process, adapts to
new data inputs, and improves downstream task performance over iterations. A
broad range of experiments in various downstream tasks showcases that our
approach can generate high-quality and meaningful features, and is
significantly superior to existing methods.
