  Large Language Models (LLMs) have shown exceptional performance in text
processing. Notably, LLMs can synthesize information from large datasets and
explain their decisions similarly to human reasoning through a chain of thought
(CoT). An emerging application of LLMs is the handling and interpreting of
numerical data, where fine-tuning enhances their performance over basic
inference methods. This paper proposes a novel approach to training LLMs using
knowledge transfer from a random forest (RF) ensemble, leveraging its
efficiency and accuracy. By converting RF decision paths into natural language
statements, we generate outputs for LLM fine-tuning, enhancing the model's
ability to classify and explain its decisions. Our method includes verifying
these rules through established classification metrics, ensuring their
correctness. We also examine the impact of preprocessing techniques on the
representation of numerical data and their influence on classification accuracy
and rule correctness
