Researchers have developed the Hypotheses-to-Theories (HtT) framework, which aims to improve reasoning with large language models (LLMs) by learning a rule library. The HtT framework consists of two stages: induction and deduction. In the induction stage, an LLM is trained to generate and verify rules based on a set of training examples. Rules that appear frequently and lead to correct answers are collected to form a rule library. In the deduction stage, the LLM utilizes the rule library to reason and answer test questions. Experimental results on numerical and relational reasoning problems demonstrate that HtT outperforms existing prompting methods, achieving a significant increase in accuracy ranging from 11% to 27%. Moreover, the learned rules are found to be transferable to different models and different variations of the same problem, showcasing the versatility of the HtT framework.
[2310.07064] Large Language Models can Learn Rules
Read the complete article at: arxiv.org