  Large Language Models (LLMs) have showcased impressive performance. However,
due to their inability to capture relationships among samples, these frozen
LLMs inevitably keep repeating similar mistakes. In this work, we propose our
Tuning-free Rule Accumulation (TRAN) framework, which guides LLMs in improving
their performance by learning from previous mistakes. Considering data arrives
sequentially, LLMs gradually accumulate rules from incorrect cases, forming a
rule collection. These rules are then utilized by the LLMs to avoid making
similar mistakes when processing subsequent inputs. Moreover, the rules remain
independent of the primary prompts, seamlessly complementing prompt design
strategies. Experimentally, we show that TRAN improves over recent baselines by
a large margin.
