  Recently, numerous new benchmarks have been established to evaluate the
performance of large language models (LLMs) via either computing a holistic
score or employing another LLM as a judge. However, these approaches suffer
from data leakage due to the open access of the benchmark and inflexible
evaluation process. To address this issue, we introduce $\textbf{TreeEval}$, a
benchmark-free evaluation method for LLMs that let a high-performance LLM host
an irreproducible evaluation session and essentially avoids the data leakage.
Moreover, this LLM performs as an examiner to raise up a series of questions
under a topic with a tree planing strategy, which considers the current
evaluation status to decide the next question generation and ensures the
completeness and efficiency of the evaluation process. We evaluate $6$ models
of different parameter sizes, including $7$B, $13$B, and $33$B, and ultimately
achieved the highest correlation coefficient with AlpacaEval2.0 using only
around $45$ questions. We also conduct more analysis to show the robustness and
reliability of TreeEval. Our code can be accessed via the provided
https://github.com/Ashura5/TreeEval.
