  Smart contracts are decentralized applications built atop blockchains like
Ethereum. Recent research has shown that large language models (LLMs) have
potential in auditing smart contracts, but the state-of-the-art indicates that
even GPT-4 can achieve only 30% precision (when both decision and justification
are correct). This is likely because off-the-shelf LLMs were primarily
pre-trained on a general text/code corpus and not fine-tuned on the specific
domain of Solidity smart contract auditing.
  In this paper, we propose TrustLLM, a general framework that combines
fine-tuning and LLM-based agents for intuitive smart contract auditing with
justifications. Specifically, TrustLLM is inspired by the observation that
expert human auditors first perceive what could be wrong and then perform a
detailed analysis of the code to identify the cause. As such, TrustLLM employs
a two-stage fine-tuning approach: it first tunes a Detector model to make
decisions and then tunes a Reasoner model to generate causes of
vulnerabilities. However, fine-tuning alone faces challenges in accurately
identifying the optimal cause of a vulnerability. Therefore, we introduce two
LLM-based agents, the Ranker and Critic, to iteratively select and debate the
most suitable cause of vulnerability based on the output of the fine-tuned
Reasoner model. To evaluate TrustLLM, we collected a balanced dataset with
1,734 positive and 1,810 negative samples to fine-tune TrustLLM. We then
compared it with traditional fine-tuned models (CodeBERT, GraphCodeBERT,
CodeT5, and UnixCoder) as well as prompt learning-based LLMs (GPT4, GPT-3.5,
and CodeLlama-13b/34b). On a dataset of 263 real smart contract
vulnerabilities, TrustLLM achieves an F1 score of 91.21% and an accuracy of
91.11%. The causes generated by TrustLLM achieved a consistency of about 38%
compared to the ground truth causes.
