  Blockchain smart contracts have emerged as a transformative force in the
digital realm, spawning a diverse range of compelling applications. Since
solidity smart contracts across various domains manage trillions of dollars in
virtual coins, they become a prime target for attacks. One of the primary
challenges is keeping abreast of the latest techniques and tools for developing
secure smart contracts and examining those already deployed. In this paper, we
seek to address these challenges from four aspects: (1) We begin by examining
ten automatic tools, specifically focusing on their methodologies and their
ability to identify vulnerabilities in solidity smart contracts. (2) We propose
a novel criterion for evaluating these tools, based on the ISO/IEC 25010
standard. (3) To facilitate the evaluation of the selected tools, we construct
a benchmark that encompasses two distinct datasets: a collection of 389
labelled smart contracts and a scaled set of 20,000 unique cases from
real-world contracts. (4) We provide a comparison of the selected tools,
offering insights into their strengths and weaknesses and highlighting areas
where further improvements are needed. Through this evaluation, we hope to
provide developers and researchers with valuable guidance on selecting and
using smart contract analysis tools and contribute to the ongoing efforts to
improve the security and reliability of smart contracts.
