  Smart contracts are programs deployed on a blockchain and are immutable once
deployed. Reentrancy, one of the most important vulnerabilities in smart
contracts, has caused millions of dollars in financial loss. Many reentrancy
detection approaches have been proposed. It is necessary to investigate the
performance of these approaches to provide useful guidelines for their
application. In this work, we conduct a large-scale empirical study on the
capability of five well-known or recent reentrancy detection tools such as
Mythril and Sailfish. We collect 230,548 verified smart contracts from
Etherscan and use detection tools to analyze 139,424 contracts after
deduplication, which results in 21,212 contracts with reentrancy issues. Then,
we manually examine the defective functions located by the tools in the
contracts. From the examination results, we obtain 34 true positive contracts
with reentrancy and 21,178 false positive contracts without reentrancy. We also
analyze the causes of the true and false positives. Finally, we evaluate the
tools based on the two kinds of contracts. The results show that more than
99.8% of the reentrant contracts detected by the tools are false positives with
eight types of causes, and the tools can only detect the reentrancy issues
caused by call.value(), 58.8% of which can be revealed by the Ethereum's
official IDE, Remix. Furthermore, we collect real-world reentrancy attacks
reported in the past two years and find that the tools fail to find any issues
in the corresponding contracts. Based on the findings, existing works on
reentrancy detection appear to have very limited capability, and researchers
should turn the rudder to discover and detect new reentrancy patterns except
those related to call.value().
