  Serverless computing is an emerging cloud computing paradigm, which allows
software engineers to develop applications at the granularity of function
(called serverless functions). However, multiple identical runs of the same
serverless functions can show different performance (i.e., response latencies)
due to the highly dynamic underlying environment where these functions are
executed. We conduct the first work to study serverless function performance to
raise awareness of this variance among researchers. We investigate 59 related
research papers published in top-tier conferences, and observe that only 40.68%
of them use multiple runs to quantify the variance of serverless function
performance. Then we extract 65 serverless functions used in these papers and
find that the performance of these serverless functions can differ by up to
338.76% (44.15% on average), indicating a large magnitude of the variance.
Furthermore, we find that 61.54% of these functions can have unreliable
performance results at the low number of repetitions that are widely adopted in
the serverless computing literature.
