  In recent years, many software engineering researchers have begun to include
artifacts alongside their research papers. Ideally, artifacts, including tools,
benchmarks, and data, support the dissemination of ideas, provide evidence for
research claims, and serve as a starting point for future research. However, in
practice, artifacts suffer from a variety of issues that prevent the
realization of their full potential.
  To help the software engineering community realize the full potential of
artifacts, we seek to understand the challenges involved in the creation,
sharing, and use of artifacts. To that end, we perform a mixed-methods study
including a survey of artifacts in software engineering publications, and an
online survey of 153 software engineering researchers. By analyzing the
perspectives of artifact creators, users, and reviewers, we identify several
high-level challenges that affect the quality of artifacts including mismatched
expectations between these groups, and a lack of sufficient reward for both
creators and reviewers. Using Diffusion of Innovations as an analytical
framework, we examine how these challenges relate to one another, and build an
understanding of the factors that affect the sharing and success of artifacts.
Finally, we make recommendations to improve the quality of artifacts based on
our results and existing best practices.
