  Much of software engineering research focuses on tools, algorithms, and
optimization of software. Recently, we, as a community, have come to
acknowledge that there is a gap in meta-research and addressing the
human-factors in software engineering research. Through meta research, we aim
to deepen our understanding of online participant recruitment and
human-subjects software engineering research. In this paper we motivate the
need to consider the unique challenges that human studies pose in software
engineering research. We present several challenges faced by our research team
in several distinct research studies, how they affected research, and motivate
how, as researchers, we can address these challenges. We present results from a
pilot study and categorize issues faced into three broad categories including
participant recruitment, community engagement, and data poisoning. We further
discuss how we can address these challenges and outline the benefits a
full-study could provide to the software engineering research community.
