  Real-time crowd-powered systems, such as Chorus/Evorus, VizWiz, and
Apparition, have shown how incorporating humans into automated systems could
supplement where the automatic solutions fall short. However, one unspoken
bottleneck of applying such architectures to more scenarios is the longer
latency of including humans in the loop of automated systems. For the
applications that have hard constraints in turnaround times, human-operated
components' longer latency and large speed variation seem to be apparent deal
breakers. This paper explicates and quantifies these limitations by using a
human-powered text-based backend to hold conversations with users through a
voice-only smart speaker. Smart speakers must respond to users' requests within
seconds, so the workers behind the scenes only have a few seconds to compose
answers. We measured the end-to-end system latency and the conversation quality
with eight pairs of participants, showing the challenges and superiority of
such systems.
