  In this work, we propose an interactive general instruction framework
SketchMeHow to guidance the common users to complete the daily tasks in
real-time. In contrast to the conventional augmented reality-based instruction
systems, the proposed framework utilizes the user sketches as system inputs to
acquire the users' production intentions from the drawing interfaces. Given the
user sketches, the designated task instruction can be analyzed based on the
sub-task division and spatial localization for each task. The projector-camera
system is adopted in the projection guidance to the end-users with the spatial
augmented reality technology. To verify the proposed framework, we conducted
two case studies of domino arrangement and bento production. From our user
studies, the proposed systems can help novice users complete the tasks
efficiently with user satisfaction. We believe the proposed SketchMeHow can
broaden the research topics in sketch-based real-world applications in
human-computer interaction.
