  Human-robot collaboration has benefited users with higher efficiency towards
interactive tasks. Nevertheless, most collaborative schemes rely on complicated
human-machine interfaces, which might lack the requisite intuitiveness compared
with natural limb control. We also expect to understand human intent with low
training data requirements. In response to these challenges, this paper
introduces an innovative human-robot collaborative framework that seamlessly
integrates hand gesture and dynamic movement recognition, voice recognition,
and a switchable control adaptation strategy. These modules provide a
user-friendly approach that enables the robot to deliver the tools as per user
need, especially when the user is working with both hands. Therefore, users can
focus on their task execution without additional training in the use of
human-machine interfaces, while the robot interprets their intuitive gestures.
The proposed multimodal interaction framework is executed in the UR5e robot
platform equipped with a RealSense D435i camera, and the effectiveness is
assessed through a soldering circuit board task. The experiment results have
demonstrated superior performance in hand gesture recognition, where the static
hand gesture recognition module achieves an accuracy of 94.3\%, while the
dynamic motion recognition module reaches 97.6\% accuracy. Compared with human
solo manipulation, the proposed approach facilitates higher efficiency tool
delivery, without significantly distracting from human intents.
