Contributed by the Computers and Information Division of ASME for publication in the JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING. Manuscript received August 16, 2016; final manuscript received May 2, 2017; published online June 15, 2017. Editor: Bahram Ravani.
Existing techniques for motion imitation often suffer a certain level of latency due to their computational overhead or a large set of correspondence samples to search. To achieve real-time imitation with small latency, we present a framework in this paper to reconstruct motion on humanoids based on sparsely sampled correspondence. The imitation problem is formulated as finding the projection of a point from the configuration space of a human's poses into the configuration space of a humanoid. An optimal projection is defined as the one that minimizes a back-projected deviation among a group of candidates, which can be determined in a very efficient way. Benefited from this formulation, effective projections can be obtained by using sparsely sampled correspondence, whose generation scheme is also introduced in this paper. Our method is evaluated by applying the human's motion captured by an RGB-depth (RGB-D) sensor to a humanoid in real time. Continuous motion can be realized and used in the example application of teleoperation.
Suleiman, W. , Yoshida, E. , Kanehiro, F. , Laumond, J.-P. , and Monin, A. , 2008, " On Human Motion Imitation by Humanoid Robot," IEEE International Conference on Robotics and Automation (ICRA), Pasadena, CA, May 19–23, pp. 2697–2704.
Chalodhorn, R. , Grimes, D. B. , Grochow, K. , and Rao, R. P. , 2007, " Learning to Walk Through Imitation," 20th International Joint Conference on Artifical Intelligence (IJCAI), Hyderabad, India, Jan. 6–12, Vol. 7, pp. 2084–2090.
Kim, S. , Kim, C. , You, B. , and Oh, S. , 2009, " Stable Whole-Body Motion Generation for Humanoid Robots to Imitate Human Motions," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), St. Louis, MO, Oct. 10–15, pp. 2518–2524.
Safonova, A. , Pollard, N. S. , and Hodgins, J. K. , 2003, " Optimizing Human Motion for the Control of a Humanoid Robot," 2nd International Symposium on Adaptive Motion of Animals and machines (AMAM), Kyoto, Japan, Mar. 4–8.
Ott, C. , Lee, D. , and Nakamura, Y. , 2008, " Motion Capture Based Human Motion Recognition and Imitation by Direct Marker Control," Eighth IEEE-RAS International Conference on Humanoid Robots, Daejeon, South Korea, Dec. 1–3, pp. 399–405.
Do, M. , Azad, P. , Asfour, T. , and Dillmann, R. , 2008, " Imitation of Human Motion on a Humanoid Robot Using Non-Linear Optimization," Eighth IEEE-RAS International Conference on Humanoid Robots, Daejeon, South Korea, Dec. 1–3, pp. 545–552.
Yamane, K. , Anderson, S. O. , and Hodgins, J. K. , 2010, " Controlling Humanoid Robots With Human Motion Data: Experimental Validation," Tenth IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, Dec. 6–8, pp. 504–510.
Koenemann, J. , and Bennewitz, M. , 2012, " Whole-Body Imitation of Human Motions With a Nao Humanoid," Seventh ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boston, MA, Mar. 5–8, pp. 425–425.
Koenemann, J. , Burget, F. , and Bennewitz, M. , 2014, " Real-Time Imitation of Human Whole-Body Motions by Humanoids," IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 31–June 7, pp. 2806–2812.
Aleotti, J. , Skoglund, A. , and Duckett, T. , 2004, " Position Teaching of a Robot Arm by Demonstration With a Wearable Input Device," International Conference on Intelligent Manipulation and Grasping (IMG), Genova, Italy, July 1–2.
Neto, P. , Pires, J. N. , and Moreira, A. P. , 2009, " Accelerometer-Based Control of an Industrial Robotic Arm," 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Toyama, Japan, Sept. 27–Oct. 2, pp. 1192–1197.
Stanton, C. , Bogdanovych, A. , and Ratanasena, E. , 2012, " Teleoperation of a Humanoid Robot Using Full-Body Motion Capture, Example Movements, and Machine Learning," Australasian Conference on Robotics and Automation (ACRA), Wellington, New Zealand, Dec. 3–5, pp. 260–269.
Van der Smagt, P. , and Schulten, K. , 1993, " Control of Pneumatic Robot Arm Dynamics by a Neural Network," World Congress on Neural Networks, Portland, OR, July 11–15, Vol. 3, pp. 180–183.
Jung, S. , and Hsia, T. , 1996, " Neural Network Reference Compensation Technique for Position Control of Robot Manipulators," IEEE International Conference on Neural Networks (ICNN), Washington, DC, June 3–6, Vol. 3, pp. 1765–1770.
Larsen, J. C. , and Ferrier, N. J. , 2004, " A Case Study in Vision Based Neural Network Training for Control of a Planar, Large Deflection, Flexible Robot Manipulator," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan, Sept. 28–Oct. 2, Vol. 3, pp. 2924–2929.
Wang, D. , and Bai, Y. , 2005, " Improving Position Accuracy of Robot Manipulators Using Neural Networks," IEEE Instrumentation and Measurement Technology Conference (IMTC), Ottawa, ON, Canada, May 16–19, Vol. 2, pp. 1524–1526.
Huang, G.-B. , Zhou, H. , Ding, X. , and Zhang, R. , 2012, " Extreme Learning Machine for Regression and Multiclass Classification," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., 26(2), pp. 513–529.
Deng, W. , Zheng, Q. , and Chen, L. , 2009, " Regularized Extreme Learning Machine," IEEE Symposium on Computational Intelligence and Data Mining (CIDM), Nashville, TN, Mar. 30–Apr. 2, pp. 389–395.
LaViola, J. J. , 2003, " Double Exponential Smoothing: An Alternative to Kalman Filter-Based Predictive Tracking," ACM Workshop on Virtual Environments, Zurich, Switzerland, May 22–23, pp. 199–206.
Lapeyre, M., Rouanet, P., Grizou, J., Nguyen, S., Depraetre, F., Le Falher, A., and Oudeyer, P.-Y., 2014, " Poppy Project: Open-Source Fabrication of 3D Printed Humanoid Robot for Science, Education and Art," Digital Intelligence (DI2014), Nantes, France, Sept. 17–19.
Zheng, Y. , Chan, K. C. , and Wang, C. C. L. , 2014, " Pedalvatar: An IMU-Based Real-Time Body Motion Capture System Using Foot Rooted Kinematic Model," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, Sept. 14–18, pp. 4130–4135.
Eight basic poses are reconstructed by our method (left of each pair) and compared with the ground truth (right of each pair). The similarity metrics, Mmax and Mavg, of each pair are also reported. The evaluation is taken on a projection defined by using 1644 landmark pairs.
Statistics in eight motions for the change of two metrics in degree: Mmax (the upper curves in each sub-figures) and Mavg (the lower curves in sub-figures). The evaluation is also taken on a projection with 1644 landmark pairs.
Jin S, Dai C, Liu Y, Wang CL. Motion Imitation Based on Sparsely Sampled Correspondence. ASME. J. Comput. Inf. Sci. Eng. 2017;17(4):041009-041009-7. doi:10.1115/1.4036923.