Abstract
This paper introduces a piano playing robot in views of smart house and assistive robot technology to care the affective states of the elderly. We address the current issues in this research area and propose a piano playing robot as a solution. For affective interaction based on music, we first present a beat gesture recognition method to synchronize the tempo of a robot playing a piano with the desired tempo of the user. To estimate the period of an unstructured beat gesture expressed by any part of a body or an object, we apply an optical flow method, and use the trajectories of the center of gravity and normalized central moments of moving objects in images. In addition, we also apply a motion control method by which robotic fingers are trained to follow a set of trajectories. Since the ability to track the trajectories influences the sound a piano generates, we adopt an iterative learning control method to reduce the tracking error.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bien ZZ, Park KH, Jung JW and Do JH (2005). Intention reading is essential in human-friendly interfaces for the elderly and the handicapped. IEEE Trans Ind Electron 52(6): 1500–1505
Bondi P, Casalino G and Gambardella L (1988). On the iterative learning control theory for robotic manipulators. IEEE Trans Rob Autom 4(1): 14–22
Cohen C, Conway L (1996) Dynamical system representation, generation, and recognition of basic oscillatory motion gestures. In: Proceedings of the 2nd international conference on automatic face and gesture recognition, pp 60–65
Cutler R and Davis L (2000). Robust real-time periodic motion detection, analysis and applications. IEEE Trans Pattern Anal Mach Intell 22(8): 781–796
Davis J, Bobick A, Richards W (2000) Categorical representation and recognition of oscillatory motion patterns. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1, pp 628–635
Fawcett J, Scheftner W, Fogg L, Clark D, Young M, Hedeker D and Gibbons R (1990). Time-related predictors of suicide in major affective disorder. Am J Psychiatry 147: 1189–1194
Filova V, Solina F and Lenarcic J (1998). Automatic reconstruction of 3d human arm motion from a monocular image sequence. Mach Vis Appl 10: 223–231
Kato I, Ohteru S, Shirai K, Narita S, Sugano S, Matsushima T, Kobayashi T and Fujisawa E (1987). The robot musician ‘wabot-2’. Robotics 3(2): 143–155
Lee HE, Kim YM, Park KH and Bien ZZ (2005). Development of a steward robot for intelligent sweet home. Int J Hum-friendly Welf Rob Syst 6(4): 57–64
Liu F, Picard R (1998) Finding periodicity in space and time. In: Proceedings of the 6th international conference on computer vision, pp 376–383
Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the DARPA image understanding workshop, pp 121–130
Luukinen H, Laippala P and Huikuri H (2003). Depressive symptoms and the risk of sudden cardiac death among the elderly. Eur Heart J 24(22): 2021–2026
Moore K (1993). Iterative learning control for deterministic systems. Springer, Heidelberg
Oh J, Choi Y, Park J and Zheng Y (2004). Complete coverage navigation of cleaning robots using triangular-cell-based map. IEEE Trans Ind Electron 51(3): 718–726
Palacin J, Salse J, Valganon I and Clua X (2004). Building a mobile robot for a floor-cleaning operation in domestic environments. IEEE Trans Instrum Meas 53(5): 1418–1424
Paul R (1982). Robot manipulators: mathematics, programming and control. MIT Press, Cambridge, p 198
Polana R, Nelson R (1993) Detecting activities. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2–7
Seitz S, Dyer C (1994) Affine invariant detection of periodic motion. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 970–975
Sugano S, Kato I (1987) Wabot-2: autonomous robot with dexterous finger-arm. In: Proceedings of the IEEE international conference on robotics and automation, pp 90–97
Tsai P, Shah M, Keiter K and Kasparis T (1994). Cyclic motion detection for motion based recognition. Pattern Recognit 27(12): 1591–1603
Watanabe K, Kosaka H (1995) Evaluation of keyboard switches based on kansei (human sensitivity) information. In: Proceedings of the 4th IEEE international workshop on robot and human communication, Tokyo, pp 65–70
Yoshikawa T (1990). Foundations of robotics, analysis and control chap 3. MIT Press, Cambridge
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Park, KH., Jeong, SH., Pelczar, C. et al. Beat gesture recognition and finger motion control of a piano playing robot for affective interaction of the elderly. Intel Serv Robotics 1, 185–193 (2008). https://doi.org/10.1007/s11370-008-0018-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11370-008-0018-3