Automatic Human Body Feature Extraction in Serious Games applied to rehabilitation Robotics

Pedro Núñez, Eva Mogena, José Luis González


Current modern society is characterized by an increasing level of elderly population. This population group is usually ligated to important physical and cognitive impairments, which implies that older people need the care, attention and supervision by health professionals. In this paper, a new system for supervising rehabilitation therapies using autonomous robots for elderly is presented. The therapy explained in this work is a modified version of the classical ’Simon Says’ game, where a robot executes a list of motions and gestures that the human has to repeat each time with a more level of difficulty. The success of this therapy from the point of view of the software is to provide from an algorithm that detect and classified the gestures that the human is imitating. The algorithm proposed in this paper is based on the analysis of sequences of images acquired by a low cost RGB-D sensor. A set of human body features is detected and characterized during the motion, allowing the robot to classify the different gestures. In addition, this paper describes the human-robot interaction performed by the ’Simon Says’ game implementation. Experimental results demonstrate the robustness and accuracy of the detection and classification method, which is crucial for the development of the therapy.


Robotics in rehabilitation; RGBD image analysis; SVM; Decision tree; KNN


Instituto Nacional de Estadstica in

Global Health and Aging. World Health Organization. NIH Publication no. 11-7737, October 2011

M. Jara. La estimulación cognitiva en personas adultas mayores. Revista Cúpula.

Armeo Therapy Concept in

ReoGo Robot in

MIT Manus Robot in

L.V. Calderita, P. Bustos, C. Surez, F. Fernndez, R. Vicianad, and A. Bandera. Asistente Robtico Socialmente Interactivo para Terapias de Rehabilitación Motriz con Pacientes de Pediatra. ScienceDirect Revista Iberoamericana de Automática e Informática industrial Vol. 12, pp. 99-110, 2015.

J. C. Pulido, J. C. Gonzlez, C. Surez-Mejas, A. Bandera, P. Bustos García, and F. Fernndez. Evaluating the child-robot interaction of the naotherapist platform in pediatric rehabilitation, in International Journal of Social Robotics, pp 116, 2017.

D. Feil-Seifer and M. Matari. A Simon-Says Robot Providing Autonomous Imitation Feedback Using Graded Cueing. Poster Paper in International Meeting for Autism Research (IMFAR), Toronto, Canada, May 2012.

Microsoft Kinect for Windows Information in URL

RGBD Data Analysis for Real-Time Emotion Recognition from Upper Body Movements. F. Cid, E. Mogena, L. J. Manso3, and P. Núñez. In Proceedings of International Workshop on Recognition and Action for Scene Understanding (REACTS 2015), 2015.

R. Barrientos, N. Cruz, H. Acosta, I. Rabatte, M.C. Gogeascoechea, P. Pavn and S. Blzquez. Decision trees as a tool in the medical diagnosis. Revista Médica Vol. 9(2), pp. 19-24, 2009.

J. Trujillano, A. Sarria-Santamera, A. Esquerda, M. Badia, M. Palma and J. March. Approach to the methodology of classification and regression trees. Gac Sanit, Vol. 22(1), pp. 65-72, 2008.

KNN Algorithm from OpenCv in URL

A. Moujahid, I. Inza and P. Larraaga. Clasificadores K-NN. Departamento de Ciencias de la Computación e Inteligencia Artificial Universidad del Pas VascoEuskal Herriko Unibertsitatea.

G. A. Betancourt. Las máquinas de soporte vectorial (SVMs). Scientia et Technica Ao XI, UTP. ISSN 0122-1701, No 27, Abril 2005.

D.A. Salazar, J.I. Vlez and J.C. Salazar. Comparison between SVM and Logistic Regression: Which One is Better to Discriminate?. Revista Colombiana de Estadstica, Nmero especial en Bioestadstica, Vol. 35, No. 2, pp. 223-237, Junio 2012.

L. Manso, P. Bachiller, P. Bustos, P. N´unez, R. Cintas and L.Calderita. RoboComp: a Tool based Robotics Framework, In Proceedings, SIMPAR Second International Conference on Simulation,Modeling and Programming for Autonomous Robots. pp 251-262. 2010.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.