Computer vision for assistive technologies
Introduction
The term “Assistive Technologies” (AT), in the broadest sense, refers to any set of scientific achievements (products, environmental modifications, services and processes) useful to overcome limitations and/or improve function for an individual (Cook and Polgar, 2014). More specifically AT aim at helping persons with disabilities or special educational/rehabilitation needs, to deal within their daily context and to achieve a better quality of life (Lancioni et al., 2012). In general there are two main application contexts in which AT are exploited: 1) the medical application context that tries to reduce (or rehabilitate) physical or cognitive impairments and 2) the social application context that operates on the surrounding environment focusing on social barriers and discriminations (Hersh and Johnson, 2010). In the last decades there has been a tremendous increase in demand for new technological solutions allowing an improvement of the quality of life, e.g., for elderly people or people with different abilities, as well as for people without disease but willing to increase their comfort. Many researchers, working in different fields, have applied their knowledge to build advanced technologies in order to meet the needs of diverse AT’ application contexts (Bishop, 2014, Bourbakis, Papadakis Ktistakis, Tsoukalas, Alamaniotis, 2015, Dakopoulos, Bourbakis, 2010, Jo, Tsun, Theng, Hui, 2014, Lancioni, Singh, 2014, Murphy, Darrah, 2015, Pawluk, Bourbakis, Giudice, Hayward, Heller, 2015, Velázquez, 2010, Vichitvanichphong, Talaei-Khoei, Kerr, Ghapanchi, 2014, Vuong, Chan, Lau, 2015). As a consequence, works that deal (directly or indirectly) with the development of AT have proliferated in the literature and therefore the issue to properly organize them has emerged. This categorization task has been then carried out by grouping the works dealing with the same area of user needs: e.g. dementia (Vuong et al., 2015), visual diseases (Murphy and Darrah, 2015), social orientation deficit (Bishop, 2014), activities of daily living for aged persons (Vichitvanichphong et al., 2014), severe/profound and multiple disabilities (Lancioni and Singh, 2014), physical and cognitive disabilities in children (Jo et al., 2014), smart environments (Velázquez, 2010). This type of categorization could be defined as “user-need oriented” and it is the foundation of all the existent surveys concerning AT: they are very interesting and useful for readers (medics, patients, companies, etc.) who address the problems related to a specific need, hence giving the overview on the state of the art technologies supporting the related function for the individual.
However in this common way of looking at the AT world, each technology is considered as a whole and a deep and critical explanation of the technical knowledge used to build the operative tasks is usually missing. As a consequence, the cross-contextual applicability of most of the underlying techniques and methodologies involved in each specific context is not considered and then the resulting surveys are unlikely to be technically inspiring for functional improvements and to explore new technological frontiers.
To overcome this critical drawback, in this paper a “task oriented” way to categorize the state of the art of the AT works has been introduced: it relies on the split of the final assistive goals into operative tasks that are then used as pointers to the works in literature in which each of them is used as a component.
The potential usefulness of this new way to categorize the state of the art of the AT works can be easily foreshadowed by considering that each technical task can be part of different AT dealing with quite different user’s needs. For example object recognition from visual cues can be part of an assistive device for the indoor navigation for visual impaired persons, or part of an alternative communication interfaces, as well as a module in a social robot platform, and so on. Keeping this observation in mind, it follows that by collecting works dealing with the same task, regardless of the application context, it becomes easier to highlight the straightness and the open critical issues, allowing the reader to identify the research lines to be undertaken for an improvement of the existing assistive technologies.
Unfortunately, as already stated above, in literature there are no papers that use the “task oriented” approach and then this paper tries to partially fill this gap by using a set of cross-application Computer Vision tasks as the pivots to establish a categorization of the AT already used to assist some of the user’s needs (Mental Function, Mobility, Sensory Substitution and Assisted Living) pointed by the World Health Organization. For each task the paper analyzes the Computer Vision algorithms recently involved in the development of AT and finally it tries to catch a glimpse of the possible paths in the short and medium term that could allow a real improvement of the assistive outcomes. The rest of the paper is organized as follows: in Section 2 the most common areas of users’ needs are identified and, for each area, the list of the underlying relevant cross-application AT tasks involving Computer Vision algorithms are pointed-out. In Section 3, for each identified task, the analysis of the leading methods and techniques reported in literature to address it in any assistive context is given. Section 4 considers the assessment of AT considering users, medical, economical and social perspective. A discussion about open challenges is then supplied in Section 5 where a glimpse into the possible emerging new solutions is also given. Finally Section 6 concludes the paper.
Section snippets
Users’ needs and related Computer Vision tasks
The range of human needs to be addressed by AT is quite large, however, according to the Flagship programme – Global Cooperation on Assistive Health Technology (GATE), developed by the world Health Organization (WHO) (http, 2015), most of them can be grouped into the following classes:
- 1.
Mental functions
- 2.
Personal mobility
- 3.
Sensory functions
- 4.
Daily living activities
- 5.
Orthotics and prosthetics
- 6.
Communication and skills training
- 7.
Recreation and sports
- 8.
Housing, work and environmental improvement
Among the above
Computer Vision techniques involved in AT tasks
In the previous section the most relevant technologies involving computer vision tasks have been introduced for each area of user’s needs. In this section, for each of the aforementioned computer vision tasks, a literature review of the leading methods and techniques which have been already employed to address assistive issues is given.
Critical assessment of AT: users, medical, economical and social perspectives
As discussed in the previous sections, the great advances in computer vision allow to obtain new and more powerful assistive devices and to improve existing ones, thereby increasing the potential benefits that can accrue. However, at the same time, this fruitful evolving brings out the need to take proactive and sophisticated forms of assessment of AT. Critical assessment of AT is a not trivial process that can take a long time considering that there are several actors involved and that it is
Open challenges
By analyzing the state of the art reported in previous section some still open technological challenges emerge. These challenges are explained in this section and, for each of them, some methods addressing the underlying problems which have been recently presented in literature are briefly discussed, explaining also how they could allow a real improvement of the assistive outcomes in the short and medium term. This discussion about present challenges in AT and possible ways to address them
Conclusions
This paper focused on the computer vision components exploited in the existing assistive technologies (AT). In particular, a “task oriented” way to categorize the works related to AT is proposed starting from the consideration that the same computer vision task (e.g. object detection) can be part of technologies dealing with quite different user’s needs and thus, by grouping the assistive technology works dealing with the same CV task could help the reader to better identify the research lines
References (209)
- et al.
Robust multi-dimensional motion features for first-person vision activity recognition
Comput. Vision Image Understanding
(2016) - et al.
Multi-face tracking by extended bag-of-tracklets in egocentric photo-streams
Comput. Vision Image Understand.
(2016) - et al.
Wize mirror - a smart, multisensory cardio-metabolic risk monitoring system
Comput. Vision Image Understand.
(2016) - et al.
How children with autism spectrum disorder behave and explore the 4-dimensional (spatial 3d+ time) environment during a joint attention induction task with a robot
Res. Autism Spectr. Disord.
(2014) - et al.
A novel 2d/3d database with automatic face annotation for head tracking and pose estimation
Comput. Vision Image Understand.
(2016) - et al.
Design of an efficient framework for fast prototyping of customized human–computer interfaces and virtual environments for rehabilitation
Comput. Methods Programs Biomed.
(2013) - et al.
Towards a robust affect recognition: automatic facial expression recognition in 3d faces
Expert Syst. Appl.
(2015) - et al.
Towards an affordable mobile analysis platform for pathological walking assessment
Robot. Auton. Syst.
(2015) - et al.
A real-time human-robot interaction system based on gestures for assistive scenarios
Comput. Vision Image Understand.
(2016) - et al.
Expressive visual text-to-speech as an assistive technology for individuals with autism spectrum conditions
Comput. Vision Image Understand.
(2016)