Keywords

1 Introduction

The recent popularity of 3D printed technology to create prosthetic limbs is not surprising [1]. These technologies drastically reduce the cost of prosthetic arms making them available to children, a population that often time had to go without a prosthetic, until they stopped growing. This left them with a lifetime of not using a prosthetic. As a result, several groups have formed to help bring arms to children who need them [2]. One such group is Limbitless Solutions, a non-profit organization at the University of Central Florida (UCF) dedicated creating personalized prostheses for children with disabilities [3].

The mission of Limbitless Solutions if different. The organization not only creates innovative arms, but also incorporates the interests of the children receiving them into their design. Children can get arms that look like super heroes, magical creatures, video game characters, or anything the like; making the arm not just a prosthetic but an extension of their personality.

As a new generation of arms is being designed at Limbitless Solutions there is also a need to train the children to use them. In an effort to find a fun way to incorporate training and real-time feedback a game based training system has been developed.

2 Background

As of 2005 the number of cases of limb loss in the United States had increased to 1.6 million [4], with projections expecting these numbers to advance to over 3.5 million by 2050. A variety of factors are driving this rapid increase including dysvascular conditions (which in part can be due to diabetes), trauma events, cancer cases, and congenital amputations (Health Care Utilization Project National Inpatient Sample (HCUP-NIS), 1996. Studies have shown that underrepresented minorities have risk factors of 2 to 3 times as likely as non-Hispanic [5].

Congenital upper limb differences occur approximately 2.6 of every 10,000 live births [6] and impacts upper limbs at 1.6 times as frequently as lower limbs. For children, congenital birth defects are the leading cause of limb differences [7]. There remain several challenges for prosthetics adaptation (costs, children’s growth rates, insurance and health care policies just to name a few), and the limited extent of adoption has an impact on both the populations’ functionality (both children and adults) as well as their community participation. However, over 85% of artificial limbs worn are for lower limbs as the accessibility and adoption for upper limbs has far lagged behind [8].

Quality of life scoring for people with limb difference has helped to identify different aspects of the challenges associated with upper limb differences including: physical disability, pain, energy, emotional reaction, social isolation, and quality of sleep [9]. For both male and female persons, emotional reaction and social isolation scored the lowest quality of life findings, even beyond the physical limitations. This has also been observed with the pilot study of bionic kids that have worked on the prototype development here at Limbitless Solutions at the University of Central Florida. These psychosocial factors interact with children’s development progression (Psychological adjustment and perceived social support in children with congenital/acquired limb deficiencies. Journal of behavioral medicine) and can be a burden. Social support plays a large role in Lazarus’ cognitive model [10], and children with limb deficiencies have linked deficiencies with depressive symptomatology, trait anxiety, and general self-esteem. In the words of our bionic kids, Alex was afraid to go to the grocery store because of how people would state or speak to him due to his limb difference.

Functionality, children are highly adaptable and are observed to overcome nearly any challenge [7] on a day to day basis. However, these challenges cause substantial efforts and potential fatigue points in everyday tasks [11]. From the experience with our bionic kids, using their bionic arm to support their bike riding they were able to bike further with less physical strain for longer durations of time, whereby being able to engage in more meaningful engagement with the peers that may lead to a stronger support system and psychosocial development. Zachary’s (one of our bionic kids) family described to our team how the need to contort (Zachary’s) spine to have his limb difference arm reach the handle-bars limited the time he could spend in the activity and contributed in him being left behind during activities with his friends.

Further, younger children demonstrate higher degrees of usefulness with a prosthesis as compared to older children [11]. Our mission is to provide children and younger adults a functional and expressive electromyographic actuated arm experience, by increasing access to the technology that can also grow with them as needed via 3D printing modular components.

A major challenge that may limit the adoption of electromyographic bionics, particularly for children, is learning and adapting to the control schemes (Surface EMG in advanced hand prosthetics). Active controls for multiple gestures have remained complex and include machine learning techniques [12, 13] but have larger computational requirements. This may be a limiting factor towards the adoptability for children to have multi-gesture controls. More recently, an effort has been made to gamify the learning of prosthetics control [14]. This has provided feedback that better links the stimulus (a muscle flex) with a potential set of outcome engagements on a prosthetic.

Our team has taken this a step further and has created multi-gesture electromyographically triggered video games that utilize a special electromyographic controller. This has produced a study to capture the engagement of the training games for children, with excellent affinities reported thus far. In our program, a single electromyographic sensor has been employed with multi-gesture capability with short pattern recognition and magnitude control, that has been simplistic enough for children to quickly grasp and engage. This type of engagement is believed to encourage stronger affinities and subsequent adoption of the device for children and young adults, which is believed to correlate to stronger social integration, spatial (dexterity) manipulation, and community engagement. These factors correlate to long term success from the classroom, to the career field, or in the home.

3 Current State of Development

The current state of the technology has produce a fully functional prototype for both the bionic limbs and video game training system featured here in Fig. 1. The bionic arm features custom circuitry, professionally manufactured, electromyographic binary actuation for hand open and close. Battery life currently averages approximately 8 h and can be charged with equipment similar to a laptop charger. The video game training system includes a custom electromyographic actuated wearable controller, that connects via USB to a computer, and several games have been produced and demoed for children with and without limb deficiency. The system has been well received and survey information is currently being collected for evaluation.

Fig. 1.
figure 1

Bionic arm prototype (left) and video game training system (right)

Full size image

Fundamental technology improvements to the product are for development. These include improving both the (i) dexterity and (ii) degrees of actuation. The assessments proposed are designed to give feedback on the necessity of the refinements as well as the refinements implementation, across the course of the five-year study. Multi-finger actuation dexterity is of primary importance. This includes functional gestures, for example pinch grip. The current state of the prototype hardware allows for individual finger controls, but refinement to the software for assessing intentionality and cycling through gestures is still in development. Examples of the upgrade hardware are presented in Fig. 2. The training software protocols are designed to teach coordinated flexing patterns and magnitudes, while also assessing the ease of capturing intentionality of the user.

Fig. 2.
figure 2

Multi-gesture bionic hardware design on right with exploded view on left

Full size image

Also in development is an electromyographic mechatronic elbow. This, as is the case with the multi-finger actuation, can be achieved with only a single electromyographic sensor via gesture analysis. This proof of concept has been completed and tested in our prior work, and allows for a reduction in weight and hardware complexity which benefit the user’s experience greatly. The mechatronic elbow will substantially improve the functionality of the above-elbow congenital amputee population, but requires the novel training programs to ensure successful adoption particularly for children.

3.1 Video Game Prototypes

Learning to use prosthetic limbs is challenging for children. A portion of the proposed funding is to design engaging training games that teach children how to use their new limbs without boring or fatiguing them. The interdisciplinary design team, comprised of digital media faculty, researchers, engineers, and health professionals, strives to create innovative solutions. The training program will provide recipients with the opportunity to become proficient at using their prosthetics to ensure their successful, long term use of these limbs. This collaboration among physical trainers, engineers, and psychologists will create fun, kid-friendly training solutions that follow sound physical training guidance. The Limbitless Solutions game design team has already developed game controllers that utilize electromyographic sensors bridging the 3D printed prosthetic multi-gesture functions to the training games. The award-winning beta versions of these games were created from seed funding and support from the University of Central Florida (UCF). The prototype games were featured at the Smithsonian American Art Museum in Washington D.C. and the Game Developers Conference (GDC) in San Francisco. These prototypes succesfully showcased hybrid multidisciplinary interactive experiences that deliver both arts, training, and cultural impact.

Our interdisciplinary team is comprised of game design and digital media professors, researchers, and visual artists, engineers, and health professionals UCF. We seek to further evolve these individual games into a multi-faceted training program for limb recipients. The developed games will train the user by establishing a multi-phase training system varying in difficulty. Games within each phase will train different gesture types: (i) grasping and releasing; (ii) pre-set finger combinations (thumbs up, peace sign, etc.); and (iii) individual finger dexterity.

The multi-phase system, drawing from the strengths of such a variety of disciplines and fields of knowledge, will integrate encouragement, humor, and, most importantly, fun, to provide a safe and healthy training experience for children learning to use their 3D printed prosthetic limbs, placing them on the pathway to a life of increased capabilities and success.

3.2 STEAM Education: Internal and External

The merging of arts, engineering, and health care is pivotal to the success of our project. Our game design teams are composed of over 25 current Science, Technology, Engineering, Arts, and Math (STEAM) undergraduate, graduate, and doctoral students. The melding of disciplines and mindsets allows true cross-discipline innovation to occur. Externally it is important for the team to also work with their client base. The game design team receives direct input on art content and narrative from the recipients of the bionic arms. The team also works closely with area elementary schools in showcasing the impact art and engineering games have on the community, and has recently become a destination for local schools to visit as part of an engagement field trip for STEAM learning.

4 Evaluation Methods of a Prosthetic Arm

Prosthetic arms for children provide an impressive number of challenges. It is important to understand exactly how they are working and how to make them work better for the children. The same can also be said for the games research. There are a variety of assessment vehicles for quantifying the impact and use of prosthetics and video games; several methods are presented herein.

4.1 Assessment of Capacity for Myoelectric Control (ACMC)

This assessment for myoelectric device control is a Rasch rating scale that is used to detect expected change in a person’s ability using objective variables. There are 30 items that evaluate a prosthetic arm’s ability to do specific functions that involve gripping, holding, releasing, and coordinating between limbs. This is done by asking the subject to perform certain tasks and scoring their motions by a trained professional [15].

4.2 Unilateral Below Elbow Test (UBET)

UBET was designed to evaluate the functionality of individuals both wearing and without wearing a prosthesis and is comprised of nine age appropriate tasks. Two scales for quantified scoring including Completion of Task and Method of Use are used to evaluate a child’s function and adaptability on a 5 point scale. These scales range from 0 [unable to complete the task] to 4 [completes task without difficulty]. This analysis method allows for a direct comparison of the individual with and without their prosthetic device for measuring direct function improvements [16].

4.3 Pediatric Outcomes Data Collection Instrument (PODC)

PODC is a well-validated musculoskeletal health questionnaire, and published results of this measure are available for the general population and for a normal population. PODCI questions address both the activity and the participation components of function. The PODCI contains six validated scales, or domains (Upper Extremity Physical Function; Mobility/Transfers; Sports/Physical Function; Pain/Comfort; Happiness; and Global Function, which is a combination of Upper Extremity Physical Function, Mobility/Transfers, and Sports/Physical Function), with a total of 108 questions. The maximum score in any domain is 100 points, and scores below the low 80 s are considered to represent below-normal function. Parents answer for children who are two to ten years of age, and children and parents answer for those who are eleven to twenty years of age [17].

4.4 Pediatric Quality of Life (PedsQL)

PedsQL is a well-validated survey that asks twenty-three questions of both parents and children about various aspects of health-related quality of life over the past month. Published results are available for the general population. The PedsQL scoring algorithm translates the available responses to questions (“never,” “almost never,” “sometimes,” “often,” or “almost always”) into scores of 0%, 25%, 50%, 75%, and a maximum of 100% for each of four generic core scales (Physical Health, Emotional Functioning, Social Functioning, and School Functioning). The raw data used to obtain Emotional, Social, and School Functioning scores are averaged to obtain the Psychosocial Health Domain score, and the raw data used to obtain all four core-scale scores are averaged to obtain a Total Scale Score. Parents answer for children who are two to four years of age, and both the parents and the children answer for those who are five to twenty years of age [18].

4.5 Prosthetic Upper Extremity Functional Index (PUFI)

PUFI is effective at measuring the extent prosthetic limb use, the challenge or ease of use, and overall task performance as reported by a parent or the individual. It categorizes real life tasks by age group for evaluation that can include for example: zipping a zipper, grasping both handles of a bicycle, or peeling a banana just to name a few. In addition to task performance completion, the ease of use is measured for a more comprehensive assessment [11].

4.6 System Usability Scale (SUS)

SUS is an effective survey for measuring both the usability and context for video games. It is based on a Likert scale, with degrees of agreement and disagreement on a 5 point scale. The composite score is ranged from 0 to 100, and has been used in the past to measure and contrast both the software as well as the hardware systems [19].

4.7 Game User Experience and Satisfaction Survey (GUESS)

GUESS this survey helps to measure the engagement of a video game for a group of users, and is usually scored at the end of their play-testing. It includes categories such as: engagement, immersion, flow, presence, and existing gaming scales. This survey vehicle is critical for evaluating a variety of games for a variety of participants, and helping to determine the resonance and play-ability of these in-house developed training games [20].

5 A Game Design to Support Multiple Gestures

Disabled video gamers are an often overlooked community in the gaming world. In 2008, Information Solutions Group surveyed over 13,000 casual gamers. In that survey 20.5% identified as disabled gamers. Furthermore, more than 10% have stated that they have had casual games recommended by a doctor; The survey also deemed that disabled gamers play longer and more than non-disabled gamers. [21] The team previous experience of accessibility forward game design that has been showcased on an international academic level. The games research team is composed of researchers in the fields of game design, visual arts, and applied design. The games team has already successfully created and implemented EMG training controllers that interface with the 3D printed prosthetic arm of Limbitless Solutions.

These initial games utilize an interface that allows for a thresholded input. That is when a user is a rest the EMG will generate a lower value then when the muscle is flexed, and while this number can be different between users a simple calibration allows the user’s flex to be read as a single button input to a video game. Early games would use the thresholded input to shoot a gun on a space ship, manipulate a finger picking a nose, have a dog jump a fence, and many other fun activities that provided instant feedback to the player. This input worked very similarly to the input of the prosthetic arm, which opens and closes like a garage door opener on flex. Figure 3 shows a rendering of the EMG controller on the left and a screen shot of Beeline Border Collie on the right.

Fig. 3.
figure 3

Rendering of the custom EMG game controller left and Beeline Border Collie right

Full size image

Beeline Border Collie is very similar to the popular game Flappy Bird. These simple games provide a great introduction to how the prosthetic arms work, and allow for exercising prior to receiving a prosthetic arm. But as the arms improve in quality the need to train more complex behaviors is needed. These games need to consider the input needed to manipulate a more complex arm with gestural controls and the ability to individually articulate fingers. In an effort to explore this control the game Magical Savior of Friends was born.

Magical Savior of Friends follows a small elf like magician that is helping a group of frogs fight off an evil group of snakes. The snakes are led by Sir Snikelsworth, a greedy landlord that has raised the rent on the frogs to the point that they can no longer afford to live in the kingdom. The game provides a rich platforming experience, similar to what might be found in a Mario Brothers game, but with one major difference. The player can control magic based on how they flex.

Using a proprietary celebration technique, the game can not only read if the player is flexing, but can also determine intensely they are flexing, and for how long. In the game this allows for 3 types of lightning magic when using arcane magic. This might be called multi-thresholded input. It allows players to throw a rock at various angles using earth magic. This is a truly analog input. Water magic requires multiple flexes to control. Fire requires sustained flexing. Air power requires sustaining and thresholding to fly around like in Flappy Bird. These magics can be combined in complex ways and each will represent a real function in the prosthetic arm. This game is training players to flex patterns that will empower gestures or individual finger articulations in the future.

Figure 4 shows the Magical Savior standing on a platform near one of the sage frogs. A collectable coin is seen on the edge of the screen. There is also an imprisoned frog on the far right. These frogs can be found hidden in levels and freed. In the upper left is the flex meter. It looks like a hand and shows the player how much they are flexing at any given point in the game.

Fig. 4.
figure 4

A screen shot of magical savior of friends

Full size image

6 Conclusions

The ability to use games for training with this child based population is incredibly powerful. The children are engaged in the training and have created fan art and stories around the characters in the games. The development team has incorporated this feedback into the games and so the games feel custom built, but also personally designed by the children. Making learning a fun and rewarding, has allowed for more complex behaviors to be mixed in quickly. Also, the games platform can act as a prototyping platform for new ideas in the future.

6.1 Future Work

As the user progresses through the training game, they will unlock the use of different in game abilities. These abilities will directly coordinate with the controls and rhythmic patterns needed to control multiple dexterous gestures in the prosthetic arm. The games and controllers utilize a custom built electromyographic game controller that can be calibrated to the individual user. Typically, without the games, it is difficult to communicate to a child the percentage of flex required to operate a specific gesture for the bionic arm. Furthermore, it becomes a challenge to then incorporate multiple flex levels and patterns and to be able to differentiate them from other commands. The games calibration system also acts as a visual cue to distinguish different magnitude levels of flex. This allows the research team to map those different levels of flex to specific gestures for the arm. In the game play each level of flex will be mapped to a distinct mechanic that is pivotal for the games success. For our bionic kids who have tried the beta training games, we have seen immediate results in gesture learning and engagement.