Keywords

1 Introduction

Over the past few years the rise of the 3D printing industry has taken the consumer market by storm. Various companies including researchers at Limbitless Solutions and the Open Hand Project have printing and developed EMG controlled limbs for children.

The inclusion of 3D printing balances rising costs and helps give a wider community access to previously expensive technology. Based on the previous research, the goal of the present study is to create a game that address the complex issues of children’s rehabilitation process pertaining to their 3D printed appendages. The University of Central Florida’s School of Visual Arts & Design have partnered with Limbitless Solutions to take their 3D printed prosthetics and address challenges that the recipients of the limbs are facing.

UCF SVAD researchers are using the actual hardware in the prosthetics to interface with games built by University students. Leveraging the newest in Serious Game design and development techniques to help interactively train the prosthetic recipient to utilize their new addition. The games utilize this EMG data to create an alternative interactive experience to help train the child in an enjoyable and stress free virtual environment. The research team is made up of a game designer, digital maker, and digital artist. Utilizing their individual expertise they have developed an interface for training the use of these arms through video games built in Unity3D. Various games are also being designed in an effort to create gameplay that would help the child achieve various benchmarks in rehabilitation. This paper discusses the challenges that the researchers faced in, not only, developing the hardware for the game but also the need for a very specific visual gameplay experiences for the device.

2 Background

This Emerging technology allows for the development for alternative solutions for rehabilitation for those who have disabilities. The current need is for a game to assist in the rehabilitation process and training of those learning to use their newly acquired prosthetic limbs. This section explores the rising awareness of accessibility within video games as well as the solutions that are being offered to level the playing field between the typical user and those who are disabled. In addition, it will delve into the use of custom hardware and software developments to benefit in rehabilitation and training.

2.1 History of Accessibility in Digital Media

The idea of game accessibility issues for the disabled is not a new issue in the field of digital media. In the late 1990’s the World Wide Web addressed accessibility issues requiring federal websites to be accessible to the disabled. In Section 508, which focuses primarily on federal websites, prompted the World Wide Web Consortium to develop universal standards to better unify the interaction of websites with their visitors. This is an early occurrence in the effort to make digital technologies more available to those who previously could not interact with them. In addition it sets the stage for later accessibility awareness and accommodations in the video game industry.

In the video game industry the same lacking accessibility challenges occur. Disabled gamers face many challenges and frustrations that their non-disabled counter parts do not encounter. Disabled gamers face challenges through various types of visual, auditory, mobility and cognitive disorders. In 1997, the US Census states that 25.5 % of the US population suffers from these types of disabilities [1]. There was a need for accessibility in video games. The International Game Developers Association (IGDA) forms the Game Accessibility Special Interest Group (SIG). The goals of this group are to define the needs of the disabled and to develop and support the creation of technology to help aid in equality for disabled gamers [1].

2.2 Addressing Accessibility Solutions

According to Huynh [2] accessibility for games can be categorized into two groups. Specially developed software and equipment that can assist the user with commercial games and, games that are designed specifically for disability rehabilitation. Today, some games are designed to be accessible, while others are not. Developments on the software side came in the form of screen readers, magnifiers and speech recognition. The hardware side focused on specialized, custom controllers. Though not all accessibly concerns are always met, the awareness of these concerns aides in the development of new interactive technologies that can be enjoyed by all.

2.3 Rehabilitation Benefits from Gaming

What if this newly developed interactive accessible technology could be not only used for entertainment, but also be used for rehabilitation in those who are disabled? The idea of using digital media as a form of rehabilitation is not new. In a study completed in 2009, researchers complied a literature search of 11 electronic databases to identify articles on the effects interactive computer play (ICP) in correlation to children with sensorimotor disorders. [3] The search was for research published between January 1995 and May 2008. In their findings, they narrowed their results to 13 out of 16 studies that found positive results with the use of interactive computer play in children who had disabilities. Nine of these studies focused on movement quality. Only two studies showed no improvement while the other seven showed positive results. [3] Not only did the findings conclude that the children found the interactive rehabilitation techniques fun, but also the children found the rehab motivating. According to their findings, the use of ICP in rehabilitation was a “highly promising area” in which further research was encouraged. [3] In the process section of this article we will address how we plan to follow their recommendation for further research.

In 2012 researchers studied the use of commercially developed video games to aid in rehabilitation of balance in lower limb amputees. The study was aimed to examine both the safety and the benefits of balance therapy in conjunction with a commercially developed video game balance board. In this 4-week study, amputation participants gained greater balance. Furthermore, the amputees’ center of pressure was decreased and they performed closer to those of typically developing children of their age [4]. This study proved that through the use of balance training with commercial video game systems that video game therapy techniques can benefit those with limb amputation. One issue that remains is that commercial entertainment games are not specifically designed for rehab. Through our research we will further explore the development of rehab specific gaming in conjunction with specialized hardware.

Custom rehabilitation game design is an area of interest to many researchers. Dawson et al. [5] recommend that future research focus on increasing measuring and recording of performances throughout training and investigating how these training tools are impacting treatment. Their study focuses on Myoelectric training systems over the past few years.

3 Limbitless Prosthetic Arm

In 2014 the team at Limbitless Solutions met a young man named Alex. Alex was born without most of his right arm. Unfortunately, the cost of a typical prosthetic was too high for Alex’s family. The team at Limbitless built a low cost prosthetic in only 8 weeks. The arm was created in the UCF Manufacturing Lab, and was powered by low cost electromyography (EMG) sensors. The total cost of the arm was $350.00, but it was provided to Alex’s family free of charge [6].

The team at Limbitless went a step further and open sourced their arm’s design and software, so others can build their own arm as well. The design for this is available on 3D printable object sharing site Thingiverse [7]. The idea being anyone who needs these prosthetics could download and build their own.

The Limbitless prosthetic arm has a fixed elbow and a hand that opens and closes. The arms are 3D printed in the UCF Manufacturing Lab 3D printers. They have a sturdy plastic feel. New models borrow designs from Marvel super heroes, like Iron Man, and Spider-Man. Designs for girls include Disney’s Frozen.

It works through a small embedded Arduino chipset that takes in signals from the user’s muscles using EMG (as seen in Fig. 1) and then moves a small servo motor to open and close the hand. The control is similar to a garage door, where the user flexes a muscle to open the hand and flexes again to close the hand. The strength and duration of these movements does not affect the motion of the hand.

Fig. 1.
figure 1

Limbitless arm in use

Full size image

The garage door style control, as compared to a squeeze to close, release to open system is to reduce fatigue in the user. Having to hold a flexed muscle to maintain grip, could quickly become tiring. Unfortunately, this motion is not naturally intuitive to new users. In an effort to train kids this control scheme, prior to them receiving their prosthetics, the system was integrated into videogames designed to train how to utilize the controls before the arm is in the hands of a new user.

3.1 The Training Interface

The prosthetic training interface is made up of two parts. The first is the actual hardware found inside the Limbitless Prosthetic Arm. The other is a custom built interface, also powered by Arduino, that takes the input from the Limbitless hardware and interfaces into the computer game developed using Unity3D, a popular game engine.

Prosthetic Interface.

The prosthetic interface side of the training hardware is identical to the hardware found in the Limbitless prosthetic arm. The internals can be seen in Fig. 2 below. This interface includes the same Arduino chip, interfaced with the same battery, servo, and EMG interface. The only difference being, that the actual prosthetic enclosure is replaced with a small hardware enclosure.

Fig. 2.
figure 2

Internals of the arm [7]

Full size image

The reason for using the exact hardware is to exactly simulate all the delays in the operational arm. Given the desire to train users to use the real arm, representing the interface accurately is of the utmost importance. Any changes in the reaction from the arm could result in negative training.

The output from this side of the controller is the signal that is usually sent to a servo motor. The Servo motor in this case is replaced by the game interface. Games can then be designed by any number of development teams, reusing the same interface.

Game Interface.

The game side of the interface, while also Arduino based hardware, is completely agnostic to the Limbitless side of the hardware. The reason for doing this is, to ensure that the Limbitless side works exactly like the real arm hardware. The delay of the second Arduino is negligible, as no additional hardware or sensors get in the way, and there is no additional load on the Limbitless hardware (Fig. 3).

The game interface is based primarily on an Arduino Uno and a Unity3D plugin called Uniduino. Uniduino allows for interaction between the Arduino hardware and Unity3D [8]. This allows the Limbitless signals to be input directly into a game.

Fig. 3.
figure 3

Arduino interface to Unity3D game using Uniduino [8]

Full size image

4 Game Design Methodology

In an effort to design a variety of games and test out as many design concepts as possible within the games. The Limbitless Prosthetic arm interface was implemented in 14 games developed for the course Casual Games Production in the University of Central Florida’s School of Visual Art and Design’s Digital Media Game Design track, in the Spring Semester of 2016.

Each game was designed by a group of four students on average, with a reasonable mix of both artists and programmers. In some cases it might weigh more heavily to one side or the other. The games started by selecting an Atari 2600 game to base their core mechanics on, and then changed a mechanic and added a mechanic. This exercise leaves the games having the simple controls of an old school Atari game, but also made something uniquely different from what others had played before.

An example of one of these games is Time Tilt. This is a game based loosely on the game Joust, but with power-ups, guns, and a time travel motif. While the game is reminiscent of the classic game, it still feel original. This game is an especially good example, as the flappy bird like control is well suited to the prosthetic interface.

After designing and creating the games, the students were provided code to allow them to interact with the prosthetic interface. This interface was easily implemented and the EMG replaced a button in the game. The games still maintained other controls, like directions, that could be input by the other arm of the participants. This provided the ability to not only practice with the EMG interface, but coordinating that practice with their other limb.

5 Design Consideration

Through the development process 14 games were developed. Of the 14 games created some worked extremely well with the technology and some did not. While developing these games we have discovered some issue that need to be taken into account in order to develop optimally for the prosthetic interface. These are described in more detail below.

5.1 Gameplay Should Match Task

When designing a game to train a task it is important to consider the transferability of the task. In the case of designing games for prosthetic use there is a propensity in some designers to include the controls as a simple replacement for an existing button in any game. The issue is, the prosthetic is used in a very specific way. Games that simulate the act of grabbing or closing will have a larger impact on the training outcomes.

5.2 Perception vs. Functionality

The prosthetic arm uses a garage door like control scheme. This means you trigger it once to open the hand you trigger it again to close it. These messages can be interrupted but holding the command does not increase the power or produce any extra response. Unfortunately, game designers are used to programming functions around holding a button down. This is commonly used to charge an attack, or increase the height of a jump. Fighting back against this design pattern can be hard for game designers.

5.3 Number of Controls

Ideally games would only use the prosthetic controller. This limits the control input down to only one button, but it would allow the user to concentrate on learning the control. This, however, limits the types of games that could be made. Given the user most likely has an existing hand they can use. The control can be mapped to both the prosthetic control and the buttons available to a single hand on a standard controller, a keyboard or a mouse. It is recommended that this be limited to direction buttons.

5.4 Input Speed

The EMG can produce responses every 16th of a second but it can take half a second to see reaction in the arm. So, controls cannot be fast twitch even though that is generally considered fun. The controls should react in the time that the actual hand can respond, even if the hardware can take in the information at a faster rate. This delay can be simulated in hardware, but the games still need to be designed to take that into account during game play.

5.5 Complexity of Game Activities

The activities in the games should be fast enough to allow for as many opportunities to practice as possible. This means the game should be fast paced and easy to understand. While Role Playing Games are popular they often have long periods of unraveling story or exploring environments and not fighting enemies, or flapping wings on a bird, etc. These games are not well suited to this type of training. Casual or Arcade style games on the other hand are good examples of what could be made for this type of training.

5.6 Design with Interface in Mind

The previous design lessons lead to the conclusion that games should be made specifically for the prosthetic arm interface, and existing games should not be shoehorned into working with the interface. The reason for this is, games that do not take the previous concerns into account may have issues that will lead to negative training. There are, however, lots of opportunities to redesign existing games to bring them in line with the requirements of the interface.

5.7 Art Should Appeal to Target Audience

The visual appeal of the games need to be audience appropriate. In an effort to ensure that these games would appeal to young individuals, the art styles for the game built were based on existing picture books. These books provide example art styles that are appealing to a young audience and have a simple design language to make it easy to audiences to understand them, and relatively easy for developers to implement.

5.8 Games Should Be Fun

When designing games for kids, the games absolutely need to be fun. While there will be motivation to learn how to use their prosthetics, kids still have access to so much more interesting media and games, and these games need to be able to hold their attention. In an effort to accomplish this, game mechanics were borrowed from existing great games. Great effort was put into making the games fun. This includes applying the design patterns above, but also through user testing, and focus group testing, with users. It is hard to make a fun game, and making one that is fun that is also useful is so much more difficult.

6 Conclusions

As expected when designing a game for training, matching the training domain and accurately simulating the environment were incredibly important. Further designing for children is another challenge that needed to be overcome. Making games that work, train, are appealing to the audience, and fun to play is a demanding challenge. At the same time, designing games for use to train prosthetics has been a great experience. Not all the games designed for this worked out, but we still learned some design patterns that can be applied to future games.

As the Limbitless prosthetic arm reaches more children, the games developed here can be used as pre-training. Thus helping children get up and running more quickly.

6.1 Future Work

The interface should be released to the public, and be available to download for free from the Thingiverse webpage next to the Limbitless Arm design. This would allow for the greatest number of people to benefit from this work. The games could also be used with other prosthetic systems. Like those controlled by the Myo Band [9, 10]. Integrating that same technology should be a relatively low challenge, with a high impact.

The interface could also be used for non-prosthetic VR applications. Anything that could use input from EMG. This might include exercise in meditation, or more tangible tasks, like controlling robots. Now that the system is built the applications are Limbitless (pun intended).