Abstract
During neuroradiological interventions, physicians need to interact with medical image data, which cannot be done while the hands are occupied. We propose foot input concepts with one degree of freedom, which matches a common interaction task in the operating room. We conducted a study to compare our concepts in regards to task completion time, subjective workload and user experience. Relative input performed significantly better than absolute or rate-based input. Our findings may enable more effective computer interactions in the operating room and similar domains where the hands are not available.
1 Introduction
Minimally-invasive radiological intervention is a growing field in which a variety of diseases can be treated without open surgery. During such interventions, the physician in the operating room (OR) navigates a catheter or needle tip to the desired pathological structure inside the patient’s body through small incisions [21]. This is a huge advantage since smaller incisions mean faster healing and shorter hospital stays for the patient, which in turn leads to reduced costs. In such a procedure, a needle is directly inserted through the skin or a catheter is inserted into a blood vessel and navigated through the vessel system. Though the instrument is not visible to the physician, the procedure relies on x-ray images which are acquired by an interventional angiography system. The acquired images are not only used for navigation, but also stored and employed for later reference during the intervention [15]. Furthermore, preoperative datasets from other imaging modalities, e. g., computer tomography (CT) or magnetic resonance imaging (MRI), can be accessed. This requires the physician to interact with interventional imaging software while operating.
In clinical routine, navigating medical images is a cumbersome task due to the provided input methods. A common solution lies in using plastic-sheathed controls like joysticks and buttons. Surgeons therefore need to change their position when these devices are out of reach [12]. Alternatively, interaction tasks are delegated to a medical assistant located inside the OR or outside in a non-sterile control room. Verbal or gestural communication is error-prone, can lead to time-consuming misunderstandings and may even require physicians to scrub out and in again to interact with non-sterile workstations [8], [22]. To this end, intraoperative touchless interaction has been subject of research for many years. A literature review of touchless interaction in the OR provides an overview which shows a focus on optical gesture recognition [18]. Limitations in the OR such as the close proximity of other team members at the operating table, restricted movement of the upper limbs due to sterility considerations and partial occlusion of the surgeon’s body can interrupt line-of-sight for optical gesture recognition systems. Furthermore, the problem of touchless control when both hands are occupied by holding medical instruments has to be considered [22].
To tackle these problems, we investigated methods to interact with medical image data with the feet while keeping the hands free for other tasks. Foot pedals are already established in clinical routine, but suffer from a lack of visual control and the need to keep the weight on one foot [32], [35]. Therefore, our approach focuses on heel rotation since literature suggests it is the less exhausting method of foot interaction. Three concepts which apply different input techniques are presented and combined with adequate visual feedback. To evaluate our approach, we present a quantitative user study to compare these concepts with regards to task completion time, subjective workload and user experience. We contribute to human-computer interaction (HCI) by determining adequate foot input concepts for tasks with one degree of freedom when standing. This work may enable more efficient interaction in the OR as well as other domains such as industrial manufacturing where hands-free interaction is required.
2 Related Work
The following section gives an overview of related research from HCI in general, as well as from the specific field of computer-assisted surgery. Methods for touchless interaction in intraoperative scenarios are described, as breaking asepsis when using established input devices such as mouse, keyboard or touchscreens is one of the major concerns during surgery. The current state of foot interaction in the operating room as well as challenges and improvements are described next. Literature on foot movements is reviewed to connect findings from the broader field of HCI to medical requirements. In the end, input techniques are described and existing work is put into context.
2.1 Touchless Interaction in Sterile Environments
Several research projects conducted in the domain of touchless HCI in sterile environments focus on optical sensors such as the Microsoft Kinect or the Leap Motion Controller [4], [7], [18], [20]. Besides the fact that they keep the hands busy, optical tracking approaches rely on a clear view of the camera which can be obstructed by staff members or degraded by sterile plastic sheathing. Other devices such as the Myo Gesture Control Armband [11] or voice recognition have been investigated. Sole speech recognition seemed to be sensitive to background noise, pronunciation, accent, and choice of commands [1], [4]. Voice commands are suggested for fast and direct navigation to data sets via the patient name, date or imaging modality in a database, which would be cumbersome with gesture controlled menu navigation. Combined with alternative input methods for continuous control, it was found suitable for intraoperative image manipulation. Redundant input methods allowed use of gestures instead of voice commands in case of noisy situations [17].
2.2 Foot Interaction in the Operating Room
In the medical domain, interaction by foot is mostly done via foot pedals. They are used for controlling diathermic, ultrasonic or x-ray imaging equipment while the hands are occupied with handling medical instruments [15], [35]. Even though foot pedals often work like push-buttons, multiple functions can be controlled by combining pedals. Van Veelen et al. investigated common issues with foot pedals in the operating room [32]. They reported 91 % of their 45 subjects (25 surgeons, 20 residents with experience in laparoscopy ranging from 2 to 10 years) occasionally lose contact with the foot pedal and reestablish it by looking down (73 %), feeling with the foot (63 %) or asking a nurse for assistance (44 %). Further, ergonomic concerns arise from the strategy to hold the foot in dorsal flexion (i. e. toes up) and keep the weight on the other foot to prevent loosing the pedal in the first place. Nevertheless, there are tasks that can’t be delegated to assisting staff because of timing issues or because direct interaction is required [15].
Different aspects of foot interaction in the OR have been the subject of research to overcome these difficulties. To avoid loosing the pedal and provide a relaxed idle position, a foot pedal was mounted on an overhang and equipped with a hook-and-loop fastener to be strapped onto the surgeons foot [30]. With the problem of occupied hands during surgery in mind, foot input devices for different interaction patterns such as drawing shapes, stepping on discrete position and weight shifting have been outlined but not evaluated [5]. A capacitive flooring has been combined with a wrist-worn IMU to interact with medical image data sets in a lab setting [14]. Applied to image interaction in an virtual-reality OR, foot interaction performed comparable to hand gestures and verbal task delegation [25].
2.3 Suitable Foot Movements
Foot interaction to control software has been investigated in various contexts. First approaches dating back to 1985 tried to avoid switching hand positions between mouse and keyboard on a desktop workplace [27]. Pearson and Weiser described four topologies for foot movement in a seated position and discussed corresponding technical approaches [24].
When using foot interaction in confined spaces and in a standing position, gestures are limited and therefore need an adequate input device and input technique, matching the requirements of the task [13]. For seated positions, participants preferred dragging the foot over lifting it. The most comfortable interaction method was short, in-place movement such as pivoting around the heel [31], [33]. Aligning with these findings, participants preferred heel rotation over dorsiflexion, plantar flexion and toe rotation. Additionally, external rotation (i. e. turning the foot outwards when pivoting on the heel) was preferred over internal rotation (i. e. turning it inwards) [29]. Foot movement in general quickly becomes exhausting when a posture has to be held. Resting positions are therefore suggested for seated as well as standing foot interaction [2], [31]. For activation or deactivation of a function, toe lifting is considered suitable [31] and preferable over sliding gestures [14]. Heel and foot rotation has been investigated for different applications. A prototype for menu item selection by means of lifting and rotating the foot was presented by Zhong et al. [36]. Foot rotation as alternative to a pair of conventional foot pedals has already been investigated in the medical context and was able to improve ergonomic and security aspects [32].
2.4 Input Techniques
Another important factor is the input technique. Techniques to use foot postures for continuous input can be divided into discrete rate-based, absolute and relative approaches. Rate-based input continuously modifies a parameter as long as a certain condition is met, like joysticks or a car’s accelerator pedal. Custom foot pedals deliver feedback and can be used for 1D and 2D-tasks [16] but may be triggered involuntarily when the foot is at rest [34]. Pressure distribution performed well for rate-based interaction in a non-mobile setup [28] but needed to be trained to the user when investigated for daily situations [6]. Kicks were found advantageous over holding a foot posture, since the foot could be rested between subtasks [2]. Absolute input maps foot movement to one or more parameters in a direct fashion. Applied to 1D and 2D tasks, the difficulty in reaching small targets was reported as the biggest challenge when using foot movements under the desk [33]. This aligns with the reported median selection error of 8.52° for heel rotation [29]. When heel rotation is used for item selection via absolute mapping, the usable range limits the maximum number of items that can be selected easily [36]. Relative input allows repositioning of the feet, similar to using the mouse wheel or scrolling on a touch screen. Large trackballs were found sufficient for non-accurate spatial tasks [23]. Foot mice and relative mapping under the desk were described but not yet evaluated [3], [5], [31].
3 Design Considerations
In this work, we present methods to interact with medical images. Based on the advantages reported by literature, we focused on heel rotation and ball lifting/tapping as the input method. Unlike most of the research conducted on foot interaction so far, our approach is developed with a specific scenario in mind. Therefore, our methodology is to describe the task to be fulfilled, identify restrictions and develop suitable foot interaction techniques.
3.1 Intra-operative Interaction Task
One of the most common intra-operative interaction tasks is scrolling back and forth through a stack of images. To understand the role of this task, the medical workflow is described in the following: In minimally-invasive radiological interventions, a catheter is navigated through the patient’s blood vessels. Unfortunately, the vessels and the blood itself are effectively transparent to x-rays and can only be seen when contrast agent is administered. A sequence of x-ray images which shows the distribution of contrast agent through the blood vessels therefore holds valuable information for the physician. A “reference image” [15] or “road map” [11] in which all vessels are visible can be used to guide the next steps without the need to administer even more contrast agent. Furthermore, the distribution pattern tells the physician details about the vessel configuration, overlappings, blood flow and so on. Gathering this information might require scrolling back and forth multiple times [15]. Preoperative volume data from imaging modalities such as CT or MRI is provided as a stack of slices which can be navigated in a similar manner. As described by Mentis et al., the need for scrolling through 20 images back and forth in the middle of an intervention arises when there is a concern or problem [17]. Therefore, interacting with an image stack covers a wide range of situations and requires only one degree of freedom. Depending on the kind of data set, the number of images may differ significantly, which requires a method to allow for an arbitrary number of slices to be navigated.
3.2 Restrictions and Requirements
During radiological interventions, the physician has to operate different instruments and devices and therefore needs both hands most of the time. The feet, especially in a standing position, are used constantly to maintain a stable stance. Lifting a foot when using a foot pedal is possible but uncomfortable, since it disturbs the balance. Furthermore, there is no visual control over the pedal; it may get lost and is hard to find under the table [15], [35]. Additionally, heavy lead aprons have to be worn during radiological interventions to shield against radiation. The added weight makes it even harder to maintain unstable positions.
Van Veelen et al. aimed to improve foot pedals for surgery. In the process, they developed guidelines for the design of foot pedals by means of OR observations, questionaires, ergonomic literature study and clog measurements [32]. Their guidelines contain aspects regarding human factors as well as technical restrictions for the specific scenario they investigated. Since this research focuses on input techniques rather than a technical solution, we discarded guidelines 5 and 8. Guideline 5 gives recommendations about the external dimensions of the shoes a device should be designed for, while guideline 8 demands that the device be controllable with or without clogs. Requirements for foot interaction in the OR we derived from [32] therefore are:
The design of the input device must avoid a static standing posture.
A dorsal flexion of more than 25° to control the device is not allowed.
The force for activation must be maximum 10 N.
A frequent dorsal flexion of the foot should be avoided.
The input device must be controlled without looking at the foot pedal.
The chance of accidentally activating the wrong function must be minimal.
The chance of losing contact with the input device must be minimal.
Foot scrolling concepts for manipulating one degree of freedom via heel rotation. Discrete buttons (1) increment (a,b) or decrement (c,d) the current image as long as being stepped on. The outmost buttons (a,d) change the position by one at a rate of 0.2 seconds, the innermost (b,c) at 0.8 seconds. Foot scrolling (2) allows for continuous change of the current image every 10° by rotating the foot (e) while the foot tip is on the floor. By lifting, the foot can be repositioned without changing the current image. Step and scroll (3) combines the discrete buttons for fast rate-based input (a,d) and an area in between for foot scrolling (e).
4 Foot Interaction Concepts
In the following, we present three interaction approaches for one degree of freedom where only rotation around the heel and lifting the tip of the foot are utilized. All concepts are designed in such a way that the foot never has to be held tip-up for long periods of time. To establish consistent behavior, the mapping between the medical image stack and foot rotation is always realized in a fashion that scrolling back (decrementing) the image stack involves rotation towards the left (i. e., turning the right foot in) and scrolling forward (incrementing) is towards the right (i. e., turning the right foot out). Lifting the foot disables all virtual foot controls, but the visual representation of the foot position follows the foot, similar to hovering with a computer mouse. For simplicity, we focused on the right foot, but all the concepts could be mirrored.
4.1 Discrete Buttons
Corresponding to the finding of [36], the discrete buttons concept virtualizes buttons which are arranged around the foot in a fan-like fashion (Fig. 1.1). To make these positions easily selectable and safely distinguishable, we draw upon the reported foot angle selection error and range by [29]. Each button occupies 20°, which allows five buttons to fit in an area of 100° in front of the foot. As external rotation is preferred over internal rotation, the interaction area is set from −40° to 60°, where 0° is at a relaxed foot position pointing straight forward. Interaction is realized with a rate-based approach. To allow scrolling through image stacks of arbitrary length, two buttons for different rates of the image scrolling speed (0.8 and 0.2 seconds) are provided for each direction. The virtual buttons are activated by rotating the foot over a button and lowering it to the ground, similar to triggering a physical foot pedal. According to the principles listed above, the buttons left of the center decrement the currently shown image, and the ones on the right increment it. The center is left without function, to provide a position to put the foot at rest.
4.2 Foot Scrolling
To account for the arbitrary size of medical image data sets, foot scrolling connects to the relative input technique. By rotating the foot inwards or outwards, the current position inside the image stack is incremented or decremented every 10°, depending on the direction of movement (Fig. 1.2). At the end of the interactive area, the tip of the foot has to be lifted, rotated in the opposite direction and placed back on the floor. In contrast to the first concept, there is no dedicated area to rest the foot, since it can be rested anywhere without triggering a function.
4.3 Step and Scroll
A combination of both former concepts called step and scroll integrates rate-based and relative input. It consist of two buttons at the outermost positions of the interactive area for fast, rate-based scrolling described in the first concept (Fig. 1.3). The area in between consists of a scrollable area similar to the second concept. It is possible to switch seamlessly between both kinds of control elements, where the function activated is determined by the exact position of the cursor’s center.
5 Prototype
To investigate our proposed concepts further, a prototypical input system was created. According to the interaction concepts, the device measures two parameters: the foot orientation, and whether the tip of the foot is lifted or lowered. The setup consists of off-the-shelf sensors which are connected to a microcontroller via I2C-Bus (Fig. 2). The device is mounted on an OR-Shoe. Foot orientation is gathered by reading the Z-axis of a three-axis gyroscope (MPU-9250, InvenSense, San Jose, CA, USA). A downward-facing time-of-flight distance sensor (VL6180X, STMicroelectronics, Geneva, Switzerland) is utilized to determine the height of the foot tip in relation to the ground. Orientation and tip-to-floor distance are read by a microcontroller with an integrated Bluetooth low energy stack (RFD22102, RFduino Inc., Hermosa Beach, CA, USA) and sent to a computer. To account for different floor colors which influence the distance sensor, calibration of the system is done by setting the value read at a resting position as zero. For robust tap detection, we use a bi-level threshold with an upper threshold of 10 mm and a lower threshold of 5 mm above the initially calibrated distance to floor in a resting position. Orientation is computed by adding up the gyroscope readings. For simple sensor offset correction, the average value at a resting position over one minute was gathered and is subtracted from each gyroscope reading.
Hardware prototype mounted on an OR shoe with Velcro fastener consisting of a microcontroller with Bluetooth stack (a), gyroscope (b), distance sensor (c) and power supply (d).
Second, a medical image viewer with visual feedback for the foot position was developed using MevisLab [26] (Fig. 3). The upper part shows one image of a medical data set consisting of a series of images. The image format is the clinically common DICOM standard (Digital Imaging and Communications in Medicine). The number of the current image is shown in the lower left corner, which corresponds to common DICOM image viewers. The lower part provides visual feedback. It contains a representation of the position of the foot tip as well as the virtual input areas. The foot tip direction is displayed as a green cursor, which shrinks to 60 % of its original size when the foot is lifted. Input areas are displayed in light gray and correspond to the areas described in Fig. 1. For evaluation purposes, a line with textual instructions can be shown directly below the medical image.
Graphical user interface showing medical image data at the top and visual feedback for foot interaction at the bottom. A green cursor indicates the foot’s position and shrinks to 60 % of its size when lifting the foot.
6 Evaluation
We conducted a study to compare the performance of the concepts described in section 4 with respect to task completion times, subjective workload and user experience. Ten right-footed participants (all male) between 25 and 30 years (M = 26.2, SD = 1.8) recruited from our university took part in the study. Prior experience with foot interaction was stated as high by one participant and medium by two participants (on a 5-point Likert scale from 1 = no experience to 5 = very experienced). The remaining participants stated no experience with foot interaction (M = 1.8, SD = 1.4). Shoe size varied from 42 to 49 in EU size (M = 44.1, SD = 2.3).
The study took place in a computer laboratory. The sensor described in section 5 was mounted on a pair of OR-Shoes with Velcro fastener. A patch of linoleum flooring 170 cm × 105 cm in size was mounted on the floor to create a similar friction between shoes and floor as in the OR. In front of the flooring, a 40″ monitor with 3840 × 2160 pixel resolution was placed at a height of 140 cm (monitor center), which resembles the Large Display of an Siemens Artis angiography suite (56″, same resolution). The distance between participant and screen was approximately 100 cm but was not restricted to a specific distance.
To assess task completion times, the participant had to navigate to given target slices inside the stack. The target positions were identical for all participants and evenly distributed to require scrolling different distances in both directions. The current target slice number and whether the current task was for practice or a measured task was displayed in the graphical user interface. An audio signal indicated when to start a task. The participant conveyed verbally when the target slice was reached. Task completion time was measured between the start sound and the last sensor data that indicated foot movement before the participant signaled task completion. To assess the subjective workload we used the NASA-Task Load Index questionnaire without the weighting process, commonly referred to as Raw TLX (RTX) [9]. For user experience, the items usefulness, usability, positive and negative emotions, intention of use and overall rating from the meCUE questionnaire were employed [19].
During the study, a demographic questionnaire extended with questions about shoe size and experience with foot interaction was filled out first. After that, the participant selected the best-matching pair of shoes out of three pairs (size 41/42, 43/44, 45/46), which was then equipped with the sensor. Each participant completed a fixed series of tasks with each concept (within-subject design). The task sequence was identical for all concepts. The order of concepts was balanced over all participants.
For data collection, the first of the three interaction concepts was introduced and explained by the instructor. A practice phase consisting of five tasks was completed and followed by ten measured tasks. During all tasks, the same data set of radiological images ranging from 0 to 18 was used. Participants were asked to navigate to a specific image in a stack of medical images shown on a monitor in front of them. Before each task, they were instructed to set their foot on a central position, the system was reset to a center position and the image at position 9 in the stack was set as current image. The current image number and target image number were shown on the screen all the time. After finishing practice and measurement phase, questionnaires for subjective data were filled out by the participant. The training and data collection process was repeated for the remaining two concepts.
Summary of the test statistics for subjective workload, user experience and task completion time for concepts discrete buttons, foot scrolling and step and scroll.
| Dependent variables | df | F | t | p | sig |
|
d | Effect |
| Subjective workload | 2, 18 | 5.93 | 0.01 | * | 0.4 | large | ||
| foot scrolling vs. step and scroll | 9 | 3.67 | 0.02 | * | 1.16 | large | ||
| discrete buttons vs. step and scroll | 9 | 1.59 | 0.44 | n. s. | 0.5 | medium | ||
| foot scrolling vs. discrete buttons | 9 | 1.88 | 0.28 | n. s. | 0.6 | medium | ||
| User experience | ||||||||
| Overall | 1,49, 13,41 | 6.37 | 0.02 | * | 0.42 | large | ||
| foot scrolling vs. step and scroll | 9 | 4.99 | < 0.01 | * | 1.58 | large | ||
| discrete buttons vs. step and scroll | 9 | 2.48 | 0.11 | n. s. | 1.11 | large | ||
| foot scrolling vs. discrete buttons | 9 | 0.76 | 1.00 | n. s. | 0.24 | small | ||
| Task completion time | 2, 18 | 12.25 | < 0.01 | * | 0.58 | large | ||
| foot scrolling vs. step and scroll | 9 | 5.34 | < 0.01 | * | 1.69 | large | ||
| discrete buttons vs. step and scroll | 9 | 1.70 | 0.37 | n. s. | 0.54 | medium | ||
| foot scrolling vs. discrete buttons | 9 | 3.06 | 0.04 | * | 0.97 | large | ||
The data was analyzed by one-way ANOVA for repeated measures with three levels representing the three interaction concepts described in section 4. If the ANOVA revealed significant results, Bonferroni corrected post-hoc t-tests were performed to determine which concepts exactly differ. In addition to the average workload, RTX subscales were analyzed and reported individually, which is a common evaluation variation to pinpoint performance problems [9].
Mean results for the RTX dimensions Mental Demand (MD), Physical Demand (PD), Temporal Demand (TD), Performance (P), Effort (E) and Frustration (F) with standard error bars. (0 = low/good, 20 = high/poor).
Mean results for the task completion times (TCT) in seconds with standard error bars.
7 Results
The statistical results of the ANOVA and post-hoc tests are shown in Table 1. The ANOVA revealed significant result for task completion time. As seen from the Fig. 5 the participants could select the required images fastest with concept foot scrolling (section 4.2) (M = 4.03 s, SD = 0.88). They needed much longer with concept discrete buttons (section 4.1) (M = 5.11 s, SD = 0.88), followed by concept step and scroll (section 4.3) (M = 5.75 s, SD = 1.13). According to post-hoc tests, foot scrolling was significantly superior to both discrete buttons and step and scroll.
The results for the overall subjective workload as well as for the six RTX dimensions are presented in Fig. 4. The subjects perceived the lowest overall workload working with concept foot scrolling (M = 4.42, SD = 2.47), followed by concept discrete buttons (M = 5.40, SD = 2.96). The workload was highest by selecting 2D images with concept step and scroll (M = 6.67, SD = 3.28). According to one-way ANOVA, this difference was statistically significant. The subsequent post-hoc tests revealed that this is due to the significant difference between foot scrolling and step and scroll.
Mean results for the meCUE dimensions Usability(U), Usefulness(F), Positive emotions (PA,PD), Negative emotions (NA,ND) and Intention of use (IN) with standard error bars. (1 = strongly disagree, 7 = strongly agree).
Mean results for the meCUE overall rating with standard error bars. (−5 = bad, 5 = good).
The results for the selected meCue dimensions are presented in Fig. 6. The overall rating leads to significant ANOVA results which are presented in Fig. 7. All three concepts were rated positively. However, foot scrolling (M = 3.05, SD = 1.62) was rated slightly better than discrete buttons (M = 2.70, SD = 1.34) and much better than step and scroll (M = 1.75, SD = 1.44). The post-hoc tests indicated significant difference only for foot scrolling and step and scroll.
8 Discussion
Considering basic restrictions in the OR, we worked out different input concepts, applied them to heel rotation as the most promising input method, realized a technical prototype and evaluated the concepts in a user study. Our evaluation revealed significant excelling results for foot scrolling regarding task completion times, subjective workload and user experience. This is a surprising result, since relative input requires repeated movements, which were believed to be exhausting [31] and observed as slower in relation to rate-based approaches, especially when standing [2]. The results might be explained by experience due to the similarity to slide gestures on touch screens, in contrast to the unfamiliar setup of multiple foot pedals for fixed rates, possibly in combination with the clinically common but relatively low number of images. This explanation is supported by the fact that the RTX dimensions Mental Demand and Effort for foot scrolling are rated relatively low while Physical Demand differs less compared to the remaining concepts. Furthermore, the meCUE dimensions for positive emotions show similar ratings for all tree concepts while negative emotions are less present for foot scrolling, which might be interpreted as some kind of familiarity with this method. The concept step and scroll provoked involuntary button activation when the end of the foot scrolling area was reached, which led to worse results and higher frustration, which is shown by the high ratings for the RTX dimensions Effort in combination with Frustration.
Compared to the requirements derived from van Veelen et al. [32], our techniques fulfilled every aspect at least partly. Our design allows a dynamic posture as far as lifting the foot in the resting position does not trigger any functionality. Dorsal flexion of more than 25° is avoided because the upper threshold we used required lifting the top of the foot 10 mm, and can even be set lower under ideal conditions. Since the system uses no physical pedals, the force for activation is lower than 10 N. Frequent dorsal flexion of the foot is not completely avoided, but in respect to the functionality provided by our techniques, we used the minimally necessary number of foot lifts. Controlling the device without looking at the feet is possible due to visual feedback on the screen. This requirements is fulfilled, but might be improved by vibrotactile feedback in the future to match the sensation of touching a physical pedal. The chance of activating a function accidentally is minimal since our functions only contain two directions to navigate, which are represented by inner or outer rotation of the foot, similar to the approach van Veelen et al. presented. Losing contact with the input device is impossible since it is mounted on the shoe. In conclusion, we met the requirements for the input device with some room to improve in further iterations.
8.1 Limitations
However, the evaluation itself focused on tasks which were not integrated in a clinical workflow. From a human-computer interaction perspective, next steps are the comparison with other input methods such as hand gestures as well as clinically established methods such as verbal task delegation. Furthermore, the influence of foot interaction during delicate surgical tasks has to be investigated. Additionally, different hospital settings have to be taken into account. Even though all participants were able to perform the proposed interaction methods in the evaluated setup, hospitals might use different shoe and flooring materials, which influences the outcome. Reinschlüssel et al. reported that hospital floors prevent easy foot rotation and therefore implemented foot scrolling in a way that requires lifting the foot prior to heel rotation [25]. Depending on the kind of intervention, it can last up to several hours. Fatigue caused by foot interaction on such time scales needs to be investigated with domain experts who are used to operating conventional foot pedals in the OR on a regular basis.
Foot input might be interrupted by walking around, relaxing the foot or by using conventional foot pedals. Even though lifting and lowering the foot on the spot is possible without triggering a function, as we focused on input techniques in this work, the prototypical input system does not recognize when the user starts to walk around. Nevertheless, we believe that lifting the heel is a strong indicator for an interruption, thus a functionality for pausing the system in such a situation can easily be added. Detection of a heel lift can be done by adding a second distance sensor at the back of the shoe or by evaluating the dorsiflexion angle and the distance between the ground and the tip of the foot to recognize a complete foot lift.
9 Conclusion and Outlook
In this work, we investigated methods to navigate medical images via foot interaction while keeping the hands free. For this purpose, we analyzed a common intraoperative interaction task and identified heel rotation as suitable foot movement based on HCI literature. With these findings in mind, we developed foot interaction concepts with one degree of freedom which complies with ergonomic and security guidelines from existing work in the medical domain. We contributed the first empirical comparison of foot input techniques for heel rotation, which revealed foot scrolling as superior in terms of task completion time as well as subjective ratings. Our approaches might improve human computer interaction in the operating room as well as in various applications where hands-free interaction in a standing posture is necessary.
Since we showed that input with one degree of freedom can be provided conveniently by foot, it can be applied to a range of applications. In the operating room, additional image parameters like adjusting contrast and brightness of images, selection of data sets or additional adjustments like dimming the lights can be performed via foot. With adequate feedback and safety systems, the proposed input technique might replace existing foot pedals, providing a more integrated interface while covering common as well as new functionalities.
In the long run, a single degree of freedom is not sufficient to control the whole range of required functionalities. To overcome problems like occupied hands or noisy environments, an emerging approach is multimodal interaction. In this context, various combinations such as voice and gesture [17], gesture and foot [25] or gaze and foot [10] have been proposed. The insights this works provides allow the integration of foot interaction in more comprehensive approaches on multimodal human computer interaction. In the medical domain, such input systems would bear several advantages over todays fragmented device and interface landscape in state-of-the-art operating rooms. This idea connects to the OR.NET[1] project, which aims to provide interfaces to interconnect medical devices. A multimodal interface could easily adapt to complex and dynamic working environments or personalized workflows due to redundant controls via different input modalities. Overall, the user experience for human computer interaction in highly complex scenarios might be improved without adding additional mental workload to the expert in charge. As a next step, we therefore plan to investigate foot interaction as part of a fused multimodal interaction approach, which might incorporate input modalities such as gestures, gaze and voice.
Funding source: Bundesministerium für Bildung und Forschung
Award Identifier / Grant number: 13GW0095A
Funding statement: This work is partially funded by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung) within the STIMULATE research campus (grant number 13GW0095A).
About the authors
Benjamin Hatscher holds an MA in Interaction Design from the Magdeburg-Stendal University of Applied Sciences. Currently he is a PhD Student at the Department of Simulation and Graphics, Otto-von-Guericke University Magdeburg and part of the Computer-Assisted Surgery Group. His research focuses on multimodal human-computer interaction for the medical domain.
Maria Luz studied psychology at the Technische Universität Berlin. She received her doctoral degree at the same university. Since 2015 she has been a research assistant at the Computer-Assisted Surgery Group at Otto-von-Guericke University Magdeburg. Her research Interests include human-computer interaction.
Christian Hansen is a junior professor for Computer-Assisted Surgery at Otto-von-Guericke University Magdeburg, Germany. He holds an MSc in Computer Visualistics from the University of Magdeburg and a PhD in Computer Science from Jacobs University Bremen. His research interests include human-computer interaction, mixed reality and medical visualization.
References
[1] A. Alapetite, “Impact of noise and other factors on speech recognition in anaesthesia.” In: International journal of medical informatics 77.1 (2008), pp. 68–77.Search in Google Scholar
[2] J. Alexander, T. Han, W. Judd, P. Irani and S. Subramanian, “Putting your best foot forward: investigating real-world mappings for foot-based gestures.” In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2012, pp. 1229–1238.Search in Google Scholar
[3] R. Balakrishnan, G. Fitzmaurice, G. Kurtenbach and K. Singh, “Exploring interactive curve and surface manipulation using a bend and twist sensitive input strip.” In: Proceedings of the 1999 symposium on Interactive 3D graphics. 1999, pp. 111–118.Search in Google Scholar
[4] L. C. Ebert, G. Hatch, G. Ampanozi, M. J. Thali and S. Ross, “You can’t touch this touch-free navigation through radiological images.” In: Surgical innovation 19.3 (2012), pp. 301–307.Search in Google Scholar
[5] T. Fitzke, N. Krail, F. Kroll, L. Ohlrogge, F. Schröder, L. Spillner, A. Voll, F. Dylla, M. Herrlich and R. Malaka, “Fußbasierte Interaktion mit Computersystemen im Operationssaal.” In: CURAC (2015), pp. 49–54.Search in Google Scholar
[6] K. Fukahori, D. Sakamoto and T. Igarashi, “Exploring Subtle Foot Plantar-based Gestures with Sock-placed Pressure Sensors.” In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI’15. New York, NY, USA: ACM, 2015. pp. 3019–3028.Search in Google Scholar
[7] L. Gallo, A. P. Placitelli and M. Ciampi, “Controller-free exploration of medical image data: Experiencing the Kinect.” In: Computer-based medical systems (CBMS), 2011 24th international symposium on. 2011, pp. 1–6.Search in Google Scholar
[8] S. Grange, T. Fong and C. Baur, “M/ORIS: a medical/operating room interaction system.” In: Proceedings of the 6th international conference on Multimodal interfaces. 2004, pp. 159–166.Search in Google Scholar
[9] S. G. Hart, “NASA-task load index (NASA-TLX); 20 years later.” In: Proceedings of the human factors and ergonomics society annual meeting. Vol. 50. 2006, pp. 904–908.Search in Google Scholar
[10] B. Hatscher, M. Luz, L. E. Nacke, N. Elkmann, V. Müller and C. Hansen, “GazeTap: Towards Hands-free Interaction in the Operating Room.” In: Proceedings of the 19th ACM International Conference on Multimodal Interaction. ICMI 2017, New York, NY, USA: ACM. 2017, pp. 243–251.Search in Google Scholar
[11] J. Hettig, P. Saalfeld, M. Luz, M. Becker, M. Skalej and C. Hansen, “Comparison of gesture and conventional interaction techniques for interventional neuroradiology.” In: International Journal of Computer Assisted Radiology and Surgery (2017), pp. 1643–1653.Search in Google Scholar
[12] A. Hübler, C. Hansen, O. Beuing, M. Skalej and B. Preim, “Workflow Analysis for Interventional Neuroradiology using Frequent Pattern Mining.” In: Proceedings of the Annual Meeting of the German Society of Computer- and Robot-Assisted Surgery. Munich, 2014, pp. 165–168.Search in Google Scholar
[13] R. J. K. Jacob, Input Devices and Techniques, 1997.Search in Google Scholar
[14] S. Jalaliniya, J. Smith, M. Sousa, L. Büthe and T. Pederson, “Touch-less interaction with medical images using hand & foot gestures.” In: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. 2013, pp. 1265–1274.Search in Google Scholar
[15] R. Johnson, K. O’Hara, A. Sellen, C. Cousins and A. Criminisi, “Exploring the Potential for Touchless Interaction in Image-guided Interventional Radiology.” In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11. New York, NY, USA: ACM, 2011, pp. 3323–3332.Search in Google Scholar
[16] K. Klamka, A. Siegel, S. Vogt, F. Göbel, S. Stellmach and R. Dachselt, “Look & Pedal: Hands-free Navigation in Zoomable Information Spaces through Gaze-supported Foot Input.” In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 2015, pp. 123–130.Search in Google Scholar
[17] H. M. Mentis, K. O’Hara, G. Gonzalez, A. Sellen, R. Corish, A. Criminisi, R. Trivedi and P. Theodore, “Voice or Gesture in the Operating Room.” In: The 33rd Annual ACM Conference Extended Abstracts (B. Begole, J. Kim, K. Inkpen and W. Woo, eds.), pp. 773–780.Search in Google Scholar
[18] A. Mewes, B. Hensen, F. Wacker and C. Hansen, “Touchless interaction with software in interventional radiology and surgery: a systematic literature review.” In: International Journal of Computer Assisted Radiology and Surgery 12.2 (2017), pp. 291–305.Search in Google Scholar
[19] M. Minge, M. Thüring, I. Wagner and C. V. Kuhr, “The meCUE Questionnaire: A Modular Tool for Measuring User Experience.” In: Advances in Ergonomics Modeling, Usability & Special Populations. Springer, 2017, pp. 115–128.Search in Google Scholar
[20] N. Bizzotto, A. Costanzo and L. Bizzotto, “Leap motion gesture control with OsiriX in the operating room to control imaging: first experiences during live surgery.” In: Surgical innovation 1 (2014), p. 2.Search in Google Scholar
[21] B. C. Odisio and M. J. Wallace, “Image-guided interventions in oncology.” In: Surgical Oncology Clinics 23.4 (2014), pp. 937–955.Search in Google Scholar
[22] K. O’Hara, G. Gonzalez, A. Sellen, G. Penney, A. Varnavas, H. Mentis, A. Criminisi, R. Corish, M. Rouncefield, N. Dastur and et al., “Touchless interaction in surgery.” In: Communications of the ACM 57.1 (2014), pp. 70–77.Search in Google Scholar
[23] T. Pakkanen and R. Raisamo, “Appropriateness of foot interaction for non-accurate spatial tasks.” In: CHI’04 extended abstracts on Human factors in computing systems. 2004, pp. 1123–1126.Search in Google Scholar
[24] G. Pearson and M. Weiser, “Of moles and men: the design of foot controls for workstations.” In: ACM SIGCHI Bulletin. Vol. 17. 1986, pp. 333–339.Search in Google Scholar
[25] A. V. Reinschluessel, J. Teuber, M. Herrlich, J. Bissel, M. van Eikeren, J. Ganser, F. Koeller, F. Kollasch, T. Mildner, L. Raimondo and et al., “Virtual Reality for User-Centered Design and Evaluation of Touch-free Interaction Techniques for Navigating Medical Images in the Operating Room.” In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2017, pp. 2001–2009.Search in Google Scholar
[26] F. Ritter, T. Boskamp, A. Homeyer, H. Laue, M. Schwier, F. Link and H.-O. Peitgen, “Medical image analysis.” In: IEEE pulse. 2.6 (2011), pp. 60–70.Search in Google Scholar
[27] E. Sandberg-Diment, “A New ‘Mouse’ Resides on the Floor.” In: The NY Times (1985), Y18.Search in Google Scholar
[28] N. Sangsuriyachot and M. Sugimoto, “Novel interaction techniques based on a combination of hand and foot gestures in tabletop environments.” In: Proceedings of the 10th asia pacific conference on Computer human interaction. 2012, pp. 21–28.Search in Google Scholar
[29] J. Scott, D. Dearman, K. Yatani and K. N. Truong, “Sensing foot gestures from the pocket.” In: Proceedings of the 23nd annual ACM symposium on User interface software and technology. 2010, pp. 199–208.Search in Google Scholar
[30] S. Shimizu, K. Kondo, T. Yamazaki, H. Koizumi, T. Miyazaki, S. Osawa, T. Sagiuchi, K. Nakayama, I. Yamamoto and K. Fujii, “Hanging Foot Switch for Bipolar Forceps: A Device for Surgeons Operating in the Standing Position.” In: Neurologia medico-chirurgica 53.1 (2013), pp. 53–55.Search in Google Scholar
[31] A. L. Simeone, E. Velloso, J. Alexander and H. Gellersen, “Feet movement in desktop 3D interaction.” In: 2014 IEEE Symposium on 3D User Interfaces (3DUI). 2014, pp. 71–74.Search in Google Scholar
[32] M. A. van Veelen, C. J. Snijders, E. van Leeuwen, R. H. M. Goossens and G. Kazemier, “Improvement of foot pedals used during surgery based on new ergonomic guidelines.” In: Surgical Endoscopy And Other Interventional Techniques 17.7 (2003), pp. 1086–1091.Search in Google Scholar
[33] E. Velloso, J. Alexander, A. Bulling and H. Gellersen, “Interactions Under the Desk: A Characterisation of Foot Movements for Input in a Seated Position.” In: Human-Computer Interaction – INTERACT 2015, Vol. 9296. Lecture Notes in Computer Science. Cham: Springer International Publishing. 2015, pp. 384–401.Search in Google Scholar
[34] E. Velloso, D. Schmidt, J. Alexander, H. Gellersen and A. Bulling, “The Feet in Human–Computer Interaction: A Survey of Foot-Based Interaction.” In: ACM Computing Surveys (CSUR) 48.2 (2015), p. 21.Search in Google Scholar
[35] L. S. Wauben, M. A. van Veelen, D. Gossot and R. H. M. Goossens, “Application of ergonomic guidelines during minimally invasive surgery: a questionnaire survey of 284 surgeons.” In: Surgical Endoscopy And Other Interventional Techniques 20.8 (2006), pp. 1268–1274.Search in Google Scholar
[36] K. Zhong, F. Tian and H. Wang, “Foot menu: Using heel rotation information for menu selection.” In: 2011 15th Annual International Symposium on Wearable Computers (ISWC). 2011, pp. 115–116.Search in Google Scholar
© 2018 Walter de Gruyter GmbH, Berlin/Boston
