Keywords

1 Introduction

Internet usage has increased rapidly in recent years. Only 37 % of Americans had access to the Internet in 2000; however by 2013, 72 % of Americans could claim the privilege [1]. As Internet-use becomes more common, the number of visually impaired people who want to use the web is growing steadily. Because electronic information is primarily conveyed through the visual modality, the low-vision population often runs into difficulties interacting with graphical user interfaces. In 2012, the population who reported experiencing vision loss included 20.6 million adult Americans aged 18 and over [2]. As visual problems tend to increase with age [3], difficulty in accessing online materials can deter older users from using the Internet.

Low vision has been defined as any visual impairment that cannot be corrected by eyewear, medication, or surgery [4]. Because these individuals must overcome perceptual limitations, how information is displayed is of particular importance. Keeping users with visual impairments and other disabilities in mind, various groups have since come together to make the web more accessible. In 1997, the World Wide Web Consortium (W3C), an international standards organization for the Internet, launched the Web Accessibility Initiative (WAI), through which members of the W3C undertook the responsibility of improving the accessibility of the World Wide Web for people with disabilities. These guidelines support technical accessibility by ensuring a webpage is constructed in such a way to accommodate for assistive tools, two of which include screen magnifiers and responsive web design (RWD).

1.1 Screen Magnifiers

The most popular assistive technology used by people with low vision to interact with visual computer displays has traditionally been the screen magnifier [5]. Screen magnifiers essentially enlarge a portion of the application screen in the same way as an ordinary magnifying glass. The enlarged portion is typically the region around the cursor, since this is the user’s point of focus. However screen magnifier users must contend with a number of trade-offs in the process.

One such trade-off is that as the magnification level increases, the amount of contained information on the viewing window decreases [6]. In other words, users with greater impairment and a need for more extreme magnification will see less content on the screen at once, thus forcing them to scroll back-and-forth much more frequently. Low vision computer-users report that following a cursor to read is a visually taxing task [7].

Another trade-off when using screen magnifiers arises from the loss of spatial orientation. The most commonly used magnification mode is full screen, which provides the user with the largest magnified visual field but sacrifices contextual awareness and spatial orientation [8]. As magnification increases in this mode, the amount of original content that fits on the page decreases, which results in a “lack of page context” [9] and inhibits access to global information. Such a trade-off decreases the user’s effectiveness in skimming text to find information [10] and inputting information on an e-commerce website [7].

Additionally, people with low-vision should not be prevented from computer use by the high cost commonly associated with assistive technology. This comes as a problem considering that an estimated 90 % of the visually impaired population resides in low-income areas [11]. Despite the inherent issues faced by low-vision computer users when working with screen magnifiers, they must also face the huge price tag on these software packages.

1.2 Responsive Web Design (RWD)

In the past ten years, the booming smart phone industry has offered new platforms for Internet browsing. As a result, Ethan Marcotte [12] proposed RWD to accommodate for the number of platforms and to decrease the workload put out by web developers. Through web design, the content of application windows can wrap in such a way to present optimal presentation, regardless of the size of the device screen or magnification level.

Although not all responsive web sites are necessarily accessible, the principles of RWD promote accessibility. In a survey aimed to understand how people with low vision interact with computers to read content, 27 % of responders said they use the enlargement capabilities within the browser [13], a capability that RWD has direct implications for. Unfortunately many web developers have not adopted RWD. Integrating fluid layouts is an important consideration for professional designers who want their sites to be adaptive, accessible, and equally usable on a variety of platforms.

1.3 Present Study

The purpose of the present study is to compare the usability of a screen magnifier with RWD through efficiency and accuracy measures across different tasks web–based tasks. For one task, participants were asked to read a short or long passage and answer questions designed to assess their comprehension of the material. In another task, participants were asked to complete a data input form, simulating a payment screen on an e-commerce website. These tasks were chosen, as they simulate commonly performed web-based tasks. Participants completed these tasks using both a screen magnifier and RWD. Accuracy, time-on-task, and subjective usability assessments were measured from all tasks. Due to the increased action of scrolling horizontally, we predicted that participants would be less accurate and less efficient when using a screen magnifier for all tasks, as the additional action of scrolling will decrease the amount of resources normally used for comprehension. However similar performance measurements across conditions could indicate that the act of scrolling horizontally had no impact on comprehension thus demonstrating negligible differences in the two magnification methods.

2 Method

2.1 Participants

Sixteen California State University, Long Beach students (12 female, 4 male, Mage = 23.28 years, age range = 19–27 years) participated in the current study. They were compensated $25 for their time (less than 2 h). All participants had normal or corrected-to-normal vision. To simulate a low-vision environment, a tablet was moved back to the participant’s reported visual acuity limit (Macuity limit = 1322 mm, range = 920–1600 mm), so that content was barely discernable but not readable. The rationale behind testing participants with normal or corrected-to-normal vision was to test the study platform and demonstrate that good assistive technologies also benefit typical users under constrained situations (e.g., reading in low light conditions; viewing web content on a small device). All participants reported having experience with a smart phone or tablet, and 93.75 % reported using these devices to read.

2.2 Design

Participants performed a series of tasks using one of two magnification techniques. They were to use either RWD, in which the text wrapped to the size of the screen as magnification increased, or a screen magnifier, in which one portion of the screen magnified. The order of each magnification technique was counterbalanced across participants, and the tasks were counterbalanced within each condition block.

From each task, we measured accuracy by dividing the number of correct responses by the total number of possible responses. We also recorded the time it took for participants to work through the task (time-on-task) to measure task efficiency. At the end of each condition, participants completed a System Usability Scale (SUS) [14] to capture participants’ subjective usability assessment. SUS is a brief questionnaire that asks responders to rate their level agreement to ten different statements. Responses fall in a range of 1–7, with 7 denoting a higher level of agreement with the statements. After calculation, final SUS scores fall between 0 and 100, with scores above 70 indicating acceptable usability. Scores between 50 and 70 indicate moderate usability, while scores below 50 indicate the system as unusable.

To further capture subjective user-experience, we also measured the expectancy of ease of task. Prior to each task, participants were asked to rate how easy they expected the task to be on a 1–7 point scale, with higher numbers denoting an easier expectation. After each task, participants were asked to rate how easy the task actually was, using the same rating scale. The expectancy of ease of task was calculated by subtracting the first rating score from the second. Negative numbers indicate that the task was harder than expected, while positive numbers indicate that the task was easier than expected.

2.3 Materials

All tasks were completed on a Dell Venue 8 tablet. For the testing station, the tablet was nested in a stand, with the display at eye-level. The stand started 457.2 mm from the table that the participant was seated at. To simulate mild low vision, the stand was moved back until the participant could no longer read the content on the page. This produced an average visual angle of 0.06 [15] for the original sized-content. An enlargement of 200 % produced a mean visual angle of 0.12, while an enlargement of 300 % produced an average visual angle of 0.17. The keyboard and mouse, which were connected to the tablet wirelessly, rested upon the table in front of the participant. A Cannon video camera recorded the screen of the tablet. This camera was positioned on a stand and was occluded from the participant’s line of vision so as to not be a distraction.

Participants were given a total of six tasks to complete for each condition, but the present paper will only focus on three of them. Two tasks were reading comprehension passages from an SAT Practice Test book published by the College Board [16]. The passages differed in length, so that participants completed a short passage with questions and a long passage with questions. The third task was a data input task. Using a fictitious person’s mailing address and billing information, participants were instructed to complete a page similar to an e-commerce website. The other tasks, which are not part of the present paper, were two math tasks and a proofing task.

The tasks were displayed in webpages that were presented through Mozilla web browser. The text was Verdana in 12-point font. RWD was programmed within the web browser to adjust according to the number of ems per line using media queries with relative units. For the screen magnification condition, we used Ai Squared’s ZoomText, as that is one of the most popular screen magnifier currently used today. Participants were provided with a list of hotkeys for each magnification method at the beginning of each condition.

2.4 Procedure

After completing a consent form and demographics questionnaire, participants were asked to sit at the testing station. The tablet displayed an Orientation Page, which consisted of a passage from a short novel. A researcher moved the stand with the tablet away from the participant until the participant reported no longer being able to decipher the passage on the page. The researcher then provided a brief explanation of each hotkey required for the condition, while the participant followed along by using the keys. Once he or she felt comfortable with the hotkeys, the participant was handed instructions for the first task to read aloud. The participant then rated how difficult he or she expected the task to be. At this point, the participant pressed “Next” to begin the task, and the researcher stepped out of the room. Once the participant completed the task, he or she was instructed to turn over the instruction sheet and rate how difficult the task actually was. This process was repeated for each task. At the end of each condition, the participant completed a SUS for the magnification method. A break was given after the first condition. Upon completion of the second condition, the participant was asked for any longstanding reactions and was debriefed upon the purpose of the study.

3 Results

3.1 Reading Comprehension

Time-on-Task. A 2 (magnification technique: screen magnification /responsive design) × 2 (length: short /long) within-subjects ANOVA was conducted. Results indicated a significant main effect of magnification technique on time-on-task, F(1, 15) = 9.35, p = .008, n 2 = .384, such that when participants were performing the reading task, they were significantly quicker when using responsive design (M = 344.55, SE = 24.10) than a screen magnifier (M = 435.67, SE = 28.42). A significant main effect for length was also found, F(1, 15) = 435.77, p < .001, n 2 = .967, such that participants took less time completing the reading task with a short passage (M = 185.48, SE = 140.50) than a long passage (M = 594.73, SE = 30.65).

Accuracy. A 2 (magnification technique: screen magnification /responsive design) × 2 (type: short /long) within-subjects ANOVA was conducted. Results indicated a significant main effect of magnification technique on accuracy, F(1, 15) = 5.58, p = .032, n 2 = .271, such that participants answered a higher proportion of questions correctly when using responsive design (M = .74, SE = .054) than a screen magnifier (M = .57, SE = .072). No difference in accuracy was found between the types of reading passages, F < 1.0 (Fig. 1).

Fig. 1.
figure 1

Time-on-task (left) and accuracy (right) for reading comprehension. As shown in the figure, participants took longer to finish the task and were less accurate when using a screen magnifier.

Full size image

Expectancy of Easy of Task. To investigate the difference in expectancy of ease of task ratings, a 2 (magnification technique: screen magnification /responsive design) × 2 (type: short /long) within-subjects ANOVA was conducted. Results indicated a significant main effect of magnification technique, F(1, 15) = 13.97, p = .002, n 2 = .482, such that participants rated the reading task to be significantly harder than originally expected when using a screen magnifier (M = -1.06, SD = 1.08) than responsive design (M = -0.16, SD = 1.37). No difference was found between the types of reading passages.

3.2 Data Input

Time-on-Task. A one-way ANOVA was conducted on time-on-task, with magnification technique as the within subjects variable. Results indicated that participants were significantly faster to complete the data input task when using responsive design (M = 138.25, SD = 99.32) than a screen magnifier (M = 159.72, SE = 10.25), F(1, 15) = 5.27, p = .036, n 2 = .26.

Accuracy. A one-way ANOVA was conducted on accuracy, with magnification technique as the within-subjects variable. No effects were significant, F < 1.0 (Fig. 2).

Fig. 2.
figure 2

Time-on-task (left) and accuracy (right) for the data input task. As shown in the figure, participants took longer to finish the task when using a screen magnifier. However accuracy was not affected by magnification method.

Full size image

Expectancy of Easy of Task. A one-way ANOVA was conducted on expectancy of ease of task, with magnification technique as the within-subjects variable. A main effect of magnification type was found to be significant, F = 4.84, p = .044, n 2 = .244, such that participants rated data input as significantly easier than originally expected when using responsive design (M = .50, SD = 0.97) than when using a screen magnifier (M = -0.19, SD = 1.05).

3.3 SUS

Participants rated responsive design (M = 92.81, SE = 2.16) as significantly more usable than a screen magnifier (M = 67.66, SE = 4.72), F(1, 15) = 25.81, p < .001, n 2 = .63.

4 Discussion

The purpose of the present study was to investigate the usability of magnification methods through the comparison of a screen magnifier and RWD. There was a difference in performance between the two methods, thus indicating that the manner in which information is displayed for accessibility has an impact on comprehension tasks, even for participants with normal vision.

The results for the reading tasks showed that for both short and long passages, participants were not only slower but also less accurate while using a screen magnifier than RWD. According to the expectancy questionnaires, reading was much more difficult than originally expected while using a screen magnifier. During debriefing, many of the participants explained that the additional action of scrolling horizontally impeded upon comprehension. Participants were forced to reread lines and move back-and-forth repeatedly. This indicates that because less information could be displayed on the enlarged screen, participants were forced to rely more heavily on their working memory then when the information was displayed within the boundaries of the screen. Participants often reported feeling fatigued and complained of eyestrain, particularly after completing the long reading passage.

Results from the data input tasks did not show a difference in errors committed between the magnification techniques, thus indicating that participants were just as accurate when inputting information across both methods. However participants were slower to complete the task while using a screen magnifier compared to RWD. The increased time-on-task did not come from navigation style, as participants reported using the “tab” key to jump from one box to the next in both conditions. More likely, the efficiency of the participants’ performance was degraded in the screen magnifier condition by the inability to predict what information would be needed next, as not all input boxes were visible, and participants would need to move their mouse back-and-forth to view all boxes and check that they were indeed filled in. Furthermore, participants needed to reorient themselves once the magnified portion switched locations after the “tab” key was used. Participants reported not feeling sure that all information had been inputted, an implication for e-commerce websites in general, especially with security measures like time limits.

Ultimately the results of this study have several implications for accessibility standards in web design. Screen magnifiers have inherent usability problems, such as horizontal scrolling, that impair efficient performance, and so should not be the sole assistive tool that low vision users rely on. Second, the findings suggest an error in the classification of accessibility levels in the WCAG 2.0 Guideline 1.4. In particular, success criterion (SC) 1.4.4, conformance level AA, requires that if text can be enlarged up to 200 %, but does not wrap to the size of the device screen, it still conforms to the WCAG 2.0 [17]. According to this criterion, if text does not fit within the device window, scrollbars should be provided. While this solution has been argued to be inferior to the ZoomText magnification provided in the current study, it is still considered sufficient to meet the criterion. Success Criteria 1.4.8, conformance Level AAA (part 5) however requires that text enlargement should be wrapped within the device window. This experience was provided by RWD within the current study. As it stands, level AA is legally binding, while Level AAA is voluntary. Thus WCAG 2.0 Level AA excludes enlargement with reflow as a legally binding accommodation for reading with low vision.

The decrease in efficiency and accuracy while using screen magnifiers indicate a cost in reading while scrolling back-and-forth. This cost was seen with a population that does not struggle with additional perceptual limitations, such as blurred vision, sensitivity to contrast or brightness, or loss of central vision. Horizontal scrolling has been deemed a “web design error” [18] and is one of the few features that users consistently report negatively to [19]. So the question remains why popular assistive technologies subject people with low vision to such demanding conditions. Should we not allow computer access to everyone, and make assistive technologies, which some rely on, as easy-to-use as possible?

To further provide support for changes in accessibility guidelines, this study must be done using the affected population, and indeed, follow-up studies are currently in place to investigate the effect of magnification methods on the performance of participants with low vision. While this study focuses on word-wrapping and horizontal scrolling, still much more should be done to fully understand how computer displays are accessed and perceived by individuals with visual impairments. We can no longer focus on the average, normal-sighted user but must also consider those with disabilities and not exclude the 6.6 million individuals with low vision [20].