Keywords

1 Introduction

The iPad is closing in on a decade of incorporation and use in multiple phases of K12 and Higher Education (HE). It’s acknowledged potential to transform education by promoting motivation, engagement and deep learning, is stated in numerous studies [2,3,4,5,6,7,8]. Furthermore, its role in the initiation and proliferation of the ubiquitous learning paradigm, is evident [9, 10]; that is flexible, autonomous, student-managed, situated learning (where applicable) without restrictions of time, place, load or status [11].

Although investigation in its role in various disciplines (mainly in STEM, Humanities, Social and Applied sciences) is widespread, more vertical and exhaustive research on the interaction between device and learners, using collaborative software applications that are characteristic of a particular domain, is still limited; this creates a gap of specialized implications for designers and educators. Adding to this gap are additional parameters which are inherent in today’s complex learning contexts: physical or digital, co-located, remote or blended/hybrid, synchronous or asynchronous.

In this paper, we seek to examine the role of the iPad in supporting both co-located and distant Computer Supported Collaborative Learning (CSCL) in a HE UX Design module, during the early design stages of sitemapping and mind-mapping of concepts. By observing the collaborative activity and investigating the perceived outcomes (experience and learning) of groups of student dyads, we seek to outline the implications for instructional designers and educators and propose ways to support the implementation of m-learning in similar design topics.

We are primarily interested in the utility and usability of the iPad as a device embedded in the social learning process, yet we hypothesize that the perceived experience of hardware and software is intertwined and often difficult to isolate. Through comparison with a control group (laptop users), we anticipate to extract those outcomes that are specific to the use of the device.

This research is guided by the following questions:

  1. 1.

    What is the role and contribution of the iPad to the experience and learning outcomes of co-located and remote groups of students in the design field?

  2. 2.

    How do the iPad’s affordances affect the social learning process in the design field?

In the following sections, we provide related research concerning the use of iPads in CSCL, describe the research context, explain the methodology employed for data collection and analysis, outline and discuss the findings as well as propose instructional guidelines for m-learning in design disciplines.

2 Related Work

Portability, a concept that is important in modern-day education [12], is the iPad’s primary asset [13] which supports ubiquitous and situated learning in multiple contexts (lecture hall, laboratory, home, office/workplace, outdoors, public places) [9, 14], in different conditions such as formal (lectures, workshops, meetings), and informal social (coffee, gatherings, travelling, events) learning settings [11, 15], and in different times and states (day/night, on the move, at leisure, at rest etc.).

Up to date, the iPad has been incorporated in various educational fields such as STEM [16,17,18], architecture, art, design [19,20,21,22], language learning [15, 23], health and physical education [24, 25] and many more. Amongst others, they have been used for the purposes of information seeking, research and analysis, resource management and sharing digital content generation, presentation and assessment [9, 22] and have been particularly important in Computer Supported Collaborative Learning (CSCL) contexts [26,27,28].

We haven’t yet seen studies focusing on the area of UX design, a socially responsible design philosophy, which focuses on empathizing with users in the development of products that support users and have implicit or explicit social implications [71]. This study focuses on the early design stages of analysis and prototyping, that amongst others, involve collective externalizations, mind maps and sitemaps, as part of the conceptual planning and the information (application) architecture. Subsequent stages, such as prototyping, are investigated in a few studies only, which primarily look at collaborative sketching through the use of multiple devices [29, 30]. However, substantial research on the role of m-learning in design education, especially in the creation of social design artefacts, compared to the spread and importance of this field, is scarce.

Mind mapping, is described as memorizing, accessing, processing, connecting and outputting information stored in our brains “in a visually expressive way” [31,32,33]. In team-driven terms, it is a visual form of brainstorming, which “serves as an extension of the mind” [34]. Karakaya and Demirkan [35] state its importance by reporting an increasingly strong focus on such processes in the digital creative industries domain today.

Sitemapping is similar in the sense that it externalizes information but it has a different goal; that is to organize and structure information to reflect the hierarchical flow of a digital application [36]. This is typically initiated on paper, using pens and markers, through the contribution of one or all team members; the iPad has been known to be an effective alternative to the paper and pen media [37], yet, to the best of our knowledge, no site or mind-mapping-specific studies have been reported up-to-date.

Collaboration is a vital prerequisite for the generation of sitemaps and mindmaps and can be realized in co-located environments, remotely or in a blended setting, that is a mixture of both modalities [38,39,40].

The advantages of face-to-face collaboration are indisputably notable, as proximity facilitates “interpersonal interaction and awareness” especially in small groups [41]. Kraut et al. [42] for example, report that pairs of researchers accommodated in near-by offices, are four times more likely to produce published research collaboratively than in remote teams. Technology, can augment this process further, for both co-located and remote collaborative teams, through the synergistic use of PCs (desktops, laptops), tabletop surfaces, wall-mounted displays, motion-sensing systems and handhelds [43,44,45]. Interestingly, despite being primarily intended for personal use, handhelds may reportedly encourage better face-to-face collaboration, since they are smaller and easier to handle in close proximity, than PCs [46]. Even so, they do invite concerns about visibility due to screen size, angle and distance as well as restrictions in “multi-touch interactions” that may lead to user redundancy [47].

These devices’ contribution in the interaction of geographically distributed teams of learners, teachers and other stakeholders, cannot be overlooked [15, 34, 48]. Mobile devices are unique in facilitating synchronous collective work, on shared artefacts, strengthening flexibility through the “mobility of learning and the mobility of learners” [10]. The sum of the hardware features and online multi-user apps, can augment learning by transferring it outside of traditional classrooms [26], if only designed effectively. Falloon [49] notes that the design and content of apps can heavily impact the overall user experience: the absence of provisions for distant communication & coordination, material organization, negotiation, interactivity, mobility and peer awareness at the software-level, can severely hinder the perceived quality of learning with technology [50,51,52]. Early and iterative formative usability evaluation of these, as well as their interaction with specialized hardware, is thus essential for designing successful instructional interventions whose outcomes depend on such synergies [38].

3 Method

3.1 Study

Eleven (11) out of 22 second-year multimedia students were provided an iPad each to complete assignments collaboratively throughout weeks two to four of the semester, for a unit of study (13-week UX design module). All students had used the iPads extensively in their first-year modules. The rest of the students were asked to collaborate using their laptops or desktops in the lab.

Following a short lecture session in class, each dyad used a single iPad or laptop, to create an account and work on building a sitemap for a website, using Lucid Chart sitemap builder app. Students were also provided a printout of a step-by-step procedure of (a) the deliverables for each of the following weeks and (b) a brief software manual and sharing-setup guide.

At the end of the lesson, the owner of the sitemap file in each dyad invited their partners to register, in order to continue collaborating on the project remotely, in – ideally - a synchronous manner, to complete the assignment. Since all of the students had Gmail accounts, all teams were prompted to use Google Hangouts (already installed on the iPads) for communication and coordination during the process. Following the iPads’ return back to the instructor on the following week, the students participated in an online survey and a focus-group session.

In lesson/week four, students were assigned a semester-long group project – that was to re-design the Cyprus Red Cross Society website with an aim to enhance the experience of users who visit with different backgrounds, purpose, expectations as well as limitations. The project required them to collaborate on building a mind-map, as part of the initial brainstorming phase. An alternative software, Coggle (freeware mind mapping app), was used in place of Lucid Chart for this assignment, following student reports of interface and resource limitations of the former. While still in the same teams as week two, the remaining half of students, got hold of the iPads and worked jointly on their mind-maps in class, using a single device. Like week two, they were asked to share their mind-map and continue collaborating from home, returned the devices on the following week, and completed a second survey and focus group session.

3.2 Analysis

This study uses the ACAD (Activity Centered Analysis and Design) framework for the design and analysis of the collaborative network, process and outcomes. ACAD is based on the notion that collaborative learning is activity which is “physically, socially and epistemically situated” [53]. More particularly, it seeks to investigate a learning network and the interactions that occur within that, rather than a single device or a tool. Instead, multi-faceted assemblies are taken into account, planned for, captured and analyzed: components such as the physical or digital, blended/hybrid learning spaces, the social dynamics and the activities that are shaped based on the synergies between them, form the units of research [54].

Likewise, in this work, the facets of these components refer to a multi-dimensional Set, that is the equipment, materials and resources used, on physical and digital levels, the Social, that is the group structure and particular work arrangements i.e. divisions of labor and interaction amongst the members, and the Epistemic, that is the assignments that are supposed to generate knowledge outcomes, in this case, the sitemapping and mind mapping subjects. The instructional design should be flexible in allowing changes that could originate both from the outside (top-level action – i.e. instructional designers, instructor) and from within (participatory action i.e. the group members), what ACAD proposes as the Co-configuration component. These components are entangled and overlap one another, in practice. Isolated analysis is therefore feasible on a theoretical - rather than on a practical level.

As we are primarily investigating the usability of the iPad and its effects on learning in a domain-specific context, to further analyze the Set component (the digital in this case), we draw from five primary usability principles for the design and evaluation of products, systems or services [55,56,57]. These are: (a) Learnability – the ease by which novices become accustomed to the system (b) Memorability – experience of the system’s functionality by users who haven’t used it for some time, (c) Efficiency – the speed and level of accuracy and completeness achieved by users reaching their goals, (d) Satisfaction – the emotional or physiological attitude of users during the use of the system and (e) Error incidence and recovery – the occurrence of errors and the ease by which users regain control. Memorability did not apply in this case, due to the uninterrupted use of the device throughout the study. The abovementioned principles were explored, largely based on the general user perception of the system as a whole, rather than the particular device per-se, for two reasons:

  1. (a)

    We were aware of the difficulty in eliciting explicit outcomes for hardware and software separately, as these are rarely understood in isolation

  2. (b)

    We could retrieve such findings indirectly through comparison between the two groups from both the quantitative and qualitative data collected.

4 Results

In this work, we triangulate the results from (a) two questionnaires employing 5-point-likert scale questions (1 = highest and 5 = lowest) and open-ended comments and (b) four focus groups (two for each device groups) carried out in weeks three and five of the semester.

Quantitative data was not normally distributed. Non-parametric tests, such as the Wilcoxon signed-rank test [58] were thus used to conclude on significant statistical differences between the two groups and between in-class and remote conditions. Findings are analyzed according to the four components of the ACAD framework.

4.1 Set Component: Usability Factors

Learnability.

The iPad cohort reported significantly longer in-class learnability times than in remote settings following a within-groups comparison test (Ζ = −2.111, p = .035). This was admittedly an anticipated effect, for both iPad and Laptop groups, as the learning process for the particular task was instigated during the lesson and progressed outside class, producing a decreasing learning curve over time. The laptop group, however, did not report such results, due to a greater level of familiarity with their device. This outcome disputes the “duality” of hardware and software in user experience, as the two are typically perceived as one - acting in favor or against each other, depending on the case [59, 60]. In this case, hardware unfamiliarity transferred onto and hindered the software experience.

Efficiency.

Device-specific differences were detected, regarding the perceived effectiveness in creating the sitemaps/mind-maps, in the classroom, weighing negatively for iPad users (Ζ = −2.449, p = .014) (Table 1). Such were not recorded for out-of-class settings. Coinciding with results from the previous section, disparate in-and out-of-class outcomes can be connected and attributed to the gradually lessening residuals of the learnability process.

Table 1. iPad versus PC in-class and remote usability results
Full size table

Further investigation into efficiency indicated that iPad users agreed to needing significantly more time (M = 1.73, SD = 1.1) to resolve the exercises than those who worked on laptops (M = 4, SD = 1.6), a phenomenon recorded more prominently during the first cycle of the experiment, weeks two to three (Ζ = −2.061, p = .039).

Interestingly, the effect was minimized when the second group took hold of the iPads in week four as most of the students appeared undecided - yet - not negative - in this question. This indicates that since the second group had the chance to familiarize themselves - not with the device - but with the software instead, reinforces the point that hardware and software usability was not perceived in isolation. In other words, in this case the experience gained in using the software tool for two weeks, affected the perceived iPad usability in the second group of users.

Error Incidence.

Although the two groups had similar quantitative responses concerning the errors encountered, iPad users mentioned a wider range than those of laptop users.

Open-ended comments referring to in-class use fixated on software issues - such as problematic user interface affordances which were amalgamated with hardware affordances (i.e. the touch interface). In the case of remote use, adding to the poor interface affordances, connectivity issues caused delays and prolonged response times: unsynchronized activity caused confusion and insecurity in users who commented that they were ‘blind’ as to their peers’ contributions (edits) in the shared workspace:

“There was no indication as to what my partner was doing and I didn’t know if I should add new elements because maybe they would end up appearing twice… I didn’t know what was old or new…”

Expectations were set, based on the functionality provided by such synchronous collaborative tools and when these failed to perform as anticipated, students were confused and felt doubtful as to their own and peers’ actions. Collaboration compromised due to lack of timely response and feedback, is not new in CSCL. Users cannot evaluate their activities against those of others and therefore ‘awareness information’ becomes insufficient in these shared spaces [51, 61].

Although mentioned by both groups, such problems were heightened in using the iPad’, caused by parallel delays of failed struggles for accurate onscreen actions - especially in the sitemap design case; erratic selection, accidental activation, unsatisfactory target selection precision, also dubbed ‘read-to-tap asymmetry’ phenomenon [62], failure to draw straight lines or to align elements, plus the constant pinching and spreading to zoom in for detailed views [63], further compromised the poor experience due to connectivity issues:

“But I prefer it (laptop) because I can point somewhere with the mouse and I don’t have to zoom in and out …”

“…with the touch… it is extremely sensitive, you might touch something and something else gets activated on the sitemap…”

“…you couldn’t easily select the small lines so you can play with them …it was annoying”.

Difficulties with sharing content between linked accounts also surfaced, and together with aforementioned issues, lead to a stronger preference towards co-located collaboration, especially in the case of iPad users.

User Experience.

Outcomes of the study in relation to user experience indicated no significant differences between the two devices, for both in class (Ζ = −1.469, p = .142) and remote collaboration (Ζ = −1.933, p = .053) (Table 1), although use of the iPad was more adverse in the case of remote settings due to the extent of problems described in the previous section. When probed whether they would use - or not - the devices to work on similar assignments in the future, a question directed to iPad users, the responses were either neutral or negative: students were not sure or tended to disagree with the prospect of using it again for future projects.

4.2 Social Component

According to qualitative outcomes, the predominant choice of working mode for both groups, was co-located, rather than remotely. However, the tendency to continue this mode of collaboration for learning in more social, out-of-class setting was only observed in the iPad group’s case.

The majority of participants commented that should circumstances allow it, they would rather work with their peers and “resolve problems there and then” than remotely. In their point of view, distant collaboration was helpful but should be utilized only if a face-to-face alternative was not feasible. For them it was important to be proximally close, while having the chance to talk and elaborate on their opinions, “think together”, make mistakes together, verify correctness and reach consensus for solutions collectively.

“… when you are together, you are close, you think together, you tell one another…”

Literature also refers to the advantages of co-located collaboration, due to proximity allowing for real social interaction and contextual understanding [41] the unrestricted articulation of comments, questions, negotiation over common goals [64], and prevention of challenges which often affect remote settings [48, 65]. However, when it comes to iPads, this was a surprising outcome. Working with a small screen, excludes everyone but one user from operating with the device while the rest typically observe or provide verbal feedback in a co-located setting [37]. Yet that did not seem to have a negative impact on the social aspect of learning.

To confirm this finding, the iPad group added that not only did they prefer class-based face-to-face collaboration but also transferred this working mode to other informal environments, such as cafés, outdoors and in unconventional physical settings such as on the move or at rest:

“…an example: I am working with C. in the park cause I like to do so – being outside in the open – and as I walk we talk about this and at the same time we are watching it on the iPad… but with the laptop, I need to plug it in…. you need an office to do that…”

“…you can both lie down and work on it…”, “…Its far more restful…”, “…you feel more comfortable with it…”.

Although, no statistical differences were recorded between the two groups with regards to informal learning, qualitative data ascertains that the device’s distinctive portability traits [37] can foster learning through extended social interaction beyond the classroom. That, in effect, prolongs group interaction through discourse, reciprocal decision-making and negotiation, all of which contribute towards better learning [66].

Agreeing with previous literature, negative feedback for co-located collaboration stemmed from delays, superfluous conversations and other sidetracking activities both in and out of class [67].

4.3 Epistemic Component

Epistemic refers to the activities or results that are closely linked to learning. Quantitative results revealed significant differences in the students’ perceived learning outcomes in the thematic area of diagrams (Ζ = −2.326, p = .02). Laptop users agreed that the specific exercise process had a positive impact on their learning in the scope of sitemap/mind-map design (M = 1.6, SD = .69), whereas iPad users were skeptical about this (M = 2.55, SD = .82). Subject-centric skills like “developing practical experience in the development of flow diagrams” were rated low in this case. However, the social learning skills rates such as “collective problem solving” and “remote communication and collaboration” were elevated for iPad users. Agreeing with literature, technology that supports both formal and informal social learning methods, can expand time-on-task, enhance engagement, trigger incidental learning and allow for more interpersonal experimentation [51, 66]. Indeed, the socio-physical process of collective “trial and error” and iterative experimentation was practiced and remarked heavily on by co-learners in this study. Reportedly this enables learners to perceive this type of tacit learning as deepening their understanding of the subject matter [68, 69].

“And as soon as we would finish one category – the other one would check to see if ok…when one makes a mistake, the other will see it and will try to solve it…”

“It was a way to make sure (ensure) that whatever we were thinking was correct… it’s much better to collaborate from the point of view of errors/mistakes… rather than being alone…”

Adding to the informality of learning, we were also able to infer that students implicitly associated touchscreen devices with specific types or levels of learning activities. Our findings indicate a consensus towards the device’s affordances for spontaneous and experimental types of tasks, like brainstorming, quick sketching and online research.

“It’s easy for searching purposes to find it at the time… so not to setup the ‘whole’ laptop thing…”

“Use it like for illustration… not using an extra device – since it is touch…. And it is more direct… for drawing…”

“I can work with brainstorming yes… but in a much more abstract and loose way… it’s flexible”.

As the project lifecycle progressed, the need for finer manipulation and control, target precision and advanced functionality (Fig. 1) warranted more stable and accurate equipment (personal computer, larger screens, pointing/input devices), that could also maintain the focus on the task-at-hand rather than dealing with the quirks of the device.

Fig. 1.
figure 1

Lucid chart user interface for sitemap design

Full size image

4.4 Co-configuration Component

The learning network in this study was modified to ensure that the goals of the assignments could be accomplished (epistemic) and that efficient group collaboration was feasible (social). Two run-time interventions on the software tools (Set) were necessary to resolve problems that arose from (a) combination of poor tool interface affordances and bad internet speeds and (b) preferred method of communication for coordination purposes – in remote collaboration.

In phase one of the study, negative feedback was received regarding the tool used (Lucid Chart) from participants. Reportedly the lack of visual cues of their collaborators’ actions caused disorientation while working online (4.1 Error Incidence). In addition, the students had already reached the maximum amount of shapes allowed by the software. Following search for alternatives and agreement from all stakeholders (researchers, instructor, and students), the next assignment was implemented using a different tool (Coggle).

Likewise, in supporting their need of communication and coordination during remote collaboration, and although another software tool was installed, students switched to other favored tools to expedite the process, as they claimed it was “faster for them” to work in that way. They used Skype or Facebook messenger to either chat but mostly conduct video calls while working on the shared artefact.

5 Discussion

Based on the results, this study has derived seven important m-learning guidelines for instructional designers and educators in UX design and related areas. These stemmed from outcomes concerning: (a) the preferential mode of learner collaboration in using iPads (b) the devices’ social informal learning potential, (c) its contribution in the exploratory phases of the design cycle (d) the necessity for a flexible co-configurable learning Set and (e) the ascertainment that exact device-only evaluation is often pervaded by adjacent factors such as software affordances and connectivity performance. The seven guidelines that derive from the above are as follows:

  1. (a)

    Preferential mode of learner collaboration in using iPads

  2. (b)

    The potential of mobile devices in informal learning

  3. 1.

    Collaborate with stakeholders (learners, instructors) in selecting and evaluating area-specific software tools, matched against the design goals, their synergy with other tools (if any) and their online performance, prior to instructional interventions.

  4. 2.

    Embed design tasks that encourage co-located and remote student groups to extend their learning in informal, social settings and activities through the use of mobile devices.

A few studies indicate the odd tendency to share single-use devices in small co-located groups [70], nonetheless, it was this study’s initial hypothesis that the opportunity for synchronous single-user control of the work-in-progress, through online collaboration, would be of preference to users. In agreement with prior research, its findings present the preferred work approach as the immediate, “there-and-then”, single-screen type of co-located - versus - online collaboration. Close-proximity, face-to-face communication and coordination, even when working with small displays which might prevent active participation by everyone, were the dominant factors for this. Most interestingly, learners were initially prompted to use a single device in their groups in class – however – this approach was also taken to out-of-class settings and in various states (being sedentary, on the move, lying down etc.), a phenomenon occurring with iPad only. This indicates the device’s key potential in promoting social ubiquitous learning and knowledge that is both implicitly and explicitly acquired in informal conditions and environments.

The possibility that the co-located behavior was chosen over low network speeds, cannot be ignored. We therefore seek to run additional investigations on the quality and extend of distance collaboration - relying on stable infrastructures – versus the co-located method, in order to obtain more accurate results.

  1. (c)

    The iPad’s role in the exploratory phases of the design cycle

  2. 3.

    Incorporate iPads predominantly in the early, exploratory stages of the design lifecycle, such as brainstorming, sketching and research to enhance intuitive collective experimentation

  3. 4.

    Ensure that intermediary input devices (i.e. stylus pen, apple pencil etc.) are incorporated in later development and refinement stages of the design lifecycle

In regards to the pedagogical (epistemic) outcomes, this study elicits that (a) the device had not explicitly augment learning in the students’ perception and (b) that the type and state of the task dictates the working approach, which in-turn specifies the choice of appropriate device on two stages: exploratory and refinement. The stricter, more refined and greater-detailed visual artefacts to be produced, the more accurate selection and handling mechanisms are required. These are far from the fingertip-sized target manipulation offered by tablets and smartphones which are evidently a better fit for earlier experimental steps in the process; these being online research, brainstorming, mind-mapping and sketching. On the other hand, the design and fine-tuning of sitemaps with hairline-width connector lines and arrows, tiny transform vertices and pixel-perfect positioning, call for advanced precision tools, which are lacking in these devices.

  1. (d)

    The importance of a flexible co-configurable learning Set (hardware, software, physical)

  2. 5.

    Engage into thorough, run-time observation and investigation of all instructional system components (Set, Social, Epistemic) throughout the design lifecycle, and allow for or reconfigure accordingly if required.

In this study, we are able to confirm that the instructional design should be malleable enough to expect and adopt ad-hoc restructurings on any of the three components of a learning network (Set, Social or Epistemic). Likewise, the software constituent of the Set component, was co-configured twice, both externally (top-down) from the instructional researchers and internally (within) from the team members, to accommodate concerns regarding (a) tool UI complexity and (b) preferred method of communication. Such a design directed participants along a well-defined learning path while at the same time encouraged learner autonomy and self/team-management to counterbalance pragmatic limitations at the time of execution.

  1. (e)

    Specific Set component usability evaluation is often difficult to isolate

  2. 6.

    Design specialized methods of investigation – depending on the context - for the evaluation of m-learning, in order to elicit targeted findings for each separate unit of interest (i.e. hardware vs software)

Lastly, this study infers that users are most often unable to have distinct hardware and software experiences. This reinforces the view of the inter-dependency and the cross-impact of the two major constituents of CSCL, hardware and software.

We sought to investigate participants’ experience in building sitemaps and mind-maps using the specific device, yet responses referred predominantly to software. Undeniably the software UI “impacts experience more than the underlying hardware” and this by itself calls for more specialized human factors-driven research [60].

6 Conclusion

This study provides findings regarding the impact of incorporating iPads in the curriculum to support UX design activities for the creation of socially-responsible products, during the early design stages of sitemapping and mind-mapping. The outcomes denote that these devices can be suitably utilized in the early exploratory stages of the design lifecycle, their unique potential in extending learning beyond the classroom into informal social settings and that additional manipulation devices (i.e. apple pencil) are categorically required in the more advanced and refinement stages of the design cycle.