Abstract
In this study, we use augmented reality (AR) technology combined with the visual framework of the Concept Map (CM) to help children with autism spectrum disorder (ASD) apply the learned abstract social concepts and complex social relationships to the conceptual connection and establishment of social relationships in a structured and visual way. We hope to reduce the complexity and difficulty of social training for children with ASD through a teaching framework with social context and visual structure, and at the same time give children with ASD more contextual change and flexibility in teaching.
In the study, a total of four 7- to 9-year-old children with high-functioning ASD were recruited as the research subjects. They were given the AR teaching aid based on the CM design. The AR-based Concept Map (ARCM) social training system can generate corresponding 3D virtual characters and corresponding virtual social situations for different social relationships, and simultaneously give children with ASD 3D animation feedback while they watch the 3D virtual characters and social reciprocity behavior. Children with ASD can master and understand different social-relationship cognitions according to different social conditions and situations in the game. It also can be extended to the interaction state between different social objects, according to the current social context clues, through role-playing to show appropriate social reciprocity behavior. Finally, we believe that AR technology combined with the CM Framework is helpful to train autistic children’s social reciprocal ability, and thus, is worth developing as a teaching strategy and training method.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
- Augmented Reality (AR)
- Autism Spectrum Disorder (ASD)
- Concept Map (CM)
- Nonverbal social cues
- Social reciprocity skills
- Social relationships
- Tangible User Interface (TUI)
- Visual framework
1 Introduction
1.1 Congenital Social Defects in Children with ASD
Children with ASD have innate social defects that make it difficult for them to maintain eye contact during social interaction with others and to make appropriate social reciprocal behavior judgments based on different social relationships and social situations. They also have difficulty recognizing and using other people’s nonverbal social cues, such as judging others’ facial expressions and body language, to understand the meaning behind these social cues. In terms of interpersonal communication, children with ASD rarely show social behaviors such as cooperation or active social relationships because they are unable to understand other people’s emotions and to show appropriate social interactions. Children with ASD are typically characterized by social impairments in interacting with others, and they show social cognitive deficits and inflexible thinking. Social disorders in people with ASD include difficulties in responding to appropriate social greetings, the inability to understand other individuals’ nonverbal social cues, and the inability to look into others’ eyes to communicate emotionally. This makes it difficult for children with ASD to develop deep friendships, maintain social relationships, and integrate into conversations or social activities with peers.
Baron-Cohen pointed out that children with ASD have social defects in their Theory of Mind (ToM) ability [1]. That is, children with ASD cannot understand their mental state as it relates to others, so they cannot judge, identify emotions, feel emotions, understand ideas, lie, or play pretend games in social situations. ToM ability is considered to be the basis for understanding the behavior of others in social interactions. Therefore, children with ASD often have problems in understanding other people’s social messages and integrating social situations, and they act inappropriately in the process of establishing, maintaining, or interacting with others [2]. These improper behaviors often cause others to be puzzled or even label them negatively (for example, giving everyone the same greeting may cause others to view them as strange). Such behaviors can also have a negative impact on the social relationships of autistic individuals with family and friends [3].
1.2 The Barriers to Social Reciprocity in Children with ASD
Children with ASD have deficiencies in understanding other people’s expressions, distinguishing emotions, and ToM ability, so their social development is limited. This is one of the key issues for children with ASD because it causes them difficulty in interacting with others and prevents them from establishing further social relationships. Bellini, Peters, Benner and Hopf suggest that the main social-skill deficits of children with ASD are in the beginning stage of social communication, nonverbal communication, social reciprocity, and social cognition [2]. These abilities are important factors for deeper social interactions with other children.
Social-reciprocity ability is the most important key to these capabilities, and it is also the most important social-training project in this study. Social reciprocity refers to being able to judge a social situation, status, or other individuals’ nonverbal social cues to understand others’ true thoughts and intentions, and to interact with others in a socially suitable way. This ability is accompanied by comprehensive aspects such as social cognitive ability and situational-understanding ability. Because of the aforementioned defects of ASD-affected children, social skills have a great impact on the quality of life of these children, so proper-social-skills training is very important. Appropriate social training can help children with ASD to improve their social adaptation barriers, and also increase their opportunities to integrate into society and build social relationships with others.
2 Related Work
2.1 Social-Skills-Training Strategies Commonly Used in Children with ASD
Common social-skills-training strategies today include the use of role-playing or game interaction to improve the social reciprocity of children with ASD, enhance their emotional-grasp skills, and their ability to communicate with others. Gresham, Sugai and Horner believe that the training content of social-skills training should be designed to address the social deficiencies of children with ASD, such as increasing the degree of fluency in social interaction between autistic individuals and others, or improving autistic individuals’ mastery of social situations [4]. Social Stories™ is one of the most commonly used methods to help children with ASD understand and recognize social content.
Social Stories™.
Social Stories™, mainly proposed by Gray and Garand, is widely used to teach children with ASD to understand social situations and social concepts to increase appropriate social reciprocity [5,6,7] and reduce negative behaviors in children with ASD. Social Stories™ is a special story or short scenario script written by a special education teacher from the perspective of an autistic child. Special education teachers can observe and understand the problem behaviors of children with ASD often in daily life or the types of obstacles that affect personal social development before writing social stories. Researchers use the obstacle behavior of children with ASD as the training theme.
Each social story has a clear theme and specifically describes the events, causes, time, place and participants of the story. Social stories usually use text, pictures, photos, and videos to explain how others will act, think, or feel in the same context, and point out important social cues for training [8]. The main purpose of Social Stories™ is to let children with ASD understand the skills and concepts of social reciprocity, and show appropriate social behavior in social situations. Furthermore, in addition to Social Stories™, Video Modeling (VM) has also been proven to be effective in improving strategies for children with ASD to learn social skills. The VM uses prerecording or direct shooting to target the problem behaviors or learning goals of children with ASD, and broadcasts them to children with ASD for viewing under the guidance of educators to learn social skills [9].
2.2 Applying Visual Strategies and Tangible Structured Teaching to Social Skills Training
The study found that visual strategies and tangible structured-teaching characteristics play an important key role in intervention teaching for children with ASD. Relevant past research has pointed out that social-skills-training strategies for children with ASD must be carried out through visual, tangible, and structured social-interaction scenarios. Therefore, special educators often use Social Stories™ to conduct social training or use VMs or character play games as a strategy for social-skills training.
Traditional social-skills-training methods mostly use a single role-playing and 2D pictures and video text as scripts. Through repeated training, children with ASD can remember and recite the social meaning in the picture. Although this method of teaching can improve the social skills of children with ASD, it is often unattractive for children with ASD and easily confuses them during the learning process. It even makes it difficult to understand the complex content of training and makes them bored and impatient, which often frustrates educators.
As technology advances, interactive technologies give more choices to frontline educators. Educators can choose appropriate teaching strategies and develop assistive technology based on the uniqueness and teaching goals of children with ASD, and use games and visual media to attract the attention and improve the learning effectiveness of children with ASD. Therefore, many researchers have brought different media technologies (such as AR or VR) into social-skills courses as a medium to present their teaching strategies. In contrast, traditional 2D graphics, photos, or videos that allow children with ASD to force memory or study in a single viewing mode have become unattractive to children with ASD. Therefore, the application of interactive technology, AR, to children with ASD is an indispensable development trend for the future.
2.3 Advantages of AR in Social Training for Children with ASD
AR technology can overlay virtual materials into real environments. As a result, many studies have applied AR technology to symbolic games for children with ASD [10, 11]. For example, AR technology is used for social-reciprocity-skills training [12,13,14,15] and emotion perception [16,17,18,19]. The findings confirm the effectiveness of AR for children with ASD who have the advantage of visual learning. AR technology presents learning materials to children with ASD through visual and interactive learning methods, thereby improving the inherent motivation of children with ASD and helping them understand the learning goals and content more easily. Through AR technology, children with ASD can focus more on learning materials and make the learning process more interesting.
Related researchers have pointed out that AR technology can provide visual cues to children with ASD to maintain their focus on observing specific nonverbal social cues, such as facial emotions, body movements, and other nonverbal social cues. In addition, AR technology can provide children with ASD with abstract social-interaction situations in the form of 3D virtual interactions to reduce the predicament of children with ASD who cannot understand the social situations described by educators due to a weak imagination [20, 21]. Interacting with virtual characters through AR technology can also prevent children with ASD from facing the fear of real people in a real environment and the risk of psychological harm, and increase the likelihood that children with ASD will transfer their learned social skills to a real environment.
2.4 The Application of AR Technology in ASD Social Training Lacks a Clear Learning Framework and Organizational Structure
Although AR technology can effectively attract the attention of children with ASD in teaching and has quite a rich media effect, the 3D pictures and interaction concepts presented by AR often lack a clear learning framework and organizational structure to help ASD. Children need to sort out the teaching content behind a large amount of social concept knowledge; especially for children with ASD, it is more difficult to understand the complex social relationships and social content. Therefore, CM as a teaching tool and visualization strategy for concept presentation and contextualization is quite suitable for the understanding of social concepts in ASD-affected children; especially for children with ASD, overly complex or organizationally abstract social messages are more difficult to understand.
This study combines a CM strategy to provide a teaching framework and learning cognitive framework to help children with ASD organize complex and abstract social information for learning, as a prerequisite for the instruction of children with ASD. Knowledge is the basis for learning new ideas. Therefore, the use of AR with CM as a medium for presenting social stories to provide an environment for the contextual learning of children with ASD is a very innovative and effective teaching strategy.
2.5 CM Helps Children with ASD to Integrate and Organize Learning Content Organization
The CM method was first developed by Novak and his research team at Cornell University, USA, in 1970. CM design is based on Ausubel’s cognitive learning theory [22]. In cognitive learning and constructivist epistemology, cognitive class-learning theory emphasizes that learners must first have the relevant prior knowledge framework when acquiring new knowledge. The concept is that the prior knowledge is used as the basic framework for the learner to understand the new knowledge, and the new concepts and knowledge for the learner are taught in the known-knowledge background. The learner can combine the newly learned knowledge with his original cognition, and appropriate architecture generates conceptual connections and familiarity to produce meaningful learning [23].
Ausubel believes that meaningful learning mainly occurs during the process of teaching learning materials. These processes can effectively help learners acquire a large amount of knowledge in a very short period of time [24]. The CM method is often used by educators as a teaching tool, and is widely used in teaching more complex and abstract subject areas, such as mathematics, chemistry, biology, and so on. For example, related research uses CM and AR technology as training tools to teach elementary school students’ science courses to help students organize learning structures and understand ecological food chains [25]. The above related research results prove that the AR system using the CM method has a better learning effect than the AR system without CM, and CM can help students focus more on the learning goals and master the curriculum and content. The reason for this is that the CM method facilitates visualizing the patterned structure of human thinking; then, through the point-link-network, the prior knowledge and new knowledge are organized, and the visual framework is transformed into a knowledge-learning structure to help learners acquire new knowledge. At the same time, complex information and abstract concepts are presented through a visualized and structured teaching method to reduce the cognitive load on learners. In addition, the visual cognitive deficits of children with ASD make it easier to focus on specific details and ignore the interrelationships between different social contexts, content, and social objects, and children with ASD themselves have difficulty understanding abstract things. Therefore, the application of CM to decomposing complex and abstract social relationships and states is more appropriate and has research significance.
2.6 CM Helps Visual-Information Processing in Children with ASD
It is pointed out in related research that children with ASD tend to understand different abstract concepts and knowledge through vision. Cognitive load theory also shows that children with ASD who have the advantage of visual processing can use conceptual maps to reduce their cognitive load in learning tasks and help them link to learn relationships under different information concepts, thereby achieving progress in learning tasks. Related studies have also shown that the educational benefits of using visual strategies to educate children with ASD outweigh the use of written information, and that the information obtained is better retained. Roberts and Joiner also believe that children with ASD perform better in visual processing than with text [26]. Therefore, the visual-learning framework and teaching strategies of CM are very suitable for the learning needs of autistic learners. Because CM provides a link between social behavior and social-concept features, allowing children with ASD to concatenate all known details, all such teaching methods can significantly reduce the innate cognitive deficits caused by ASD symptoms.
2.7 A Case Study of the Application of CM in the Teaching of Children with ASD
In their research on CM, Roberts and Joiner teach and train autistic students with CM and require autistic students to create their own sentences according to the concept-link-concept model [26]. The results show that after CM intervention teaching, children with ASD demonstrate a positive improvement in their ability to learn and understand teaching content, which indirectly contributes to a significant improvement in learning effectiveness, and which represents four times the traditional teaching intervention [26]. This proves that CM can greatly help children with ASD in the area of organizational learning materials. In addition, some researchers have used CM and AR technology as training tools for instruction in autistic social-skills-training teaching strategies, and to teach children with ASD to focus on nonverbal social cues and recognize the social relationships of others to improve their social skills [13]. The results show that the use of CM in conjunction with AR significantly improves the cognitive ability and situational understanding of social relationships in children with ASD.
2.8 Summary
Based on the above, it can be seen that CM is of great help in teaching children with ASD to learn about abstract ideas and conceptually link abstract social information. In addition, CM can integrate learning materials into a knowledge framework. Using CM as an excellent learning and teaching framework, the educator can conduct constructive teaching of patients in a more structured manner. At the same time, learners can also use CM as a tool for meaningful learning. Therefore, this study will use the visual framework of the CM method to present different social links and social relationships in a systematic and structured way. Through the learning structure combining CM and AR, children with ASD can see the linked relationship of important concept elements in social situations and assist in the systematic presentation of social situations through the use of physical graphics cards and AR technology.
3 Method
3.1 Participants
This study mainly designed a set of ARCM social-training system based on conceptual graphics for four 7- to 9-year-old children with high-functioning ASD. Through board games, they were given structural instruction and training in social situations and relationships.
3.2 Design Purpose of the ARCM Social-Training System
The main design purpose of the ARCM social-training system is to allow children with ASD to understand social situations and relationships and show appropriate social-reciprocity behaviors. The system will simulate the full picture of real social situations and present it through 3D images to reduce the cognitive-learning load of children with ASD and attract their visual attention. The ARCM social-training system includes 3D CM, social scenes, social objects, text scripts, and dialogue voices. All content is designed based on social situations common in ASD case life to provide children with ASD with more non-spoken visual cues. These visual cues are mainly to help children with ASD to observe and understand abstract social situations and environments, and the causality of social events. The social scenes include virtual scene objects such as homes, schools, streets, and parks. The social objects include virtual characters such as family, friends, strangers, and so on. The presentation of each social-situation topic will be designed based on the social cognitive ability and level of children with ASD. The text and title content presented in the system are marked with pinyin and accompanied by a voice description, and the system will automatically determine the content selected by the autistic child and give text, voice, sound effects and visual feedback.
3.3 ARCM Social-Training-System Development
The ARCM social-training-system mainly uses the unity platform for construction and development, and combines 3Dsmax to construct the 3D space models (including homes, schools, parks, supermarkets, etc.) required for social scenes, and builds 3D character animation (including classmates, neighbors, strangers, etc.). After fine-tuning the post-production of character animation, the system uses AE to make the animation option required for the social-training response. After the system is completed, the therapists, related experts, and normal children of the same age perform operational tests and evaluations to ensure that children with ASD will not feel afraid or uncomfortable during the procedure (Fig. 1).
3.4 Design of ARCM Social-Training System Interface
The ARCM social-training system includes several physical cards with (1) social definitions, (2) concept boards for social situations, and (3) CM for social relationships. Those socially defined physical cards are mainly used when children with ASD operate AR systems. Children with ASD can present AR content through the ARCM social-training system developed by this institute when operating physical cards. The ARCM social-training system recognizes the physical cards with social definitions through the back-end program, and sees the 3D characters and extended social content in the game interaction from the 15-in. laptop’s screen in front. Each physical card represents a specific social line of information, and a complete social situation concept board consists of several cards. The superposition of different physical cards is equivalent to the superposition of social relationships and social concepts (Fig. 2). At the same time, the cards are linked in the process, and children with ASD can clearly see the social relationships and relationship status between each social role.
3.5 Operational Design Between ARCM Social-Training System
ARCM social-training system’s teaching mainly includes a series of interactions between social processes and social objects. Its design draws on the CM social-training strategies used in ASD teaching in the past, and designs social situations and content through strategies with visual structure, social relationships and specific social-training strategies. The content design of ARCM social-training system teaching materials includes three sections of content and procedures: (1) the background of social events and social situations; (2) the conceptual map of the relationship between social situations and social roles; (3) a quiz to determine the individuals’ emotion and social reciprocity.
The Background of Social Events and Situations.
After contacting the children with ASD, the researchers understood the social-skills issues of children with ASD. Then they developed a social-story script based on the theme of social disorders that often occur in the daily lives of children with ASD, and set social-learning goals according to the abilities of children with ASD. ARCM social-training system describes and organizes social-situation problems that often occur in children with ASD through social-story strategies, and uses them as social-training tasks, and then considers the abilities of children with ASD to give them an ASD textbook. In addition, in order to ensure that the teaching content is consistent with the reading ability and social awareness of children with ASD, the teaching content was reviewed by 2 experts with experience in social-story-strategy script intervention teaching to ensure that the social-story situation script used in this research was suitable for children of that age with ASD.
Conceptual Diagram Between Social Situation and Social Relationship.
The researchers deconstructed the content of the described social-story scripts into several parts, including (1) social objects, (2) situational context of social events, and (3) main social-event content. This content will be provided for the social training of children with ASD. During the experiment, researchers will further ask children with ASD to make the following social judgments: (1) judging the current emotional state of the social character; (2) the inner thoughts of the social character; (3) what the social characters should do in the context action. According to the CM strategy, the social story presented by the social process is decomposed into simple concept nodes, and the relationship between different points is communicated through link lines to establish a CM framework. The “concept-link-concept” is organized into a social situation CM (Fig. 3).
In addition, in the CM, the social relationship between different social objects and children with ASD is constructed, and it is divided into three levels according to the degree of intimacy, including intimacy, acquaintance and strangeness. Each social object is an independent node, and the distance between the link lines between the nodes represents the relationship between the two. Through the connection between points and lines, the difference in the distance between social objects and children with ASD is also presented. The closer to the center, the closer the social relationship (Fig. 4).
In addition, this study also uses two corresponding CMs to allow children with ASD to repeatedly compare and generate conceptual relationship structures. Children with ASD can understand different social relationships and interaction structures through operations and games, and use the visual content presented in the CM to express them and make appropriate interactive content (Fig. 5).
Quiz to Identify Each Other’s Emotions and Social Reciprocity. This research not only aims to enable children with ASD to grasp the scenes in daily life and show appropriate social-reciprocity behaviors according to different characters, but also constructs the conceptual framework emotionally. Researchers make social emotional judgments using the six major emotions defined by Ekman: happiness, anger, sadness, disgust, surprise, fear, and make social reciprocal actions based on the emotions. The formulation of these actions is based on the social greeting behaviors common in daily life in Taiwan, including: (1) waving hands, (2) nodding and smiling, (3) shaking hands, (4) holding hands, (5) hugging, (6) shaking the head, (7) bowing, (8) patting the shoulder, (9) clapping the hands, (10) placing hands on shoulders, and so on, as shown below (Fig. 6).
3.6 System Operation and Teaching Environment Observation
In this study, the therapists, special education teachers, and researchers observed and recorded the ASD-affected children’s situation in the game and the teaching process of operating the ARCM social-training system. During the game, children with ASD must first place the social object’s character graphics, field graphics, and event descriptions in the corresponding spaces according to the order arranged on the social-situation concept board. At this time, the ARCM social-training system will superimpose 3D virtual characters and virtual scenes on the physical-graphics card through the screen display, and then the social-context animation will be played when the system detects the correct graphics-card pairing. Special education teachers will require children with ASD to watch social-context animations, and put the emotion and action-graphic cards corresponding to social situations into the corresponding spaces. At the same time, the system will play subsequent social-context animation according to the physical-graphics option selected by the children with ASD. In these processes, children with ASD can see not only different social situations, but most importantly, they can see the connection between the social information represented by physical graphics. In addition, after using ARCM social-training system for children with ASD, special education teachers and researchers scored according to the learning status of children with ASD as a result of evaluating the use of the ARCM social-training system. Children with ASD must complete the following operational tasks when using ARCM social-training system for social-skills training, as shown below (Fig. 7). Those conducting this study hope that children with ASD can complete different social context tasks through the ARCM social-training system and improve their grasp of social context and reciprocity behavior during the game.
Social-Training Content Covered by the ARCM Social-Training System. The social-training content covered in the ARCM social-training system mainly addresses the establishment of two main social-cognitive concepts: (1) cognitive concepts among social objects; (2) cognitive concepts between social context and social reciprocity. In the establishment of cognitive concepts among social objects, first of all, children with ASD will learn about the cards of all social objects and 4 main social environments (including their own rooms, homes, school classrooms, urban streets, etc.) in a ARCM social-training system. Then, the special education teacher will ask the children with ASD to place the social object graphic card in the corresponding social environment field space according to the problem’s instructions, and then connect the links between these social objects (the line segments represent the social correspondence between themselves and others, such as close family members, friends, strangers, etc.) (Fig. 8).
In addition, in the establishment of cognitive concepts between social contexts and social reciprocity behaviors, researchers use social context concept plates to present the social reciprocity behaviors that should be generated in different social contexts. A total of nine spaces on the social situation CM represent different social structural units, including (1) oneself, (2) social objects, (3) the place of occurrence, (4) the event, (5) their emotions, (6) the emotions of the social object, (7) the actions of the social object, (8) the inner thoughts of the social object, and (9) the response of the social object. This series of social structural units constitutes a complete social context and corresponding social-reciprocity actions (Fig. 9).
4 Results
In this study, we conducted multiple game observations with 4 children with ASD and a special education teacher and therapist to understand whether the ARCM social-training system intervention strategy would help children to grasp the social context and social reciprocity. A score of 0–7 was assigned to measure whether the child with ASD could master abstract social concepts and the structure of complex social relationships (1 means complete lack of understanding; 7 means full understanding).
Researchers also observed whether the ARCM social-training system could effectively attract their attention to the relevant social-training details during the game, especially whether they could grasp the social-context content in the social story and the state of social-reciprocity behaviors that should be displayed in social links, which indirectly would affect their social-skills improvement.
After 5 weeks of observation, this study referred to 6 aspects of Positive Technological Development (PTD) to observe the social-reciprocity behavior of children with ASD during social training with special education teachers using ARCM social-training system. (The PTD included (1) Communication; (2) Collaboration; (3) Community Building; (4) Content Creation; (5) Creativity; (6) Choice of Conduct.) Discussion and evaluation with therapists and special education teachers by recording the results of the film then followed.
After discussion and observation, the therapist pointed out that 4 children with ASD were quite invested in ARCM social-training system game training. This shows that the ARCM social-training system is quite attractive for children with ASD and performs well with respect to all six indicators. Such performance will help children with ASD be more integrated into teaching and training, and such training results are also reflected in their mastery of the social-training content.
In the observation record of 5 weeks, the therapist and special education teacher gave 4 children with ASD a different score performance on the ARCM social-training system-learning and social-training-content mastery-score performance, as listed below: (a) A score of 4.75 was achieved in the expression of social-speaking ability (0 represented lack of oral interaction and 7 represented full oral expression). (b) A score of 4 points was achieved in facial expression and emotional grasp (0 meant completely difficult to grasp, 7 meant completely grasps). (c) A score of 4.25 points was achieved in the social situation and understanding of social relationships (0 meant complete lack of understanding; 7 meant full understanding). (d) A score of 3.75 points was achieved in social behavior, understanding of body movements, social reciprocity behavior ability (0 meant completely unable to master; 7 meant fully able to master). (e) A score of 5 points was achieved for the performance of other people’s social physical interaction (0 meant lack of social interaction; 7 meant frequent social interaction). (f) Sharing the relevant social situation story with the therapist earned 5 points (0 meant not happening, 7 meant actively sharing with the therapist). The above scores show that this training method is helpful and effective for ASD-affected children’s learning performance and social training using the ARCM social-training system.
5 Discussion and Conclusion
Based on the above, it can be shown that the CM method is of great help in teaching children with ASD to learn about abstract social concepts and to conceptually link abstract social information. We attribute the success of the ARCM social-training system to the following:
5.1 The ARCM Social-Training System Reduces the Cognitive-Learning Load of Children with ASD and Makes Social-Skills Training More Efficient
CM provides structured teaching strategies to help children with ASD better understand and organize abstract social skills and to help educators know which areas children with ASD need to strengthen. In addition, in the past, in social-skills training, due to the imagination deficit of children with ASD, the therapist needs to spend a lot of time simulating, explaining the situation, preparing graphics or setting the environment for the unimaginative ASD child to understand social-story situations, all of which often take a lot of time or frustrate the therapist and case parents. Today, the visual process of the ARCM social-training system can quickly allow children with ASD to understand and explain social situations and relationships, so that therapists or case parents can directly teach social skills to enable children to see through the screen of the social situation.
5.2 Provide a Physical Operation Interface that Is Consistent with Past Gaming Experiences and Maintain the Attention of Children with ASD
Children with ASD are curious about technology aids. ARCM social-training system use graphics cards to display virtual social situations on the screen, so that children can keep their attention on the screen to help educators train their social skills. The operating system of the graphics card is also consistent with their previous experience in playing games, and the learning threshold is low, which can make it easier for children with ASD to understand how to use ARCM social-training system, and so no further learning is required. Compared with past images or tablet games, ARCM social-training system increases the interaction of physical board games, which enhances the fun of learning and the visual sensory stimulation.
5.3 Provide Lively Social-Teaching Situations and Protect Children with ASD from Physical and Mental Harm
Compared to the one-way VM, the ARCM social-training system can provide a variety of social situations and two-way interaction with virtual social objects, especially social situations that have not yet occurred. This is very important for children with ASD, because they often lack social situations and opportunities to learn and practice how to socialize with others in real life. In addition, the ARCM social-training system can present the role of the children with ASD in the scene when establishing the character model (the character model can be established according to the facial features of the ASD case). Children with ASD can use screens to observe their social situations with social partners. They can also zoom and rotate the lens to change the viewing angle and distance. The full picture of a social situation can be seen in the distance, along with the actions of social virtual objects and even facial expressions from a short distance. In addition, ARCM social-training system teaching has a controllable feature, which allows children with ASD to practice social skills over and over again. Even if they make mistakes, they will not be harmed, which helps to reduce the uneasiness of children with ASD.
5.4 Future Work
At present, the ARCM social-training system developed by this research institute focuses on teaching ASD-affected children’s cognitive abilities in social relationships, social situations, and social reciprocity behaviors to improve their social skills. By observing the behavioral responses of children with ASD during the use of ARCM social-training system, we found that CM combined with AR-technology-aided teaching aids can better maintain the focus of children and strengthen their curiosity and interest in learning tasks by heart. Finally, our research shows that ARCM social-training system can better help children with ASD understand social situations and reduce their learning load. In addition, researchers have collected considerable first-hand information to help improve ARCM social-training system to provide more evidence-based and in-depth research in the future.
References
Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind”. Cognition 21(1), 37–46 (1985)
Bellini, S., Peters, J.K., Benner, L., Hopf, A.: A meta-analysis of school-based social skills interventions for children with autism spectrum disorders. Remedial Spec. Educ. 28(3), 153–162 (2007). https://doi.org/10.1177/07419325070280030401
Webb, B.J., Miller, S.P., Pierce, T.B., Strawser, S., Jones, W.P.: Effects of social skill instruction for high-functioning adolescents with autism spectrum disorders. Focus Autism Other Dev. Disabil. 19(1), 53–62 (2004)
Gresham, F.M., Sugai, G., Horner, R.H.: Interpreting outcomes of social skills training for students with high-incidence disabilities. Except. Child. 67(3), 331–344 (2001)
Karal, M.A., Wolfe, P.S.: Social story effectiveness on social interaction for students with autism: a review of the literature. Educ. Train. Autism Dev. Disabil. 53(1), 44–58 (2018)
Rao, P.A., Beidel, D.C., Murray, M.J.: Social skills interventions for children with asperger’s syndrome or high-functioning autism: a review and recommendations. J. Autism Dev. Disord. 38(2), 353–361 (2008). https://doi.org/10.1007/s10803-007-0402-4
Gray, C.A., Garand, J.D.: Social stories: improving responses of students with autism with accurate social information. Focus Autistic Behav. 8(1), 1–10 (1993)
Coogle, C.G., Ahmed, S., Aljaffal, M.A., Alsheef, M.Y., Hamdi, H.A.: Social narrative strategies to support children with autism spectrum disorder. Early Child. Educ. J. 46(4), 445–450 (2018). https://doi.org/10.1007/s10643-017-0873-7
Fitzgerald, E., et al.: Comparing the effectiveness of virtual reality and video modelling as an intervention strategy for individuals with autism spectrum disorder: brief report. Dev. Neurorehabil. 21(3), 197–201 (2018)
Bai, Z., Blackwell, A.F., Coulouris, G.: Through the looking glass: pretend play for children with autism. In: 2013 IEEE International Symposium on, Mixed and Augmented Reality, ISMAR (2013)
Syahputra, M., et al.: Implementation of augmented reality in pretend play therapy for children with autism spectrum disorder. In: Journal of Physics: Conference Series (2019)
Lee, I.-J., Lin, L.-Y., Chen, C.-H., Chung, C.-H.: How to create suitable augmented reality application to teach social skills for children with ASD. In: Mohamudally, N. (ed.) State of the Art Virtual Reality and Augmented Reality Knowhow. BoD – Books on Demand, Norderstedt (2018)
Lee, I.-J., Chen, C.-H., Wang, C.-P., Chung, C.-H.: Augmented reality plus concept map technique to teach children with ASD to use social cues when meeting and greeting. Asia-Pac. Educ. Res. 27(3), 227–243 (2018). https://doi.org/10.1007/s40299-018-0382-5
Syahputra, M., Arisandi, D., Lumbanbatu, A., Kemit, L., Nababan, E., Sheta, O.: Augmented reality social story for autism spectrum disorder. In: Journal of Physics: Conference Series (2018)
Lee, I.-J.: Kinect-for-windows with augmented reality in an interactive roleplay system for children with an autism spectrum disorder. Interact. Learn. Environ. 1–17 (2020)
Chen, C.-H., Lee, I.-J., Lin, L.-Y.: Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Comput. Hum. Behav. 55, 477–485 (2016)
Chen, C.-H., Lee, I.-J., Lin, L.-Y.: Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Res. Dev. Disabil. 36, 396–403 (2015)
Ahmad, H.B.: An augmented reality system to enhance facial expressions recognision in autistic children. J. Adv. Comput. Sci. Technol. 8(2), 46–49 (2019)
Lee, I.-J.: Augmented reality coloring book: an interactive strategy for teaching children with autism to focus on specific nonverbal social cues to promote their social skills. Interact. Stud. 20(2), 256–274 (2019)
Lorenzo, G., Gómez-Puerta, M., Arráez-Vera, G., Lorenzo-Lledó, A.: Preliminary study of augmented reality as an instrument for improvement of social skills in children with autism spectrum disorder. Educ. Inf. Technol. 24(1), 181–204 (2019). https://doi.org/10.1007/s10639-018-9768-5
Keshav, N.U., Vogt-Lowell, K., Vahabzadeh, A., Sahin, N.T.: Digital attention-related augmented-reality game: significant correlation between student game performance and validated clinical measures of attention-deficit/hyperactivity disorder (ADHD). Children 6(6), 72 (2019)
Novak, J.D.: Concept mapping: a strategy for organizing knowledge. In: Glynn, S.M., Duit, R. (eds.) Learning Science in the Schools: Research Reforming Practice, 229-245. Routledge, Abingdon (1995)
Zheng, R.Z., Dahl, L.B.: Using concept maps to enhance students’ prior knowledge in complex learning. In: Song, H., Kidd, T. (eds.) Handbook of Research on Human Performance and Instructional Technology, pp. 163–181. IGI Global, Hershey (2010)
Novak, J.D., Bob Gowin, D., Johansen, G.T.: The use of concept mapping and knowledge vee mapping with junior high school science students. Sci. Educ. 67(5), 625–645 (1983)
Chen, C.-H., Chou, Y.-Y., Huang, C.-Y.: An augmented-reality-based concept map to support mobile learning for science. Asia-Pac. Educ. Res. 25(4), 567–578 (2016). https://doi.org/10.1007/s40299-016-0284-3
Roberts, V., Joiner, R.: Investigating the efficacy of concept mapping with pupils with autistic spectrum disorder. Br. J. Spec. Educ. 34(3), 127–135 (2007)
Acknowledgments
We are grateful to the Executive Yuan and Ministry of Science and Technology for funding under project No. MOST 107-2218-E-027 -013 -MY2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, YC., Lee, IJ. (2020). Using Augmented Reality and Concept Mapping to Improve Ability to Master Social Relationships and Social Reciprocity for Children with Autism Spectrum Disorder. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Applications and Practice. HCII 2020. Lecture Notes in Computer Science(), vol 12189. Springer, Cham. https://doi.org/10.1007/978-3-030-49108-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-49108-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-49107-9
Online ISBN: 978-3-030-49108-6
eBook Packages: Computer ScienceComputer Science (R0)