Keywords

1 Introduction

We do not yet have a conceptual model for defining adaptive instructional systems (AISs), however, it is typically understood that unlike traditional computer-based instructional systems, AISs guide learning experiences by tailoring instruction and recommendation to each learner based on their goals, needs, and preferences in a specific learning domain [1]. AISs describe a class of software that includes intelligent tutoring systems (ITSs), adaptive learning technologies, interactive media, and other learning tools or methods that are used to personalize and optimize instruction for a particular learner or teams. They typically seek to maximize learning outcomes and efficiency toward knowledge acquisition, skill development, retention, performance, and/or transfer of skills from the instructional environment to that of the “real world” where those skills would be applied.

1.1 AIS vs. Non-AIS

In a typical non-AIS, instructions are delivered to all students in the same way, often consisting of a fixed set and sequence of reading materials, videos, and/or exercises to be completed by all students. An AIS, on the other hand, may use individual variability in learning performance, learning pace, preferences, motivation, affective states, and other learner or team attributes together with instructional conditions to identify appropriate learning strategies and/or tutor actions. Recent advances in artificial intelligence, sensing technology, and data mining methods afford modern AISs exciting new ways to engage students in more open-ended tasks and draw new insights into the learning process. From increasing integration of natural language processing and affective state monitoring, to applications of simulations and interactive media, AISs are increasingly able to capture, process, and fuse high-frequency interaction data and natural rich modalities of communication, such as speech, writing, and nonverbal interaction during real learning activities. This provides unprecedented insight into the moment-to-moment development of a number of learning experiences, especially those involving multiple dimensions of activity and social interaction, enabling researchers to get far more nuanced and complex understanding of student learning processes, something that we have only begun to study at scale.

In many AISs, machine learning algorithms are typically used to describe the inter-connection among the learner’s state, the context of the learning experience, and AIS decisions, to recommend a learning path or actions that can maximize learning outcome. The adaptive architecture for different AISs differ by application across content, students, and learning objectives, but they can often be described as having two loops: an outer loop and an inner loop (Fig. 1; [2]). Let’s take an example of a simple AIS that involves many multi-step math problems, where the primary instructional mechanism is student answering questions and receiving feedback. In this AIS, the outer loop tailors the problem set that a student sees, and the inner loop personalizes instruction at the level of individual problem-solving steps. The outer loop executes once for each problem and iterates over the problems, giving feedback on the problem level (i.e. correct or incorrect) and selects the next problem that is appropriate for the student. The inner loop executes once for each problem-solving step and give feedback or hints on each step. The inner loop assesses and updates the student’s proficiency state or their learner model, which is used by the outer loop to select the next appropriate problem for that student [3]. It does this by looking at the skills that the student has currently mastered, evaluating the student’s knowledge state, and selecting the next optimal learning task.

Fig. 1.
figure 1

Source: https://www.carnegielearning.com/blog/truly-personalized-learning-all-about-those-loops/

A high-level description of an intelligent tutoring system.

Full size image

This two-loop model can involve much more complex interactions in a more sophisticated AIS that involves different types of learning tasks. An outer loop interaction may involve videos, interactive simulations, writing or speaking prompts, etc., in which proficiency estimation is not straightforward. The inner loop interaction also depends on the task. For a task that evaluates a student’s speaking skill, for example, the inner loop would need to evaluate the student’s speaking pattern against an optimal expert. This dialog-based inner loop adaptivity would require a task-specific ontology, one that is separate from that of the outer loop.

AISs involve a highly complex process that requires a technology and data-driven system of integrating instructional resources, learning objectives, and assessment activities into single, progressive modular learning elements that can be adapted to individual learners, reordered, or shared between learning systems. A key component of an AIS that enables this process is the domain model. Domain models define the structure of a particular domain, including learning objectives (i.e., a knowledge map) and the learning content, measures, assessments, interventions (e.g., feedback, dialog), associated with those learning objectives in that domain (i.e., a content map). The domain models can define these while taking into account the learner’s goals, prior knowledge, assessed skills, and other attributes such as motivation and interest. Taken together, the knowledge map and the content map can be thought of as layers of the learning ontology of an AIS. In order to appropriately measure a student’s knowledge and provide recommendations, this ontology often requires the knowledge components to be very fine-grained and well-defined.

The other three common components of AISs are the learner model, the instructional model, and the interface model [4, 5, cf. 1]. The learner model obtains and interpret data from the student through learning records, physiological or behavioral sensors and surveys, from learning record stores (LRSs) or learning management systems (LMSs). Instructional models assess the student’s progress toward learning objectives and recommend appropriate next steps. The interventions by the tutor/instructor and the student are managed through a user interface [1].

1.2 Need for Interoperability

With the large amount of learning interactions that can take place in AISs, there is an increasing need for standards that enhance the interoperability and extensibility of course content and configuration within an AIS. The ability to exchange models in an AIS allows for greater flexibility and enhanced instructional capabilities while reducing development costs. To do that, each of these models must be described in a standardized way to allow for an interchange of components and data exchange across components.

There is also a need for interoperability across AISs. In an ideal configuration, AISs work in concert leveraging open interoperability standards to provide a seamless experience for student and instructor, while ubiquitously leveraging rich high-frequency data to inform the learning flow.

In this paper, we focus on the interoperability among components within an AIS (vertical integration) and among separate AISs (horizontal integration) to provide services to each other. Specifically, we discuss the integration of components to support a more adaptable and extensible ecosystem within an AIS by building extensions on top of existing industry standards. We also discuss the need of interoperability standards across different AISs on the learning ontology and data models, and leveraging recent advances in federated machine learning to enable horizontal integration.

Prior work from IEEE Learning Technology Standard Committee, LTSC, IMS Global and others have established standards that would be essential for transitioning to an adaptable and extensible ecosystem. Samantha Birk [6] highlighted a set of existing IMS Global standards that play a key role in transitioning to AIS and provided a useful visualization of the existing interoperable adaptable learning ecosystem (Fig. 2).

Fig. 2.
figure 2

Source: http://www.imsglobal.org/adaptive-adaptable-next-generation-personalized-learning.

An adaptive learning ecosystem using some of the existing IMS Global standards: LTI integration, Gradebook Service, Caliper Analytics, (Thin) Common Cartridge and QTI/APIP.

Full size image

As is, however, existing standards remain limited in their capability to support the unique needs of modern AISs. We propose that extensions built upon existing standards, including LTI, Caliper Analytics/xAPI, Common Cartridge, QTI and others, can support the seamless data exchanges within and across AISs for adaptation at both the outer loop and inner loop levels.

In an ecosystem of adaptive learning process and system components (Fig. 3), a teaching and learning platform typically employs a Plan-Build-Deliver-Analyze cycle. During Plan, the learning map and the blueprint (i.e. goals) provide the basis for curriculum planning. During Build, content including the courseware, instructional, and assessment items are created. Each of these use different standards to express different corresponding data. The Competencies and Academic Standards Exchange (CASE) can be used to build the knowledge map, while Question and Test Interoperability (QTI) and Common Cartridge (CC) specifications can be employed to build assessment items and instructional items. During Deliver, the LMS is integrated with an environment like Customer Relationship Management (CRM), Student Information System (SIS), or Enterprise Resource Planning (ERP) via OneRoster or Learning Information Services (LIS). The individual components that plug into the LMS are supported by Learning Tools Interoperability (LTI) standard. Caliper Analytics or Experience API (xAPI) can be used to capture and store real-time learning event data in the Learning Record Store (LRS) for analysis and reporting. Taken all together, these existing standards provide an initial step toward vertical and horizontal integration, but all require extensions to be effective in this AIS ecosystem.

Fig. 3.
figure 3

A proposed ecosystem of the adaptive learning process and system components with extensions to existing interoperability standards (in blue). (Color figure online)

Full size image

2 Vertical Integration (Within AISs)

There is a myriad of tools and methods that can enhance student learning, but too often they are developed by different parties in isolation. Without a standardized way to transfer rich and contextual learning data among components, developers often resort to rebuilding third-party toolset within their own environment each time they seek to enhance their AIS’s capability.

At an outer loop level, extensions to existing standards like LTI and xAPI/Caliper Analytics can enable AIS developers to build learning experiences from multiple components, by linking instructional components together and track the data generated during the learning process. For example, a chemistry student can seamlessly access a third-party chemistry lab toolset from within the AIS, launched using an LTI extension, to practice applying the scientific method in a simulated lab experiment, prior to returning to the AIS for problem solving practice. The AIS can receive data about the students’ progress and performance on the chemistry lab, store them in the LRS, and use the updated learner model to adapt the learning experience when the student returns to the AIS.

Such extensions can also support interoperability at the inner loop level. In a language learning AIS, for example, a grading tool that uses natural language processing (NLP) embedded within a practice question can be hugely beneficial for learning. Alternatively, when a student answers a question correctly, an NLP-based dialog can be triggered to query the student’s reasoning process, perhaps to identify misconceptions using a mistake reasoning ontology specific to that domain learning objective. Such inner loop adaptivity requires a tremendous amount of development but allows us to capture individualized learning in ways that were not possible before.

To enable these rich learning experiences, in this new framework, there must be a mapping of corresponding ontology among learning tools and components to support continuous proficiency estimation, updates of learning records, and high-frequency user data interchange. Without interoperability standards connecting these components, AIS developers currently would have to build the tool from scratch to enable this kind of integration, and each of these tools require their fair share of research and development.

2.1 IMS Learning Tools Interoperability (LTI)

LTI standard currently prescribes an easy and secure way to connect any LMS with learning applications that range from general communication tools for chat and virtual classrooms to domain-specific learning engines for particular subjects like math or history. The LTI core establishes a secure connection and confirms the learning application’s authenticity, allowing students to switch seamlessly between, for example, a video conferencing tool and an assessment tool within the same workflow. Thus, rather than having to leave the LMS to log into an adaptive learning system outside of the LMS, with LTI a student can seamless move from an LMS to a third-party platform for the adaptive content. Possible extensions are available to add optional features and functions, such as features that support the exchange of assignment and grade data between an assessment tool and the LMS gradebook. This is a good step toward an ecosystem of tools and platforms, and apps for AIS. Such extensions are limited, however, when the LMS is adaptive or is an AIS.

An interoperability standard that extends the current LTI capabilities within an AIS framework can support the transfer of learning data among the components within the AIS while preserving the learning contexts and the user’s role within that context. When a student moves between the instructional system and an assessment tool, for example, the assessment tool can carry with it and return from it information that ensures she continues with the learning goal. The same information can be used to check whether the instructional system is providing the expected learning outcome and feed back into the adaptive engine. Importantly, this extended LTI capability can be combined with specifications from an Experience API (or xAPI) and/or Caliper Analytics standard from IMS Global Learning Consortium to receive and send data about a users’ behaviors in different AIS components in a consistent format using a single vocabulary. By providing a more comprehensive understanding of what a student is doing, such vertical integration of information within an AIS can support better predictions about student achievement for better adaptivity and enable learning analytics for new insights into how different learning interactions within an AIS relate to learning outcomes.

2.2 Common Cartridge (CC) and Thin Common Cartridge

Common Cartridge and Thin Common Cartridge are specifications useful for packaging and exchanging digital learning materials and assessments, often for importing and exporting them to or from an LMS. CC and Thin CC are useful in that they provide a standard way to represent learning course materials that can be developed and used across LMSs. They provide an easy way to add content to a course, saving developers and instructors content development time.

The content, however, is typically static (e.g., textbooks, chapters) and cannot be arranged and repackaged, thereby limiting the customizability for individual students. The content would also need to be connected to an ontology recognizable by the AIS. Thus, an extension on these standards could afford better interconnectedness among content and the transfer of richer content.

2.3 Question and Test Interoperability (QTI) and Accessible Portable AIS Item Protocol (APIP)

QTI enables the interoperability of assessment item content and results between authoring tools, item banks, and learning platforms, etc. APIP is an accessibility functionality for students with accommodation requirements. In an AIS, assessment items often provide the performance measures needed to inform the learner model and guide instructional next steps. However, current QTI and APIP do not allow for any content changes needed to tailor the learning path for a particular student. The algorithms that inform the learning path are currently tied to prescribed assessments. An extension would be needed to incorporate a level of logic that enables content to be modified and reordered as needed by the curriculum creator.

Enhancement to make systems more interoperable will likely result in higher reuse of components, lower development costs, and more collaboration in both the research and development of AISs, benefiting learners, instruction, and domains. An ideal extensible and cohesive AIS can leverage and built extensions upon these existing open interoperability standards (and others) to deliver a seamless experience for students via vertical integration among learning components while leveraging the rich data stream to inform the learning adaptation at both the outer loop and inner loop level.

3 Across AISs (Horizontal Integration)

The reform literature in mathematics and science is replete with calls for the cross-curricular integration of subjects (i.e. between STEM subjects and also between STEM and the humanities). However, there remains very few AISs (and few non-adaptive instructional programs) that can handle the prerequisite skills, knowledge bases, and experiences necessary to implement such integrated instruction. A typical AIS addresses a single subject domain at a time, and it often has a unique content ontology, adaptive engine, and data management method. A math AIS ontology, for example, consists of a knowledge map of prerequisite mappings of granular learning objectives in math, but it can have many connections to objectives in physics and chemistry, so an interoperability mechanism can allow us to combine the maps together and exchange student information across subjects. A student’s calculus knowledge, for example, could inform their experience in a physics or chemistry, where calculus is a prerequisite.

Thus, an extension of ontologies across AISs would not only expand the scope of multiple AISs but also enable a richer, cross-curricular learning experience for students. Corresponding to the knowledge graph, a “Federated” AIS approach can be also used to extend the learner model from one AIS to another. The combined and synchronized user model would require not only a set of common learner competency standard (such as CASE) but also the semantic and ontological construct for data exchange/synchronization.

3.1 Ontology

Competencies and Academic Standards Exchange (CASE).

CASE was created to address the need for competency-based educational programs to manage competency statements and students’ associated assessment results in a consistent and digital way. It supports the exchange of competencies and rubrics despite differences in the terminology, processes, and roles across different programs.

Some disciplines like math and physics have highly developed knowledge graph with well-defined learning objectives. Others are engaged in exciting debates about the best way to organize their domain knowledge. We propose that while professional associations shall be responsible for developing the knowledge graphs as they see fit, there is the need for a CASE standard extension to track and share the structural knowledge graph information across AISs.

3.2 Data Models

Caliper Analytics and Experience API (xAPI).

Caliper Analytics and xAPI provide the means for consistently capturing and labeling learning data and securely sending data to an LRS, setting the stage for an extensible adaptive ecosystem. These common data formats are particularly important because AISs are often built around proprietary standards and algorithms that are siloed, where there is little or no visibility for the users into what is happening in and across the learning environment. It is also increasingly common that students work in multiple learning environments, and possibly multiple adaptive AISs. This standardized data format allows the data to be collected and combined with another provider’s data points so that they can be shared and analyzed to get a more comprehensive understanding of the student’s progress within the curriculum. For example, scores can be passed back to the LMS via the LTI Gradebook Services, but no detailed usage and performance data (e.g., click stream, number of activities or questions answered, video views or pages read, simulation performance, etc.) that track the student’s learning progress can be sent across multiple systems to inform and trigger academic interventions. Furthermore, these current specifications sometimes do not satisfy the needs of AISs where very fine-grained data and analytics are critical to measure and monitor students’ progress.

Federated Machine Learning.

Another dimension of the cross-AIS interoperability might be borrowed from Federated Machine Learning (FML), one of the lately proposed approaches in aggregating machine learning capability among multi homogeneous or even heterogenous AI systems. FML focuses on using the power of distributed system to train and enhance machine learning models. By training the models on the devices using local data and only transporting the models (i.e. not the data) between devices and the central server, this approach ensures data security and privacy while enabling real-time prediction on local machines with minimal infrastructure [7].

A “Federated” AIS approach can be used to extend the knowledge graph, the learner model, the data reach, and even ML depth by updating, sharing and aggregating different layers or aspects from one AIS to another. In this way, each AIS benefits from the data synchronization, latency, and security features of a federated system while maintaining its own ontology, adaptive engine, and data.

4 Conclusion

In summary, these vertical integrations of components within an AIS and horizontal integrations across AISs seek to address fundamental needs in the development, enhancement, and evaluation of AISs. We believe that it is critically important to (1) create extensions on top of existing IMS Global and xAPI standards to form the foundation for the standardization of AISs, and (2) create a standards framework based on a reference architecture, within which existing standards can be extended (i.e., LTI, QTI, etc.), and new standard sets may be proposed (i.e., federated machine learning, ontology merging, or other AIS system rules).