Keywords

1 Introduction

1.1 Motivation and Learning

Motivation, which refers to a student’s desires, needs and goals in the educational context, is an important factor in learning outcomes. Motivated students have a drive to succeed that helps them learn [1], while students lacking motivation obtain lower levels of mastery and retention. Motivation has been shown to predict the level of learning goals a student will achieve [2, 3]. Motivation has multiple facets that may shape the learning process in different ways. An important categorization of motivation is intrinsic vs. extrinsic motivation [4,5,6] and describes the source of an individual’s motivation. Intrinsic motivation occurs when the source of motivation is an internal desire to achieve based on the interest and challenge of task performance. With respect to learning, a student with intrinsic motivation has an internal desire to acquire knowledge and explore challenging material. The other category is extrinsic motivation in which an individual is motivated by external sources such as monetary reward, career advancement or receipt of a certification.

There are many theories and factors pertaining to an individual’s motivation. In general, motivation theories assume that individuals differ in the way that their motivation is affected by environments, including learning environments [7,8,9,10,11,12]. Reinerman et al. [13] defined a set of motivation variables (see Fig. 1) that contribute to an individual’s motivation. These variables are interrelated such that each individual’s motivation is different. Likewise, each student’s motivation is affected differently to by his or her learning environment. However, identifying an individual’s composition of these variables is difficult because established motivation assessment surveys tend to focus on a limited amount of variables, such as Grit [14] and the 3 × 2 Achievement Scale [15]. Thus, the present research aimed to develop a comprehensive multidimensional assessment, the Motivational Assessment Tool (MAT), through systematic sampling of motivational constructs.

Fig. 1.
figure 1

Motivator variables

Full size image

In classroom learning environments, instructors can interpret verbal and non-verbal signals from their students to gauge motivation of the class in general. Unless they are able to provide one-on-one tutoring, it is not possible to tailor their instructional strategy to support an individual student’s motivational needs. ITSs have the potential to assess, plan, and implement individualized motivational strategies in real-time to optimize the student’s motivation. The challenge is determining a goodness of fit for each individual that is measured by an increase in effort, attention, goal attainment, learning outcomes, and retention within an intelligent tutor.

The motivational questionnaires, such as the Motivated Strategies for Learning (MSLQ) reviewed under this project were developed with the classroom context in mind. Given the differences between a classroom environment (social, open, competitive) and an ITS environment (private, isolated) some of the questions are not relevant in an ITS environment. Also, new features that an ITS environment brings to the learner are not included in the questionnaires that were built for the classroom, such as the availability of hints, remedial materials and real-time, personalized feedback. Also, traditional motivational questionnaires seek to categorize the learner’s motivational traits – relatively fixed characteristics that affect their motivation while learning such as self-regulation and goal-orientation. However, these assessments do not identify the impact of specific items or activities, motivators or reinforcers, that help the learner become or maintain motivated. There are reinforcer inventories built towards school age children and a few for adults, such as the Behavioral Assessment Guide [16] and Dunn- Rankin Reward Preference Inventory [17], but none are used in combination with the general motivation to the complete compilation or for an ITS. Thus, the MAT introduced here will have two sections: a general motivation assessment and the motivator assessment tool for reinforcers.

A blending of both the gaming and classroom needs to occur within the general motivation and the more specific motivators used throughout learning for an IST. This learner profile will begin to shape the relationship needed for the ITS to personalize, adapt, and maintain.

1.2 Motivation Assessment Tool (MAT)

The creation of the MAT began with a literature review in order to identify motivational influences and factors. Then, 31 existing assessments were compiled, overlaps were identified, and constructs were reduced to a basic set. Questions for the scale were arranged by similarities and regrouped until sets of thematically-related questions were sufficiently distinct [13, 18]. This process created the general motivation questionnaire. The motivator inventory was created from existing reinforcer inventories, gaming rewards measures, and definition of new motivators that can be implemented with an ITS. The first MAT had 201 questions for the motivation assessment and 102 questions for the motivator inventory. There were four open-ended questions that could help address any constructs that were missing from the MAT. It was implemented on an intelligent tutoring framework called the Generalized Intelligent Framework for Tutoring (GIFT; 19). Due to the sheer number of items in the MAT, it was broken into 7 sections, 5 for the motivation variables and 2 for the motivator inventory. This iteration of the MAT was then administered as part of a study to support an item analysis for internal consistency and for initial psychometric properties of the scales.

2 MAT Exploratory Analysis

The overarching goal of the study was to develop and refine an initial version of the MAT. This goal was accomplished through three steps. The first was to determine the number of motivational and reinforcer factors present through exploratory factor analysis. The second was to check psychometric properties of scales that were derived from the factor analysis including internal consistency (Cronbach alpha)’s for the correlations within these scales. The third step was to identify higher order factors from a factor analysis of scale intercorrelations. These steps supported a refined version of the MAT.

2.1 Participants and Procedures

The sample size was 200 participants with ages ranging from 18 through 65 (101 Males and 99 females). There were 3 participants in the 65+ age range, 33 in the 46–64 age range, 71 in the 34–45 age range, and 93 in the 18–33 age range. Participants were recruited by the web platform Amazon Mechanical Turk (AMT). They were provided a link to the GIFT [19], which was used to administer and collect the data. Prior to answering the survey items, the participants read over the consent form and acknowledged their decision to consent. Then, they answered the survey items that included demographic questions and the MAT.

2.2 Measures

The demographic questions asked the participants to report their age range, gender, level of education, income range, GPA. The MAT contained 303 items regarding variables related to student motivation and reinforcers based on the initial motivation taxonomy as described previously. Participants responded to each item on a 7-point Likert scale ranging from “strongly agree” to “strongly disagree”.

3 Results and Discussion

3.1 Initial Item Factor Analyses of the MAT

Two exploratory factor analyses were run to identify underlying constructs for the general motivation and reinforcer sections of the MAT, using the principal factor analysis extraction method. For each analysis, the number of factors was determined from the screen test. The initial factor solution was rotated using the direct oblimin method with Kaiser Normalization, which allows factors to inter-correlate.

14 factors explaining 58.4% of the variance were extracted for general motivation, and 11 factors explaining 64.3% of the variance for motivators. Scales corresponding to each factor were derived from inspection of the factor pattern matrices. Items defining each scale were selected from factor loadings ≥.5 where possible, with a minimum loading of .4. A maximum of eight items per scale was chosen.

3.2 General Motivation Scales

Scale Distributions and Alphas.

Table 1 lists the 14 MAT General Motivation dimensions identified by factor analysis, together with their working labels. The dimensions fall into three broad thematic groupings consistent with the existing literature on aspects of motivation. The first group contrasts generally high motivation with vulnerability to loss of focus and interest. The second theme identifies a contrast between positive, approach emotions and social motivations with vulnerability to stress and criticism. The third grouping refers to various aspects of constructive strategy use versus vulnerability to workload.

Table 1. Description of the dimensions for General Motivation
Full size table

Scales were constructed to assess the Table 1 dimensions as previously described. Table 2 shows the descriptive statistics of the scales, together with the provisional labels assigned to them, and their alpha coefficients. Cronbach alpha coefficients were generally acceptable: the median alpha was .803 (range: α = .735–.890).

Table 2. Descriptive statistics of the MAT General Motivation scales
Full size table

Higher Order Scale Factor Analysis.

The scales were themselves correlated. Thus, a second-order exploratory factor analysis was conducted, using the same methods as previously. The scree test suggested that three factors should be extracted. Loadings from the pattern matrix ≥.4 are listed in Table 3.

Table 3. Second order factor analysis of General Motivation scales
Full size table

Factor 1 The first factor is a bipolar factor that contrasts two sets of motivational qualities. There were positive loadings for learning driven and positive outlook scales, identifying learners who are intrinsically motivated, confident, and typically hard-working. At the negative end of the scale are learners who are demotivated by workload and prone to lose effort easily. They are also prone to worry and anxiety, and need supportive feedback. Broadly, the factor contrasts optimistic, resilient, and persistent learners with those who are more fragile, avoidant and vulnerable to loss of motivation.

Factor 2 has three major loadings referring to competitive motivations, needs for social recognition, and performance goals. It also has loadings for being motivated by challenge, and a negative loading for punishment. The factor contrasts individuals who see learning as an arena for outperforming others and receiving due recognition with learners who are indifferent to these social motivations and may require some degree of punishment to become motivated.

The third factor was defined by two loadings, both associated with self-regulation. The MAT self-regulation scale refers to motivations to monitor one’s learning and take remedial action where necessary. The preference for breaks scale may load on the factor because it refers to the learner’s time management skills and capacity to self-regulate focus and alertness.

3.3 Motivator Inventory Factor Analysis

The Motivator Inventory was factor analyzed separately using the same methods described above in the general motivation section. Eleven correlated factors were extracted and rotated, explaining 64.3% of the variance. Items for scale composition were selected on the same principles as before. Working labels and descriptions for the 11 scales are shown in Table 4.

Table 4. Description of the Motivator Inventory Scales
Full size table

Cronbach alphas were generally acceptable ranging from α = .670–.920 (median α = .876). Table 2 shows the descriptive statistics of the scales and their alpha coefficients (Table 5).

Table 5. Descriptive statistics of the MAT Motivator scales
Full size table

Scales were inter-correlated and a second-order exploratory factor analysis was conducted. Three correlated factors were extracted, explaining 68.2% of the variance. Table 6 shows the loadings from the factor pattern matrix that define each factor.

Table 6. Second Order Factor Analysis of the Motivator scales
Full size table

The Pattern Matrix identified three underlying factors that can be summarized as:

Factor 1: Items related to different types of non-tangible and tangible motivators to include rewards, feedback and interactivity of the learning content

Factor 2: Items that describe types of activities and exercising that can be used as motivators

Factor 3: Items that describe tangible motivators such as money or time to do something the learner enjoys

The first factor explained considerably more of the variance than the remaining two, and corrected with factors two (α = –.449) and three (α = .469) correlate with the first general factor, which were independent of each other. Thus, the first factor tended to reflect overall sensitivity to reinforcers, but factors 2 and 3 picked up on more specific influences that cannot be fully separated from the general sensitivity. For example, if a learner is highly motivated by opportunities to play sports (factor 2), this may reflect both general reward sensitivity and a more specific interest in sports.

3.4 Discussion

The present research aimed to explore the dimensional structure of learner motivation using an empirical approach based on comprehensive sampling of relevant constructs identified in the literature. It aimed also to distinguish general motivation dimensions associated with the personal characteristics of the learner from dimensions associated with sensitivity to specific reinforcers. Exploratory factor analyses suggested that multiple dimensions of both general motivation and sensitivity to motivators may be distinguished. Motivation dimensions can be thematically grouped in relation to existing constructs, but there appear to be psychometric distinctions that are not well represented in the existing literature. It also proved possible to develop psychometrically acceptable scales for learner assessment on the basis of the factor-analytic findings. Higher-order factor analyses suggest that broader clusters of motivational attributes may also be defined.

Assessment of Motivational Factors in Support of ITS.

The first-order factor analysis distinguished a variety of constructs familiar from the research literature e.g., [7, 9, 15]. The three thematic groupings of dimensions identify different types of motivational strengths and weaknesses. Motivation is enhanced by interest in learning and drives to achieve, by sensitivity to social factors, and by self-directed organization of the learning process. Conversely, motivation may be undermined by difficulties in sustaining effort, vulnerability to worry and threats, and poor management of task strategies. All three groupings are relevant to the intelligent tutoring context. For example, three key design issues are (1) how to maintain learner interest and engagement, (2) how to build a sense of being part of a learning community without social threat, and (3) how to enhance the learner’s capacities to structure material and utilize effective strategies.

The factor analysis supported development of acceptably reliable scales for general motivation dimensions that can be used to profile the learner’s motivational strengths and weaknesses. The second-order factor analysis, based on the inter-scale correlations, revealed a somewhat different factor structure to that suggested by the conceptually-based thematic groupings. Factor 1 pulled in elements of intrinsic motivation and effort, but also emotional concomitants of emotion including a positive outlook, and, at the low end of the scale, excessive worry. It could be seen as contrasting approach and avoidance, seeing these constructs as opposites, rather than independent aspects of motivation as in some theoretical accounts [15]. Factor 2 blends two elements of social motivation – competitive striving against others to excel in performance, and having one’s accomplishments be explicitly recognized by others. Factor 3 represents a more narrowly defined version of the task strategy thematic grouping, focusing on self-regulation and taking breaks. Notably, the first-order organization and structure scale did not load substantially on any factor, suggesting that more cognitively-infused elements of task strategy such as note-taking may be distinct from the broad motivational complexes identified in the higher-order analysis.

The present study was directed towards learner characteristics, and assessment of the motivational dispositions that individuals are likely to bring towards a learning environment. Such assessments can then guide personalization of learning. Using the motivation instructional approaches developed in our Phase I research and seen in Table 7 [13], learners that score high on Factor 1 would receive an instructional plan that provides them with control and choices in their learning to feed their intrinsic motivations. Low scorers would follow a plan that minimized excessive workload and stress, and provided frequent supportive feedback. Factor 2 identifies the challenge for online learning environments of limited social interaction. High scorers may require programming of features to support comparison with others and perhaps the ability to communicate with other learners. The role of social motivations also illustrates how discrimination at the finer-grained first-order level can supplement the ‘big picture’ provided by higher-order factor scores. Socially-motivated learners high in competition but not social recognition might require features such as leaderboards, target scores and head-to-head competitions. However, those high on social recognition but not competitiveness might require displays and badges acknowledging their accomplishments without necessarily having to excel against others. Finally, high scorers on Factor 3 might be provided with a variety of aids to self-regulation and refocusing breaks, given that they will be motivated to explore which aids are most helpful to them. Low scorers, lacking motivations to self-regulate, might be given more explicit direction to perform exercises that enhance focus or evaluate progress.

Table 7. Examples of motivational adaptations in an ITS
Full size table

One of the messages of motivational research is that some learners need more support than others. The general section of the MAT identifies various motivational strengths that will sustain learning even in challenging conditions. A design challenge for online environments, including intelligent tutoring, is how to detect and mitigate lack of motivation or maladaptive motivations such as avoidance. The motivator section of the MAT may be especially valuable in supplementing the general assessment in those cases where motivation is compromised. That is, it may identify the external reinforcers most effective for the individual in those cases where motivation is deficient.

The initial factor analysis of the motivators identified 11 types of reinforcers to which people are more or less sensitive, allowing rewards for participation or achievement to be maximized on an individual basis. The second-order factor analysis of the 11 scales suggested that most of these reinforcers loaded on the first factor. Thus, individuals may differ in general reward sensitivity consistent with psychobiological accounts of variation in the Behavioral Activation System that supports positive reinforcement [22]. However, access to sports and activities, and access to money and leisure time were shown to be somewhat separate classes of reinforcer that may be especially motivating for some individuals. As with the general motivation, assessment of sensitivity to motivators at both first- and second-order levels may support strategies for optimizing motivational support online, especially in those learners that lack intrinsic motivation, competitive drives or effective self-regulation.

Future Research Directions.

Identification of missing motivational constructs that are influential for an ITS were constructed to increase the collective depiction of the learner.

The analysis of the subscales produced were discussed and compared to motivational factors to address gaps that may cause a incomplete picture of the learner. Constructs that did not cluster together were added or rewritten. While autonomy is often cited [20] as an attribute of an intrinsic motivated learner, the assessment items in the autonomy section did not cluster together. For our second wave of development and analysis, these questions were reworded to increase the chances of clustering. The Self Determination Theory [21] of an intrinsic learner is founded upon choice and responsibility. The wave 2 analysis will determine if autonomy and level of control will factor within the larger scope of intrinsic motivation. Test anxiety, fear, and fight or flight response with feelings of fear and anxiety were not included and missing constructs for the MAT. The Reinforcement Sensitivity Theory (RST) [22] fit the missing scales and learning questions were constructed around the RST theory scales for the general motivation section.

Motivator Inventory also constructed new scales for addressing additional motivators available for the ITS. A sensor construct was added due to the increasing demand to monitor learners through real-time measures. Sensors can be used as a motivator to increase a drive or for others it may demotivate them due to their sensitivity and anxiety that causes a learner to avoid or shut down. Time was also added in the area of time during learning and negative time. Levels of Interactive Multimedia Instruction (IMI) were added to provide a degree of sensitivity to the type of task a learner prefers from passive tasks, such as PowerPoint to high levels of interaction with a simulated task. A level of support or frequency and extinction was added to measure an overall level of feedback, points, etc. to motivate the learner. It also provides a general gauge when a motivator may need to be changed because of extinction in order to maintain motivation. Some learners need more frequent positive feedback or points than others. Feedback was divided further to address the different types of feedback and the amount of feedback the learner needs. This study shows that individuals vary in degree of sensitivity to motivational factors. This sensitivity provides multiple pathways for an ITS to adapt and personalize in congruence and extend beyond cognition. The MAT addressed emotional trait characteristics. It seeks to work in congruence with affect, cognition, and traits. Perhaps, there is overlap with trait characteristics that will provide stability for learning motivation and the degree of sensitivity. However, it might be a more fine-grained process divided beyond the categorization of traits. This will be sought after in future validation studies of the MAT.

4 Conclusion

The continuing debate about external motivators and its effect on intrinsic motivation [21] is perhaps, addressed further through the MAT. Specific variables appear to serve as motivators (an external thing that motivates). The correlations for the MAT factor only partially overlapped, which indicates that there may be other motivators outside of external rewards, such as adding challenge to a learning task. Additionally, the results indicate that indeed some individuals are more sensitive to external motivators than others. One general motivator provided is not specific enough to maintain the individual learner properly thus, showing mixed results for external factors when learning. Additionally, perhaps it is more than just one motivator to address an individual needs. A complete system of tailoring externally and internally to achieve a perfect fit, may demonstrate different results. Research generally sticks to one or two controlled variables and not to a more real world messy application of multiple factors in combination that need to be explored due to complexity and furthering our understanding of learning and motivation. Categorizing learners or the fine-grained adaptations is the gateway to determine the true effect of extrinsic variables on an individual. Motivators seem to effect both the driven learner and those that fall on the continuum of extrinsic tendencies as seen in the analysis. Findings have implications for an ITS based on the level or degree of sensitivity or the combination of motivators needed for a learner to perhaps increase learning and retention. The second order factor analysis identified a number of motivation variables found in prior research on motivation and learning, and included in the motivation taxonomy, that can be used to inform the use of a specific instructional strategies designed to support the learner’s motivation. In conclusion, an overall comprehensive motivational depiction is comprised of various interrelated variables. These variables affect individuals at different levels of sensitivities that allows for an ITS to deliver adaptive personalized instruction. Adapting to the intertwinement of variables is an enhancement for optimizing a learner’s motivation.