Keywords

1 Introduction

Creating an optimal assortment of products is one of the most basic yet critical decision assortment planners must make for retailers [1]. In order to create an optimal assortment of products, assortment planners need to take into consideration qualitative and quantitative criteria [2]. These criteria include a great number of variables (e.g., past sales, retail trends, inventory, customers’ needs, sales forecast) that must be taken into account by the assortment planners, which creates a certain level of uncertainty. This level of uncertainty results from the important amount of information that needs to be considered by the assortment planners throughout their decision-making process. Hence, the information load could negatively impact the assortment decision quality [3] which could lead to losses in both current and future sales due to customers exits [1].

To reduce the impact of these trade-offs, artificial intelligence (AI) based recommender systems can be used as an aid by assortment planners throughout their decision-making process [4]. By processing a large quantity of decision relevant information, AI based recommendation agents (RAs) can help assortment planners define an optimal assortment more easily [5]. Though AI based RAs are becoming more present in the workplace, most research on RA adoption has focused on consumer adoption, not employee adoption [6, 7]. Understanding the way professionals use and perceive RAs, and more importantly, what RA characteristics encourage their adoption and continuous usage will contribute to advancing knowledge in the human-computer interaction (HCI) literature. In addition, it will inform the human interaction community on how to best present decision recommendations in order to create better RA interactions.

Therefore, the main objective of this study is to investigate how assortment planners’ usage behavior and perceptions of RAs are influenced by the way recommendations are presented. This study contributes to theory on RA adoption in B2B contexts and has implications for RA developers by providing insights on features that would enhance RA adoption by employees.

2 Development of Hypotheses

In the RA literature, two main factors have been investigated [8, 9]. The first factor, information richness, is associated with the amount of information provided by the RA to assist assortment planners throughout their decision-making process. The scientific literature shows that user acceptance towards RAs increases with perceived transparency [10]. The perception of transparency is recognized when the logical reasoning behind an RA is explained [11, 12]. The need for explanations and justifications are specifically triggered when users are supporting their decisions through knowledge-based systems [13]. The second factor, effort, is related to the number of steps required to get to the information (e.g., number of screens). Thus, in a high effort condition, the number of steps required to get to the information is greater than in a low effort condition. In addition to processing information, assortment planners need, in order to make an assortment decision, to gather the decision-relevant information which can lead to information load [10]. Due to the limited capacity of individuals to process information, information overload can result in cognitive fatigue and confusion [14]. Hence, processing and obtaining a large quantity of information can negatively affect the assortment planners’ decision-making process by reducing the quality of their decisions [3].

According to the literature, explanations about the recommendations presented have been demonstrated to positively influence the RA’s perceived credibility [15], satisfaction [16], and performance (i.e., decision quality) [8]. In this study, the source credibility dimensions of trustworthiness (i.e., the recommendations of the RA are identified by the users as reliable) and expertise (i.e., the RA is recognized by the users as knowing the right answer) have been observed [17]. In order to establish trust and show the expertise behind the recommendations of the RA, transparency of the recommendation process is crucial [18]. Explaining this process increases users trust in the RA’s recommendations [10]. Furthermore, the perceived ease of use of the RA is also related to RA trust [19]. According to Pereira [20], the cognitive load of gathering and processing information can negatively impact the perceived satisfaction and performance of the decision-making process. Therefore, finding a balance between information richness and effort seems to be crucial.

H1: Assortment planners’ perceptions toward the RA regarding credibility (H1a), satisfaction (H1b), and performance (H1c) will be more positive when the RA’s recommendations include easily accessible explanations.

Moreover, trust towards RAs is needed to foster their adoption, but trust in RAs is difficult to build [21]. Initial trust is thus essential in influencing users’ continuous usage [22, 23]. However, users experiencing information overload, even in the absence of trust, are likely to consult the RA’s recommendations more frequently [9]. Thus, accepting the recommendations of the RA due to cognitive fatigue of confusion [14].

H2: Assortment planners will consult each recommendation of the RA more frequently when the RA’s recommendations are enhanced with explanations that are difficult to access.

Research also shows that when trust is built, the risk perceived in adopting the RA’s recommendations is then diminished [23]. Hence, when RA trust increases, the usage of the RA follows [24]. Without knowledgeable explanations that are perceived as credible by the users, the recommendations of the RA are expected to be ignored [19, 25]. Therefore, understanding the logical reasoning behind the recommendations of the RA seems to be important for users.

H3: When the RA’s recommendations are explained, assortment planners will allocate their visual attention more towards the information explaining these recommendations at the beginning of the decision-making process, rather than at the end.

According to the literature, the presence of information load diminishes users’ reluctance towards using RA recommendations [9]. By perceiving the usefulness of the RA’s recommendations through explanations, users are more inclined in adopting these recommendations during their decision-making process [19, 26]. The users’ intention in adopting the RA also increases when RA credibility is perceived [21, 23, 27]. Hence, users are known to prefer transparent recommendations to non-transparent recommendations [18].

H4: Assortment planners will have a higher intention of adopting the RA throughout their decision-making process (i.e., RA used as a delegated agent or as a decision aid) when the RA’s recommendations include explanations.

Furthermore, the important number of trade-offs made by the assortment planners during their decision-making process, trying to balance a great number of variables (e.g., past sales, retail trends, inventory, customers’ needs, sales forecast), indicates that the product assortment created by an assortment planner will always vary from one assortment planner to another [1]. Consequently, an optimal product assortment might not be fully optimal, which in turn diminishes the performance of the assortment planners. The cognitive load of gathering and processing information can negatively impact the quality of the decisions taken by users [20]. With the help of RAs, this cognitive load can be reduced through recommendations [28]. However, in order to increase the quality of the decisions taken by the users, the RA’s recommendations need to be considered [20, 29].

H5: When the RA’s recommendations are complemented with easily accessible explanations, assortment planners will have a higher performance (i.e., decision quality).

3 Method

To test our hypotheses, a within-subject laboratory experiment was conducted. A total of twenty logistics and marketing professionals (Mage = 26, SD = 3.92; 9 women) participated in the study. During the experiment, participants used the experimental RA prototype for assortment planning developed by JDA Labs (Montreal, Canada). Overall, the experiment lasted two hours and each participant received a $30 gift card as a compensation. This project was approved by the Institutional Review Board (IRB) of our institution and each participant completed a consent form.

3.1 Experimental Design and Protocol

Participants had to make assortment decisions in six tasks and were allowed to take as much time as they needed for each task (about 5 min per task). These tasks were divided into two similar fictitious scenarios that were counterbalanced, three tasks per scenario. The three tasks of each scenario were counterbalanced and each task was exposing participants to a particular recommendation representation condition based on the two experimental factors (i.e., effort required to access information and information richness). Task 1 represented a low richness & low effort condition (T1), Task 2 reflected a high richness & low effort condition (T2), and Task 3 was a high richness & high effort condition (T3). Due to time constraints in the experiment and to avoid participant fatigue, the low richness & high effort condition was not included in the experiment. In order to familiarize participants with the assortment planning software, each scenario began with a practice task with no RA.

A total of 24 distinctive products per task were presented to participants for each scenario (24 products × 6 tasks). Participants were required to make an assortment decision by selecting the optimal assortment of products from the 24 displayed products (see Fig. 1). Each scenario specified the total number of products (ranging from 6 to 7) that needed to be selected for each task. Figure 1 represents the elements that were displayed for each product and in each condition to the participants. The product score, RA’s recommendation, was generated using AI. This product score varied between 0 to 100 and was surrounded by a circle that changed colour depending on the score number (i.e., green > 66, 66 ≥ orange > 33, and red ≤ 33).

Fig. 1.
figure 1

For each task, 24 products were presented to the participants. Each product was presented with an image, its name including its brand, and its product score (i.e., RA’s recommendation). (Color figure online)

Full size image

For T1 (low richness & low effort), the product score represented in Fig. 1 was the only source of information made available to the participants. For T2 (high richness condition & low effort) and T3 (high richness & high effort), the product score was also made available to participants, however, participants could also acquire further product information by clicking on each product. Additional information included various product characteristics, e.g., attributes, past sales, margin and comparative products. The effort required to access additional information varied between T2 and T3. For T2, the information was made available through a modal window (see Fig. 2). As for T3, accessing the information necessitated additional navigation through a different page, thus requiring more effort from participants (see Fig. 3).

Fig. 2.
figure 2

For T2, each product had a modal window that was used to access the explanations of its product score (i.e., RA’s recommendation).

Full size image
Fig. 3.
figure 3

For T3, each product had a new page presenting an additional layer of difficulty to access explanations of the RA’s recommendation.

Full size image

3.2 Apparatus and Measures

The experimental prototype for assortment planning by JDA Labs (Montreal, Canada) was made available to the participants through a 1680 × 1050 resolution monitor. This prototype was developed with Axure RP 8. All statistical analyses were performed with SAS 9.4.

Psychometric Measures.

After each task, participants completed a questionnaire. This questionnaire used validated measurement scales to assess the participants’ perceptions towards the RA regarding credibility [17], satisfaction [30], and type of future usage (i.e., RA used as a decision aid or as a delegated agent [24]). Participants were also asked to rate their perceived task performance (from 1 to 10).

Performance Measures.

The performance of the participants was measured exclusively for the first scenario, because JDA Labs was only able to provide predetermined optimal assortments for this scenario. The guidelines of the first scenario, combined with all the additional information made available to the participants, led to a predetermined optimal assortment for each task. Each final assortment, for each task and each participant, was compared to the predetermined optimal assortment. When a product in the final assortment of the participant was also in the predetermined optimal assortment, one point was given. For each participant and each task, results were calculated as percentages. Thus, a performance score was created.

Behavioral Measures.

During each task, the visual attention of the participants was captured at a 60 Hz sampling rate with a Smart Eye Pro system (Gothenburg, Sweden). A 9-point calibration grid was used. For each participant, calibration was repeated until sufficient accuracy was obtained (± 2 degrees of accuracy). The MAPPS 2016.1 software was used to analyse the eye tracking data. For each RA product score (see Fig. 1) and for each modal window or new page with additional information that was consulted by the participants, areas of interest (AOIs) were generated. The number and duration of ocular fixations, for each AOI, were collected. An ocular fixation was accepted at 200 ms [31] (Fig. 4).

Fig. 4.
figure 4

Experimental set-up.

Full size image

4 Results and Analysis

H1 stipulates that the assortment planners’ perceptions toward the RA regarding credibility (H1a), satisfaction (H1b), and performance (H1c) will be higher in the high richness & low effort condition. A linear regression with random intercept and a two-tailed level of significance adjusted for multiple comparisons was used to test the difference between the means of assortment planners’ perceptions for each combination of conditions (see Table 1). First, results suggest that the perceptions of the participants toward the RA regarding credibility and satisfaction are positively affected when information on the different variables included in the calculations of the RA’s recommendations is present (T2 and T3 greater than T1 with one-tailed level of significance, respectively 0.7055, p ≤ .0001 and 0.6139, p ≤ .0001; 0.6068, p = .0027 and 0.5103, p = .0077). However, no difference was found between the high richness conditions (T2 and T3) and the low richness condition (T1) for the participants’ perceived performance toward the RA. Second, although results show that the assortment planners’ perceptions toward the RA in terms of performance is impacted negatively by the effort required to access information (T2 greater than T3 with one-tailed level of significance, 0.4706, p = .0474), no difference between high richness & low effort (T2) and high richness & high effort (T3) was found on the perceptions of the participants toward the RA regarding credibility and satisfaction. Hence, H1a, H1b, and H1c are partially supported.

Table 1. Participants’ perceptions results
Full size table

In order to test H2, a Poisson regression with a mixed model adjusted for multiple comparisons with a two-tailed level of significance was performed. Thus, the difference between the least square means of the number of fixations for the AOIs of the different conditions was tested (i.e., the number of fixations on all the product scores of one condition versus another). Results revealed that participants, when in the high richness & high effort condition (T3), are consulting the product scores more frequently than when in the low richness & low effort and high richness & low effort conditions (T1 and T2), which provides strong support for H2 (T3 is greater than T1 and T2 with one-tailed level of significance, respectively 1.1617, p = .0054; 0.7471, p = .0225).

We also hypothesized that assortment planners, when in a high richness condition (i.e., high richness & low effort and high richness & high effort), will allocate their visual attention more towards the information explaining the different variables included in the product score (i.e., RA’s recommendation) at the beginning of the decision-making process, rather than at the end (H3). To test H3, the difference between the first 25% and the last 25% of the number of fixations and the task duration for each AOI of each condition was compared by using a Wilcoxon signed rank test with a two-tailed level of significance. Results revealed no difference through time concerning frequency and the time spent by the participants on the RA’s recommendations. In addition, results showed that participants, when in the high richness & low effort condition (T2), consulted more frequently and for a longer period of time the information on the different variables included in the calculations of the RA’s recommendations at the beginning of the task (p = .0137 and p = .0171, respectively with a one-tailed level of significance). Furthermore, results also revealed that when in the high richness & high effort condition (T3), participants consulted this additional information for a longer period of time, but not more frequently, at the beginning of the task (p = .0279 with a one-tailed level of significance and p = .1104, respectively). These results were also in line with a similar test that was conducted to compare the difference between the first 40% and the last 40% of the number of fixations and the task duration for each AOI of each condition. Hence, these results partially confirm H3.

Moreover, the difference between the means of assortment planners perceived intention to adopt the RA throughout their decision-making process (i.e., RA used as a delegated agent or as a decision aid) for each combination of conditions was analyzed with a linear regression with random intercept and a two-tailed level of significance adjusted for multiple comparisons (see Table 2). Results show that, when in a high richness condition (i.e., high richness & low effort and high richness & high effort), participants had a higher intention of adopting the RA as a delegated agent and as a decision aid than when in the low richness & low effort condition (T2 and T3 are greater than T1 with one-tailed level of significance, respectively 0.8503, p = .0001; 0.5647, p = .008 and 1.0705, p = .0001; 0.9681, p = .0002). These results confirm H4.

Table 2. Participants’ intention to adopt the RA for future usage results
Full size table

To compare the difference between the intention to adopt the RA as a delegated agent and the intention to adopt the RA as a decision aid for each condition, a Wilcoxon signed rank test with a two-tailed level of significance was conducted. For all three conditions, results propose that participants are more inclined to adopt the RA as a decision aid than as a delegated agent (T1: p ≤ .0001; T2: p ≤ .0001; T3: p ≤ .0001).

H5 specifies that the assortment planners’ performance will be higher in the high richness & low effort condition compared to the other two conditions (i.e., low richness & low effort and high richness & high effort conditions). In order to test this hypothesis, a linear regression with a mixed model adjusted for multiple comparisons with a two-tailed level of significance was used. Results show that participants had a higher performance in the high richness & low effort condition, which confirms H5 (T2 is greater than T1 and T3 with one-tailed level of significance, respectively 1.0500, p = .0001 and 0.2330, p = .0001). Furthermore, no difference between low richness & low effort (T1) and high richness & high effort was found (T3) (Table 3).

Table 3. Participants’ performance results
Full size table

To explore different AOIs which most attracted the visual attention of the high-performance and the low-performance participants, a Poisson regression with a mixed model, and a linear regression with a mixed model, both with a two-tailed level of significance adjusted for multiple comparisons, were performed. Thus, testing the difference between the least square means of the number of fixations and the duration of the AOIs for the different conditions for each level of performance. Results revealed that when in a high richness condition (T2 and T3), high-performance participants consulted the product scores more frequently but not for a longer period of time (T2 and T3 greater than T1, respectively 1.6405, p ≤ .0001 and −0.5500, p = 0.2052; 1.6323, p ≤ .0001 and −0.8942, p = 0.1145). In addition, no difference between high richness & low effort (T2) and high richness & high effort (T3) was found concerning the frequency, or the period of time in which the information explaining the different variables included in the product score was consulted. Compared to the high-performance participants, the low-performance participants spend the same amount of time and consulted the product scores as frequently throughout the different conditions. Moreover, a significant difference between high richness & low effort (T2) and high richness & high effort (T3) was found concerning the frequency but not the period of time in which the information explaining the different variables included in the product score was consulted (0.7539, p = 0.005 and 0.1535, p = .7379), thus indicating the effort required to access information impacted the low-performance participants.

5 Discussion and Concluding Comments

Results revealed that: (1) an RA providing rich information is perceived as more credible (H1a) and satisfactory (H1b); (2) users’ perceived performance is negatively influenced by the effort required to access information (H1c); (3) the RA’s recommendations are consulted more frequently by the assortment planners when the product scores are enhanced with difficult to access explanations (H2); (4) users are consulting consistently the product scores throughout their decision-making process; (5) when the RA provides additional information, planners are consulting this information more often at the beginning of their decision-making process (H3); (6) the intention to adopt the RA significantly increases with the richness of information, thus indicating that assortment planners seem to favor the recommendations of an RA when they are enhanced with additional information about the variables included in their calculations (H4); (7) users are more willing to adopt the RA as a decision aid than as a delegated agent; (8) the performance of the users is significantly higher when the RA provided rich information that was easily accessible (H5).

This paper’s main theoretical implication is related to the advancement of RA adoption in B2B contexts. As AI is becoming more common in the workplace [32], it is essential to understand how to best present AI based recommendations in order to positively influence professionals’ usage behavior and perceptions toward a RA. The findings in this study show the importance of RA’s recommendations that are enhanced with easily accessible information. A RA providing rich information on the variables included in the calculations of its recommendations (i.e., product scores) increases assortment planners’ perceptions toward the RA regarding credibility and satisfaction. These findings are in line with the literature focusing on the RA adoption of consumers [15, 16]. Furthermore, the intention of the professionals to adopt this RA also increases with information richness. However, planners seem more willing to adopt the RA as a decision aid than as a delegated agent. Though in a consumer RA adoption context, this could be explained by the importance of a product purchase [24]. In a professional context, assortment planners could feel that following only the RA’s recommendations in their decision-making process would diminish their performance, thus negatively affecting the quality of their assortment decisions [33]. Moreover, the RA’s recommendations, when enhanced with additional information that is accessible through an increased effort, are consulted more frequently by the users. This indicates the degree of importance that is given by the assortment planners to the product scores [34] when experiencing information overload [9]. This could also explain why the planners perceived a higher performance when the additional information was easily accessible. In addition, the performance (i.e., decision quality) of the assortment planners increases with information richness and decreases with the effort to access this information.

This study also has implications for RA developers and UX designers. First, results show that professional users need to have access to information about the variables included in the calculations of the RA’s recommendations in order to increase perceived credibility and satisfaction. That being said, the tremendous amount of data behind the AI based recommendations needs to be brought forward to the users in a condensed intuitive form [35]. Hence, a visual representation of this condensed form must be created by designers. Second, this additional information seems to be a key element in the adoption of a RA. Results revealed that assortment planners consulted the additional information that was easily accessible to them more frequently and for a longer period of time at the beginning of their decision-making process. This usage behavior indicates that the degree of importance [34] and cognitive engagement [36] toward this information decreases over time, emphasizing the importance of initial trust. Explaining the recommendation process increases professionals’ initial trust towards the RA’s recommendations, which then influences positively their continued usage of this RA [10, 22]. These insights can contribute to building best practices in UX design for AI.

Before applying these results, two limitations of this study need to be acknowledged. First, the low richness & high effort condition has not been tested in this experiment, due to time constraints and to avoid participant fatigue. Thus, future research should include this condition in order to extend the results of this study. Second, the two fictitious scenarios, in this experiment, informed participants that they were working for a fashion company which entailed that clothes were used as products (i.e., dresses and male upper body clothing). Hence, additional studies using a more complex category of products could help generalize the results of this study.

In conclusion, more research is needed to better understand the way RAs are used by professional users. For example, emotional and cognitive state at fixation on RA could be assessed using techniques proposed by Léger et al. [37] and Courtemanche et al. [38]. These additional studies could provide further guidelines to RA developers and UX designers. Results of this study revealed that the usage of the assortment planners changed throughout their decision-making process, which confirms that user behavior changes over time [39, 40] However, understanding how the user-RA relationship changes in terms of behaviors and perceptions over time could show that the explanations of the RA’s recommendations become decreasingly significant and indeed unnecessary after a while, with the users then relying predominantly on the RA’s recommendations to make decisions.