Novice digital service designers' decision-making with decision aids — A comparison of taxonomy and tags
Introduction
Design plays an essential role in all sectors. Design-oriented companies are characterized by higher revenue growth than their benchmark [1]. As we transition toward a digital society, face-to-face contacts are increasingly replaced by digital and human-free services [2]. Given the importance of design, digital service providers benefit from implementing design-centric processes within their organization. Specifically, selecting and applying adequate design techniques are influential tasks, as design techniques guide the design process by providing a set of necessary steps to accomplish a specific design goal [3]. However, hundreds of different design techniques are available [4]. Each design technique has specific design conditions that need to be met to take full advantage of it. Given the increasing amount of design techniques, particularly novice designers1 are confronted with too much choice, which leads to higher complexity and making the selection of suitable design techniques increasingly difficult [5].
Much research has been done on the development and use of decision aids to assist individuals in making choices [6]. An example is e-commerce [7] leveraging decision aids to support customers in the selection of products, whereby product attributes are classified into predefined categories to reduce the number of alternatives [8]. Scholars have investigated decision aids in manifold contexts, such as e-commerce [9], people-to-people loans [10], benchmarking tools [11], product recommendations [12], and medical diagnoses [13]. Decision aids have various representation formats, such as decision trees [14], rules [8], tables [15], tags [16], and taxonomies [17]. In the context of loan application, the comprehensibility of decision trees, rules, and tables has been compared and evaluated based on their performance implications [18]. In the context of library collections, tags (i.e., a classification based on a flat hierarchy) have been investigated relative to their similarity and difference with expert-based classifications to evaluate the applicability of social tagging for library systems [19]. In the context of e-commerce, product taxonomy (i.e., a classification based on dimensions and characteristics) has been investigated to suggest customized categories [17]. However, there is a research gap concerning the question of how taxonomy-based or tags-based decision aids differ with regard to task performance when considering the individual characteristics, such as decision style, of the decision-maker.
We drew on cognitive fit theory [20] as the theoretical foundation for our research on the effect of decision aids on task performance. Specifically, we investigated cognitive effort as a mediation variable, and we found that structural differences between taxonomy-based and tags-based decision aids affects individual's cognitive efforts and task performance. In addition, we suggest that individuals with diverse decision styles use the available information during the decision-making process in different ways when conducting a selection task [21]. Previous research has investigated the direct influence of intuitive and rational styles on choice quality [22] and task performance [23]. However, decision style is an aspect often overlooked by previous research on cognitive fit theory. Thus, we also investigated individuals' intuitive and rational decision style when using different decision aids, and we formulated the following research questions:
- 1)
How do taxonomy-based and tags-based decision aids affect novice designers' cognitive effort and selection accuracy when selecting design techniques?
- 2)
How do decision styles influence the relationship between decision aids and selection accuracy?
We conducted a laboratory experiment with 195 subjects to answer these questions. Our work provides several contributions. First, we advance research by constructing a nomological network around the use of decision aids, thus integrating research on information systems and human-computer interaction. Second, given the large number of design techniques and their importance for organizations, we advance the understanding of the use of decision aids and decision styles for selection tasks (e.g., [22]). Third, we extend previous research on cognitive fit theory by highlighting the importance of cognitive effort as well as rational and intuitive decision styles [24]. Finally, practitioners can benefit from our results on the effectiveness of decision aids for novice designers when considering their decision styles and cognitive effort. Beyond this specific context, we believe that our work can also inform the design of classification-based decision aids in general.
This paper is structured as follows: First, we introduce the conceptual foundations of decision aids, cognitive effort, and decision styles. Second, we suggest our research model and develop testable hypotheses. Third, we explain our research approach (i.e., the design of the laboratory experiment). Next, we present our data analysis and results. Finally, we discuss our results, contributions, and limitations.
Section snippets
Decision aids, cognitive effort and decision styles
Different decision aids — interventions that help people make decisions and improve decision quality [8,11] — to support design novices making decisions have been developed. We focus on taxonomy-based and tags-based decision aids because of the lack of research on comparing the performance between them. When referring to a taxonomy-based decision aid, we suggest that the decision aid is based on a set of dimensions that includes a set of mutually exclusive and collectively exhaustive
Hypotheses development
Cognitive fit theory explains the relationship between individual problem-solving skills, problem representation, and specific tasks. The theory has been applied to analyze individuals' problem-solving performance when using different information-presentation formats (graphs versus tables) [20,37]. An extension of cognitive fit theory suggests distinguishing between an internal and external problem representation, as both contribute to an individual's mental representation [38]. Internal
Research method
In order to test the hypotheses, we conducted a controlled laboratory experiment to gather replicable data following the guidelines suggested by Marsden and Pingry [61]. In this section, we introduce time, location, and the process relative to data collection, the experimenters and subjects involved in it, as well as the treatments and tools used in the experiment.
Data analysis and result
We used ordinary least square regression analysis to test our hypotheses (H1–H4b). Following our hierarchical regression analysis, we used Kruskal-Wallis test and Mann-Whitney U test for a detailed analysis of H1 and H2 by comparing the differences of selection accuracy and cognitive effort across the three experimental groups (a taxonomy of design techniques, tags of design techniques, or a list of design techniques). For H4b, as the R2 of the regression model with the moderating effect of
Discussion
We constructed and tested a research model describing the impact of two important decision aids on selection accuracy while accounting for cognitive effort and different decision styles. Our work thus provides several contributions to the literature. First, it contributes to the nomological net of decision aids within the specific context of designing digital services [93]. While previous research investigated taxonomy-based [17] and tags-based decision aids [16] independently from each other,
Conclusion
Decision aids can shape and influence the accuracy of decision-making in selection tasks. However, the relationship between decision aids and the accuracy of decision making is complex. In this study, we compared two design choices of decision aids in the context of novices' selection of design techniques. We developed hypotheses based on the cognitive fit theory. A between-subject laboratory experiment was designed and conducted. The use of a taxonomy-based decision aid demonstrated a positive
Acknowledgements
We would like to thank the anonymous reviewers for their insightful and detailed comments in the reviewing process. We also thank the participants of the Cologne Institute for Information Systems research seminar participants for their feedback on an earlier versions of the manuscript. Thanks also to Prof. Shirley Gregor for her feedback on hypotheses and experiment plan. Xuanhui Liu also thanks the China Scholarship Council (CSC) for providing her scholarship (ID: 201508210181) for doing
Xuanhui Liu ([email protected]) is a post doc at the Zhejiang University. Xuanhui received her doctorate degree in information systems at the Karlsruhe Institute of Technology, Institute of Information Systems and Marketing. Her research interests include digital service design and used-centered systems. Her research has been published in International Conference on Information Systems, European Conference on Information Systems, etc.
References (96)
Method engineering: engineering of information systems development methods and tools
Inf. Softw. Technol.
(1996)- et al.
Borrower decision aid for people-to-people lending
Decis. Support. Syst.
(2010) - et al.
An ELECTRE-based decision aid tool for stepwise benchmarking: An application over EU digital agenda targets
Decis. Support. Syst.
(2014) - et al.
Consumer-based decision aid that explains which to buy: Decision confirmation or overconfidence bias?
Decis. Support. Syst.
(2012) - et al.
Introducing functional classification theory to land use planning by means of decision tables
Decis. Support. Syst.
(2009) - et al.
Retail business analytics: Customer visit segmentation using market basket data
Expert Syst. Appl.
(2018) - et al.
An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models
Decis. Support. Syst.
(2011) - et al.
An empirical study of mobile commerce in insurance industry: Task–technology fit and individual differences
Decis. Support. Syst.
(2007) - et al.
The influence of decision aids on choice strategies: An experimental analysis of the role of cognitive effort
Organ. Behav. Hum. Decis. Process.
(1994) - et al.
The predictive validity of the decision styles scale: An evaluation across task types
Pers. Individ. Dif.
(2017)
Intuitive vs. analytical decision making: Which is preferred?
Scand. J. Manag.
Personality, decision-making styles and investments
J. Behav. Exp. Econ.
Less may be more when choosing is difficult: Choice complexity and too much choice
Acta Psychol.
A componential analysis of cognitive effort in choice
Organ. Behav. Hum. Decis. Process.
Information representation in decision making: The impact of cognitive style and depletion effects
Decis. Support. Syst.
On the cognitive process of human problem solving
Cogn. Syst. Res.
Cognitive, demographic, and situational determinants of service customer preference for personnel-in-contact over self-service technology
Int. J. Res. Mark.
Numerical data quality in IS research and the implications for replication
Decis. Support. Syst.
Learning in the visual association of novice and expert designers
Cogn. Syst. Res.
A short, reliable measure of subjective knowledge
J. Bus. Res.
The Business Value of Design
Design of emerging digital services: A taxonomy
Eur. J. Inf. Syst.
Service Design: 250 Essential Methods
The too-much-of-a-good-thing effect in management
J. Manage.
Investing in deliberation: A definition and classification of decision support interventions for people facing difficult health decisions
Med. Decis. Mak.
E-commerce product recommendation agents: use, characteristics, and impact
MIS Q.
Interactive decision aids for consumer decision making in E-commerce: The influence of perceived strategy restrictiveness
MIS Q.
Informed decision making
Med. Decis. Mak.
Patterns in information systems portfolio prioritization: Evidence from decision tree induction
MIS Q.
The impact of data quality tags on decision-making outcomes and process
J. Assoc. Inf. Syst.
User tags versus expert-assigned subject terms: A comparison of LibraryThing tags and Library of Congress subject headings
J. Inf. Sci.
Cognitive fit: An empirical study of information acquisition
Inf. Syst. Res.
The development and validation of the rational and intuitive decision styles scale
J. Pers. Assess.
Explaining choice quality with decision style, cognitive reflection and decision environment
J. Oper. Res. Soc.
Individual differences in the use of decision support aids
J. Account. Res.
A method for taxonomy development and its application in information systems
Eur. J. Inf. Syst.
The structure and form of folksonomy tags: The road to the public library catalog
Inf. Technol. Libr.
Usage patterns of collaborative tagging systems
J. Inf. Sci.
The folksonomy tag cloud: When is it useful?
J. Inf. Sci.
Effort and accuracy in choice
Manag. Sci.
Designing web sites: Opportunistic actions and cognitive effort of lay-designers
Cogn. Sci. Q.
Cognitive effort: A neuroeconomic approach
Cogn. Affect. Behav. Neurosci.
DSS effectiveness in marketing resource allocation decisions: reality vs. perception
Inf. Syst. Res.
Cognitive fit: A theory-based analysis of the graphs versus tables literature
Decis. Sci.
The role of cognitive fit in the relationship between software comprehension and modification
MIS Q.
Distributed cognition in software design: An experimental investigation of the role of design patterns and collaboration
MIS Q.
Using visual representations of data to enhance sensemaking in data exploration tasks
J. Assoc. Inf. Syst.
Cited by (7)
Content recommendation with two-level TransE predictors and interaction-aware embedding enhancement: An information seeking behavior perspective
2023, Information Processing and ManagementAdvancing our understanding and assessment of cognitive effort in the cognitive fit theory and data visualization context: Eye tracking-based approach
2022, Decision Support SystemsCitation Excerpt :Alternatively, a graphical representation of the problem was the most effective in supporting decision processes when solving spatial tasks. CFT, over time, gained popularity and was used across numerous domains, such as modeling [17], software tools [18], social network analysis [19], geographic representation [20], knowledge management systems [21], virtual reality [22], decision aids [23], to name a few. In the process, CFT moved beyond symbolic and spatial tasks and was deployed across varied and more complex task types, some of which had inconclusive [24,25] or contradicting results [26].
Digital service innovation (DSI): a multidisciplinary (re)view of its origins and progress using bibliometric and text mining methods
2024, Journal of Service ManagementAI-driven Digital Business Design Assistant: A Prototype Demo
2024, CEUR Workshop ProceedingsA Decision Support System for the Selection of Insulating Material in Energy Retrofit of Industrial Buildings: A New Robust Ordinal Regression Approach
2024, IEEE Transactions on Engineering ManagementContext in Design Science Research: Taxonomy and Framework
2022, Proceedings of the Annual Hawaii International Conference on System Sciences
Xuanhui Liu ([email protected]) is a post doc at the Zhejiang University. Xuanhui received her doctorate degree in information systems at the Karlsruhe Institute of Technology, Institute of Information Systems and Marketing. Her research interests include digital service design and used-centered systems. Her research has been published in International Conference on Information Systems, European Conference on Information Systems, etc.
Karl Werder ([email protected]) is a research fellow in the research group Information Systems and Systems Development at the University of Cologne. Prior, he worked as a senior researcher at paluno, the Ruhr Institute for Software Technology (University of Duisburg-Essen) and as a research assistant at the Institute for Enterprise Systems (University of Mannheim). Karl received his doctorate degree in Information Systems from the Karlsruhe Institute of Technology. His research interests include software development, systems design, game research, and data analytics. His work has been published in leading Journals (e.g., IEEE Transactions on Software Engineering, California Management Review, Information Technology & People, Information & Software Technology) and international conferences.
Alexander Maedche ([email protected]) is full professor of Information Systems at the Karlsruhe Institute of Technology. Previously he was full professor of Information Systems and managing director of the Institute of Enterprise Systems at the University of Mannheim. His research focuses on designing user-centered and intelligent digital service systems. He has published more than 100 papers in journals and conferences, such as the Decision Support Systems, Computers in Human Behavior, International Journal of Human-Computer Studies, Journal of the Association for Information Systems, and Information and Software Technology.