I Don’t Know, Is AI Also Used in Airbags?

2.1 Public Acceptance and Awareness of AI

If we take a look at several large-scale reports on public acceptance and trust regarding AI-enabled technologies, we will recognize a trend of increasing positive reactions towards AI. To begin with, in a survey with over 2000 respondents conducted in 2015 by the British Science Association, one-third of the respondents considered the rise of AI to be a danger for mankind [35]. In 2017, although some researchers indicated that users still felt uncomfortable with AI making decisions on their behalf, the majority of the 27,901 interviewed EU citizens had a generally positive view of robots and AI [22], [41], [48]. In more recent surveys, we will see even greater support for future AI developments among users [8].

In addition to the terms trust and acceptance, AI awareness has also been discussed in the context of perception of AI. A study from HubSpot of over 1,400 consumers worldwide revealed that 63 % claimed that they did not use AI tools while actually using them without being aware of it [3]. Pegasystems confirmed these findings in a study of 5000 consumers, of whom only 34 % agreed that they had previously used AI technology but of whom 84 % actually had [54]. Furthermore, researchers at the Center for the Governance of AI at the University of Oxford found that the majority of users believed that virtual assistants, smart speakers, driverless cars, social robots, and autonomous drones use AI but that Facebook photo tagging, Google Search, Netflix or Amazon recommendations, or Google Translate do not use AI [52]. The results from a Northstar survey of 3804 consumers in 2020 also indicate a significant difference in recognition between “invisible AI” (i. e., AI algorithms behind the scenes) and “visible AI” (i. e., tangible AI devices) [8]. Here, 90 % of the respondents knew that voice assistants are AI enabled, but almost one-third did not consider social media to be an AI-based technology. Davies argued for the dependency of public awareness of AI on the visibility of its application [8]. Nonetheless, their study shows that users do not define AI as humanoid and know the difference between sci-fi and reality [8].

By distinguishing between public perception of AI and public acceptance, trust and awareness, we aim to contribute to closing the gap in the literature and to study users’ definitions of AI and informal theories on how AI-enabled technologies work.

2.2 Designing Explainable Technology

To move towards a more transparent design of technology and address trust concerns [2], “explainable AI” was introduced as AI systems whose activities can be effortlessly comprehended and examined by humans [21]. Van Lent et al. [49] first used the term in 2004 to explain the behavior of the AI-enabled elements in the simulation applications. Over time, the scholarly community and professionals have paid renewed attention to explainable AI [33].

By refining users’ mental models of AI-enabled systems and resolving their misconceptions, XAI promises more effective performance of the users [41]. However, previous researchers have criticized XAI for resulting in less efficient systems and less flexible and capable output [33], [41]. To address this issue, XAI researchers have argued that explanations are not always necessary and instead introduced specific application domains in which they can bring significant benefits (e. g., healthcare and military) [12], [33], [41]. In this regard, Molnar [41] introduces human-friendly explanations as selected and focused on the abnormal. In other words, he suggests that people do not expect complete explanations that cover a full list of causes, but rather expect them to explain why a system behaved in a way they did not expect [41].

We must bear in mind that users were already interacting with AI-enabled systems long before the relatively new trend of XAI was applied to design [33] and that as the operation of these systems was opaque, they have often developed theories of how they work (folk theories) to plan their interaction with them [10], [13], [14], [31]. Folk theories function as a frame in which user expectations are formed [31]. Therefore, understanding them is not only helpful in designing effective explanations but is also necessary in order to understand users’ assumptions and expectations of the systems and discover more valid explanation use cases.

2.3 Making Sense of Technology

To make sense of technology, users often generate folk theories based on direct experiences and social interactions [14], [31]. These informal theories explain how systems operate and support users in responding to them [31]. Previous studies have thus argued that folk theories act as a window to capture users’ perceptions and assumptions [31]. However, the existing studies on mental models and folk theories in the field of AI are highly distributed [13]. Hence, by dividing earlier work into different AI domains, we provide an overview of prior research on folk theories and their role in understanding users’ perceptions, misconceptions, and expectations.

4.1 What Is AI: A Thing or Characteristic or Set of Instructions?

Conversational openings are of special interest, as they provide the context framing for what follows [16], [45]. Metaphorically speaking, the opening prepares the stage on or the guard rails within which the narrative can unfold. Therefore, when asking for respondents’ definitions of AI, we particularly noted the initial sequence. Most of the participants chose a relatively broad and vague framing (such as referring to AI as something, a thing, a system, or an entity), while some were more focused, describing AI as a device, a brain, or a computer. These ambiguities, vagueness, and difficulties in the definition of the term do not indicate a restricted or defective understanding on the part of the respondents, but are, in our view, well founded. Artificial intelligence is something that only presents itself to the user through its embodiment and its performance; it exists not as its behavior itself, but rather in what that has produced the behavior. Normally, this was described as a property or competency of the entity. However, some mentioned program code, detaching this property from the concrete thing, meaning that the computer itself does not have AI, but that AI manifests only with the correct program, which itself is not AI either. The difficulties people faced locating AI were reflected in their framing of what AI was.

Our brief analysis shows that providing a clear, uniform answer to the question of “What is AI” was tricky for the respondents. Hence, we also asked them to define AI extensionally by giving three examples of it. As expected, participants mainly focused on the thing-ness of AI, naming tangible devices, while the more abstract concept of AI as a specific characteristic or specific instruction were less prominent. For instance, machine learning was rarely listed as an example of AI, whereas Amazon’s Alexa, robots, and self-driving cars, followed by the visible but not tangible examples such as Twitter-bots and recommendations in Amazon or Google, were often mentioned.

4.2 On the Competencies of AI: From Automation to Simulation and Superpower

Despite the vagueness of these openings, they provided the respondents with certain frames to which certain competencies could be assigned. Based on these, we categorized the understanding of AI’s capabilities into six groups, shown in Table 1.

AI is automated: Like P21 (see Table 1), many participants described AI as a technology that operates autonomously with the purpose of fulfilling the interests and ideas of its developers. This view underlines the superiority of humans by emphasizing the never-ending need for human support for AI’s existence and functioning. The view also emphasizes the artificial nature of AI, placing AI in the long history of technical development and understanding it as being similar to the other everyday devices with which we interact. Artificial intelligence does not have its own agency and cannot work independently. “AI is dependent on the implementation of the programs” (P25).

AI has agency: The quote from P47 (see Table 1) indicates the understanding of AI as autonomous in a peculiar way. This becomes evident when we split the phrase into two parts. In the first part, P47 describes AI as a system that makes decisions for humans. We observed several variants of this phrase, such as that AI supports humans in everyday life, AI provides services for humans, etc. In the second part, however, P47 stresses that AI does not need humans and is independent of them, which points out the complex relationship between AI and humans. Artificial intelligence is not just independent, but independent of humans. Various participants also mentioned that independency is not a future stage of AI, but its current status.

AI is just like humans: Participants often described AI with reference to humans. To understand the folk concepts about AI, an understanding of the folk concepts of human beings is thus a prerequisite. This is notable as, for instance, common-sense descriptions of natural things (such as trees, mountains, planets, etc.) as well as technical things (such as houses, cars, toasters, etc.) typically do not involve such a reference. Here, human beings are referred to as a model for AI, which is an image of human beings and in order to conceive of which one needs knowledge of what human beings are. From this standpoint, a technology can be seen as AI if it can pass the Turing Test: “AI has the same behavior as humans and acts and reacts at the same level” (P46). Some participants characterize AI with regard to specific capabilities or properties of humans: “AI means systems that act like humans regarding learning and understanding others” (P44).

AI learns by itself: Half of the participants addressed the competency of AI to learn and improve by itself. It was remarkable that, although we did not ask interviewees about how AI learns by itself directly, they explored this topic themselves. A closer look allows us to analyze this folk concept along three interconnected issues: How learning works, what the sources for learning are, and what the results of learning are:

Learning source and strategy: There are thousands of ways to learn; for example, you can read a book, think about a problem, or become wise through experience. As in the phrase of P27 (see Table 1), most of the interview partners had a computational understanding, in which AI learns from data. Furthermore, some participants, such as P27, pointed out that AI can process large amounts of data, while some did not mention data itself, but patterns as sources of learning: “Computers analyze the patterns of human faces in such great detail and in such large quantities” (P31). In addition to technical vocabulary, many descriptions contained implicit and explicit analogies to human learning, for example, when terms are used, such as learning from experience, learning from mistakes, or learning from the analysis of facts. Such kind of anthropo- or zoo-amorphization can be found in phrases such as “AI uses previous experiences as a basis for shaping connections, learns through experiences and makes connections (example: needle à pricking à pain)” (P7). In this phrase, AI is thought of as a small child or a rat in a Skinner experimental setup that learns by operant conditioning. In a similar vein, P10, P15, and P17 referred to the common strategy of learning from mistakes: “The program itself learns from errors and improves itself” (P10). Sometimes the interviews simply referred to the ability of AI to learn without explaining how this learning takes place: “AI is the ability to learn, conclude” (P43).

Learning outcomes: “They can recognize faces relatively well and reproduce them digitally” (P31). Unlike learning strategies and sources, learning outcomes were only mentioned vaguely and could not be easily categorized into groups. We can approximately divide descriptions to those in which the learning goal and scope are quite precisely defined, those in which they are quite broadly described, and those in which they are undefined. The first category includes description in which specific goals such as speech recognition and image recognition are used as paradigmatic examples to explain AI learning. The second group includes descriptions in which AI learning is only described in very general terms, such as “AI learns through experience and makes connections” (P7) or “AI is a group of intensities that can learn from data and produce the appropriate output from input and continue to improve.” (P1). In such a folk concept, it seems that specific goals or purposes play a small role whether a machine can be considered self-learning. Instead, learning seems to present a general competency of AI that is in principle not limited to a specific area.

AI simulates human intelligence: As the Turing test also suggests, the intent and ultimate goal of AI is to reach a human-level in its performances. “AI attempts to transfer human intelligence to machines to promote their learning ability” (P23). Here, AI is perceived as an infant in its early development, moving towards reaching the human level someday: “AI is a digital brain in a computer that learns behavior like a toddler” (P18). Therefore, unlike the responders with the “just like humans” point of view, P18 does not believe that AI can act and react like humans or pass the Turing test, but rather is approaching achievement of these goals in the future.

AI is a superpower: Various participants, such as P37 (see Table 1), assumed that AI has a super-intelligence that surpasses that of humans. This superintelligence not only knows the answer of any possible inquiry but also can predict the future: “AI draws conclusions from past events about what will happen in the future and predicts beyond our knowledge” (P20). One reason that AI is understood as a superpower is its unlimited data processing and storage capacity, seen, for example, in the phrase of P4: “Artificial intelligence is when algorithms have unlimited knowledge” (P4). Superintelligence is also grounded in the super-learning competency of AI, which was mentioned by the respondents who perceived AI not as a predefined code, but as a system that can generate unexpected results: “AI means self-learning systems that generate outputs through the corresponding inputs, which are not expected” (P1). The idea of a superpower places high demands on AI, so that one can ask whether today’s systems meet these requirements. P46 posed this question and answered that these capabilities do not exist in the today’s technologies: “AI? It does not exist yet. We call them smart but they are not really smart” (P46).

4.3 On the Expectations of People: When Things Become AI

The question about what happens when things become AI helps us shed more light into the current beliefs of people and uncover how people apply their often-implicit ideas of AI and what their latent expectations are. We grouped the answers into four categories (see Table 2). The analysis reveals that the folk expectations of AI are shaped by the folk concepts about what AI is (see Table 1). Yet, how the concepts are enacted and interpreted depends on the concrete scenario and application area.

Automation: Half of the respondents mentioned that they would expect a smart door to automatically open and close (one representative is P4; see Table 2), and two believed that this feature alone is an indicator of AI: “I expect that the door opens and closes automatically” (P50). Six others added some sensors to their automatic door and introduced the intelligent door in this way: “It should recognize the movement and closes and opens based on its recognition” (P20). Therefore, we can say that some users do not see any differences between intelligent and automatic devices. P23 solved this problem by introducing different levels of intelligence: “The most primitive is the motion sensor, which detects when you approach the door. One step further would be image/face recognition. On the next level, the door could be connected to a smart home system, so the system could detect an intruder in the house” (P23). Interestingly, in the example of the bank account, only three respondents mentioned the automatic payment of the bills: “A bank account, that can buy things independently or transfer money” (P27). This might be due to the higher risk that our participants associate with the monetary services: “A bank that helps me save and transfers my monthly repeated bills automatically” (P36).

Agency: Regarding intelligent doors, only two respondents (P47 and P11; see Table 2) expect more comprehensive agency beyond a smart automatic door opening. P47 described how “The door should perceive its environment and react accordingly; for example, if needed, it should make less noise while opening and closing” (P47). Although agency did not play an important role in users’ common expectations from a door, it was the most common expectation they held of an intelligent bank account. It is worth mentioning that some users struggled with whether or not the embedded AI in their bank account should be trusted to have its own agency. Therefore, they had different expectations of the levels of control that the bank account would possess.

Improved comfort: AI was seen as smart enough to eliminate the need for manual authentication: “No bank card is required. For example, with a chip in your hand you can access your account” (P37).

Replace service worker for simple tasks: The participants said that AI could start to compete with bank employees and take over simple tasks: “The possibility to open new accounts without having to communicate with the bank employee” (P43).

Replace service worker for complex tasks: The same was true for complex tasks. Those most commonly mentioned were the bank account serving as personal investor and financial advisor. Here, the smart bank account is able to consult and suggest a financial approach to the user but is not allowed to make decisions on its own: “The smart bank account should evaluate my expenditure, suggest a consultancy and support me with my account management” (P9).

Making decisions automatically for the client: The fourth level covers the expectations in which the bank account makes decisions on its own. For instance, it prevents users from spending money irrationally by not permitting such transactions (P29; see Table 2). On this level, the AI is transparent with the users and lets them know why the decision was made. This decision might not be aligned with the users’ present interests but would fulfill users’ long-term goals.

Making decisions automatically even against the client: Users placed AI in a position in which it has full control, making decisions even against their interest. For instance, P13 envisions a scenario in which AI has full agency for better or worse: “For example if the account often has a negative balance, this could affect the creditworthiness of the users and they cannot receive loans” (P13).

Personalized services: In the case of the bank account, about one-third (16 of 50) expected personized financial services. For instance, the bank account was expected to play the role of a financial coach by analyzing transactions and payment patterns and providing users with plans for saving or investment: “A bank account can over time make recommendations for potential savings or investment opportunities” (P17). Others also mentioned improved feedback about spending patterns: “It analyses transactions on a monthly/yearly basis and evaluates how much money was spent on what and whether it differs at certain intervals” (P23). In the case of the intelligent door, this rate was much lower. Only four participants mentioned to expect personalized services from their intelligent door. P1 and P7 explained these personalized services as recognition of the behavioral pattern/schedules and customization of the door accordingly: “The door learns habits and can recognize through behavior patterns whether to lock or unlock” (P1). In addition, P9 also expected an intelligent door to begin communicating with people in a manner similar to that of a friendly porter: “I would expect a personal greeting” (P9).

Smart Security: Security was among the most common expectations for our participants. Although it made a greater contribution to the example of the smart bank account, four of the respondents also expected it from the intelligent door: “Security is the top priority here, the door should not only be protected against robbery. It should also be able to detect if an attempt is made to hack it, because anything that has to do with technology can be hacked and is very vulnerable” (P33). P15 (see Table 2) did not address the danger clearly, but in general. Here again, the expectations from a bank account were higher. Two participants even expressed their reluctance to answer the question because of distrust in AI: “If AI is embedded in a bank account, then I will have problem with trusting it” (P16) and “In a bank account, I want AI to be embedded as low as possible” (P25). P34 (see Table 2) addressed this problem by setting a condition for a smart bank account. He believed that before anything else, a smart bank account should establish trust. Some others focused on the new opportunities that a smart bank account can provide to detect phishing attacks and protect against them: “A smart bank account can keep my bank details safe and can detect whether a website is fake or not when I want to make a transaction” (P46).

Intelligent user recognition: The importance of security also covers access control, authentication, and authorization. In the example of the door, more than half of the users mentioned face or finger recognition for access as the expectation. This group includes P21, whose idea can be seen on Table 2. For a smart bank account, however, only five users mentioned expecting this capability. This might be because of two reasons. For a door, unlike a bank account, face/voice recognition is not a common feature, as many bank applications already provide this feature. Therefore, one can conclude that users expect a smart device to be capable of more uncommon actions. However, as security played the most important role in users’ concepts of a smart bank account, one can think of another possible reason, namely, that the users are still skeptical about this feature and would regard a smart bank account as too intelligent to risk the security of its users in exchange for offering more usability to them.

As we saw from the results, a higher priority was placed on some attributes such as automation and face/voice recognition for the smart door, while others such as agency and security were of a greater importance for the smart bank account. One reason might be the previous experiences of the particular participant with the object and the possible difficulties encountered. Some of the participants, such as P13, even referred to previous experiences with the object: “When I think about a door and artificial intelligence, a revolving door immediately comes into my head, as is so often the case in large shopping malls that peoples’ speed automatically slows down and vice versa” (P13). In general, the expectations from a smart bank account were higher than those of a smart door, which in most of the cases would only automatically open and close. This might be due to the abstract nature of a bank account, to which users can ascribe a human role of coach or adviser more easily than to a physical object such as a door, which is obviously not a human.

4.4 On the Impact of AI: If AI Disappears

When asked what would happen if AI were to vanish tomorrow, the respondents had to think of the technologies and services in which AI was embedded meaningfully. Our observations showed that it was a complicated matter for the respondents, some of whom, such as P30, even asked the interviewer whether they were right about AI use cases: “AI makes our everyday lives easier, for example, for driving, we have driver assistants, I don’t know, is AI also used in airbags?” (P30). Our analysis revealed that this complexity pushed respondents towards envisioning the impact of the loss of AI mainly based on the inputs they had from personal conversations, as well as from mass and social media. We categorized the mentioned impacts of AI into three following groups:

Social impact: “Disappearance of AI will cause a chaos in the world. There is definitely more AI in everything than we think” (P39). Several participants thought that AI has spread into all areas of life, so that the loss would lead to severe dislocation. P44, who took a seminar about AI at university and usually talked about the accuracy and simplicity of the roles in which AI has taken over, stated, “AI is boundless. I believe a life without AI is without development, it may not be the primitive life, but of course, there will be negative effects on all areas of life like engineering, medicine, education, agriculture, etc.” (P44).

Economic impact: “If AI disappears then the economy will get paralyzed and we cannot have a normal life anymore” (P18). Some participants, such as P18, stress that the disappearance of AI would have serious economic consequences. P1, for instance, stressed that AI is important for the digital revolution in Germany with regard to the economic progress: “Economic harm because companies like Amazon cannot use their recommender system anymore” (P1).

Personal impact: “I cannot imagine a life without AI. I ask myself, how we could drive a long distance without navigation 20 years ago” (P42). For participants like P42, who cannot even imagine the disappearance of AI, the spectrum of the personal impacts ranges from extreme cuts of personal living to small or even no personal changes: “There won’t be great personal consequences, expect for Siri and Alexa” (P2). P5 detached his personal impact from the impacts on others, saying, “There will be no consequences for average people, only those who have smart homes” (P5).

Do not have AI for better or worse: The respondents typically also expressed a position about the role of AI in today’s world and chose whether to perceive it as a positive or a negative one. In this regard, we identified four positions:

1. Deplore life without AI: “If AI vanishes that would mean that our work and our life will be more difficult” (P50). About 60 % would deplore a life without AI. A representative of this view is P50, who, believed that AI makes life easier: “If we want to make things easier, we should find more and more applications for AI in them” (P50). The extent of the regret also depends on how AI is understood: “So, if I lose my smart phone today and get back to the old phones without internet connection, then I will lose 20 % of my own identity and 80 % of my social relations” (P46).

2. AI is Janus-faced: “The fate of humans in the midst of this great upturn in science is terrifying” (P43). Less than 10 % mentioned both negative and positive sides. A representative of this position is P43, who noted, “There are positive and negative sides. Natural disasters happen all over the world, AI saves humanity by improving the methods and strengthening the means to cope with them. But if AI disappears, then the unemployment rate will decrease” (P43). This quote highlights the complex relationship between humans and AI: humans as the creators of AI, a superpower that begins to compete with them. This view was shared by others, who considered AI to be a sign of a brighter future that comes with a risk. There were also other double evaluations due to the consideration of AI in different areas. For instance, P6 differentiated between the social and the personal areas: “Without AI, the world would fall several years into the past. But for me personally, I will have more personal interactions and the social relationships won’t be digital like now” (P6).

3. Welcoming life without AI: More than 10 % of the participants were skeptical about AI and had some serious concerns and thus welcomed life without AI. One example is P26, who had several discussions with her husband about AI and the fear of being manipulated by it: “If AI vanishes, I will feel relived, although many things would be done manually again” (P26). Similarly, P22 had a conversation with her daughter about the latter’s homework on the ethical issues of AI in the area of self-driving cars: “If there is no AI tomorrow then the ethical issues, we have about the self-driving cars will be solved” (P22). The extreme case was presented by P47, who referred to AI in absolutely negative terms: “If AI vanished, the most important consequence would be more jobs for everyone” (P47).

4. Being incurious about AI: There was also a group (about 20 %) of the participants who were incurious, as they found AI to be not essential. They either believed that the disappearance of AI would bring about no specific changes or that the changes would be exceedingly small. A representative of this group was P23, who mentioned that many find AI not to be a relevant subject and avoid discuss it: “It would be a loss if we don’t have this technology anymore, but it’s not essential. Because until now, there aren’t any huge dependencies to AI. The economy can grow further with or without AI” (P23).

5.1 Meaning Ascribed to AI: The Interconnected Folk Concepts

Regarding our first research question, examining what meaning people ascribe to AI, our study updates the 30-year-old study of Schank [44]. Most of the definitions of AI noted by Schank were also among the folk concepts that we identified. For instance, users’ definition of AI as magic bullets is closely connected to the idea of AI being a superpower. In addition, the view of AI in terms of learning machines was highly prominent in our study. Complementing the work of Schank, our research shows that people have different views regarding the sources, strategies, and outcomes of the self-learning AI systems. This ranges from a highly technical understanding of machine learning to metaphorical notions and analogies with human learning. In contrast to Schank’s findings, the understanding of AI as an inference engine to turn experts’ knowledge into rules played only a subordinate role in our findings. This might reflect the prevailing concerns of the time when Schank wrote versus today, for while expert systems were an important topic 30 years ago, they are rarely present in today’s AI discourse.

When asked for three examples of AI, our respondents referred to the visible and tangible artefacts (such as Alexa, robots, and self-driving cars), which confirms the results from the research done by Northstar [8] and shows that people recognize visible artefacts as AI more easily. However, this result should be handled carefully, because here, the thing-ness of the conceptions of AI seems to be shaped by the nature of the question. In other words, when the question was formulated as “how do you explain AI?” the participants did not provide an extensional definition, but instead more abstract and conceptional answers (such as AI having agency, simulating human competencies, or being a set of instructions).

Despite having a relatively homogenous sample group (on average, representatives of a young generation of educated German users), our respondents had different understandings of AI and its capabilities. In this regard, our study confirms the previous work done on specific AI systems (such as content curation algorithms, recommender systems, and conversational agents) and the variety of folk theories and misconceptions reported by them [12], [13], [26], [29], [50], [51]. Our study indicates that mental models about specific AI systems are not isolated but shaped by common folk concepts and the grand narratives of AI that machines become superhuman. Moreover, we argue, that folk concepts are interconnected and build upon each other. To elaborate further, people’s perception of AI and AI-enabled systems influences their understanding of its meaningful and essential use cases that helps them evaluate AI’s current and future impacts in their lives.

An example of this interconnectivity can be seen in characterization of the folk concept by the complicated relationship between man and AI. Almost all explanations defined AI with a reference to humans (such as like humans, simulates humans, is independent of humans, is made by humans, etc.). Metaphorically, AI was frequently spoken of in terms of adolescent development, strongly influenced by parents and oriented towards them, while also turning away from them and claiming independence. This complicated relationship leads to a complex cooperation between the two, in which the role each plays is not clear for the users. This complex relationship also pervades the evaluation of AI as a technology that is part of our everyday lives.

5.2 Expectation Ascribed to AI: Design for the Unrealizable?

Design for user expectations presents an important guiding principle in usability engineering (ISO 9421-110) [23]. However, our study shows that people have quite diverse and high expectations of the AI-enabled artefacts such as “AI has the same behavior as humans and acts and reacts at the same level” and ambitious expectations such as “a door that recognizes me and communicates with me based on its knowledge about me” that surpass existing technological capabilities. This phenomenon suggests that people do not have appropriate knowledge when it comes to implementation, which can lead to unrealizable expectations of technology and disappointment when these expectations are not met. This may explain the phenomenon described in the literature of why users are dissatisfied with voice assistance systems such as Alexa, Google Home, and Apple Home pod [5], [32], [38], [46]: although these systems are useful for simple tasks such as turning on the light and playing a music track and work reliably, they cannot meet the high expectations of users.

Therefore, in our next study, we aim to identify this gap between expectations and possibility by using more profound research techniques to grasp folk concepts, theories and mental models of AI. Moreover, based on our study, we recommend that explainable AI should be transparent not only in how the system works, but also in what a user can expect from it. We should also take seriously the insight from research that user experience is shaped not only by the actual interaction, but also occurs before, during, and after use [28]. From this perspective, realistic expectation must be made a part of the advertising of AI-enabled products and services, as well as of the AI-related narratives in science fiction literature and movies.

One other ambiguity for the respondents regarding the current state of AI concerns its users. Is AI only used by big high-tech companies and infrastructure? By tech savvies who live in smart homes and possess self-driving cars? Is it used by the average consumer? Or can anyone be considered an AI user? To address their lack of knowledge in this area, the respondents referred to whatever input they had gathered from others. This might also be the reason for the widely diverging evaluations of AI’s impact and current meaningful use cases among people, which ranged from wider areas such as the economy, medicine, politics, transportation, and so on to narrower use cases, such as in voice assistants or gaming.

5.3 Limitations and Future Work

Our work has certain limitations that should be considered and that can motivate future work. One major limitation regards the representativeness of our sample group and the known problems of self-selection bias. Hence, our findings especially express the folk concepts of a young, well-educated, urban generation of digital natives in western society, in Germany. Less can be said about the folk concepts of elderly people, as well as folk concepts in other societies. This can be improved by using a larger sample with more diversity in terms of technical background and age group, as well as by conducting a similar study in other countries. Deeper and more detailed direct questions could also be used to identify the barrier between automation and simulation or self-learning sources and strategies as well as the extent to which users’ engagement in social conversations impacts their perception and evaluations of the AI’s meaningful use cases.

Therefore, in future, we aim to investigate these folk concepts and theories along with their interconnections and impact, as well as other influencing factors (e. g., sci-fi and social media trends) in greater depth. Our study also encourages the implementation of a more user-friendly XAI by focusing on clear communication of what users should expect from AI-enabled systems. Such transparent AI systems can help us analyze how users’ expectations can be influenced and modified for more satisfying interactions.