1 Introduction

The use of software applications in different contexts, whether for work or for entertainment purposes, has grown significantly in the last few years [13]. With this, it has also increased the number of systems that do not fully meet their users’ needs. Thus the need arises for allowing end users, with little or no knowledge in programming and software development, to create their own applications or customize existing ones, to support them in carrying out specific tasks [16].

This fact characterizes the End-User Development (EUD) approach, which enables end users to design or customize the user interface and functionality of software [7]. EUD also consists of a research area that has been under study for some time and covers a wide variety of domains and types of end users [16]. However, there is still a lack of studies that analyze how EUD research has been reflected in practice, that is, in the development of software applications based on this approach.

This paper contributes to this line of research to enable an understanding of the current scenario of EUD solutions, revealing trends that are emerging and gaps to be addressed. For this, a systematic literature review (SLR), following the methodology proposed by Kitchenham [14], was carried out, aiming to characterize the solutions that have been developed using the EUD approach. Such characterization considered particularly the context of use, platforms and interaction styles used, as well as characteristics of quality of use and limitations presented by the EUD tools.

Next, we briefly present the concept of EUD, which is the main theory on which the present work is grounded, followed by an overview of related works. Then, we present the protocol for the systematic literature review, followed by the results and discussion. We conclude by presenting our final remarks and outlining the next steps of our research.

2 End-User Development

End-user development (EUD) is a set of methods, techniques and tools that allow non-professional users to act as software developers, creating, modifying or extending software to achieve a goal [17]. EUD covers two similar concepts, end-user programming and end-user software engineering [7]. End-user programming (EUP) is the most mature subset of EUD from research and practice perspective [7] and it enables end users to create their own programs [16]. In this case, the developer himself is the user of the program, whereas in professional programming the goal is to create programs for others to use. Programs created by EUP can be extensions of existing applications, such as the creation of text manipulation macros in a text editor that are linked to keyboard shortcuts. These programs also can be new applications, as in the case of a spreadsheet that a teacher creates to manage the grades of his students. According to Burnett and Scaffidi [7], the difference between EUP and EUD is that EUD methods, techniques, and tools span the entire software development lifecycle, including modifying and extending software, not just the “create” phase.

The other concept related to EUD is end-user software engineering (EUSE). EUSE is a relatively new subset of EUD that emphasizes the quality of the software end users create, modify, or extend. Research in EUSE focuses on methods, techniques, and tools that promote the quality of such software [7]. The challenge of EUSE research is to find ways to incorporate software engineering activities into users’ workflows without requiring a radical change in the way they work [16]. An example would be to incorporate tools that simplify the identification of data entry failures as the user provides information in a spreadsheet, rather than waiting for the users themselves to enter a test phase during their programming efforts.

In this paper, we use the term EUD to refer to both EUP and EUSE. Thus, the EUD solutions that will be characterized may refer to both software artifacts created, modified or extended by end users and also methods, techniques, and tools that promote the quality of such software.

3 Related Work

Among the few studies that make a survey on End User Development (EUD) field, Ko e colleagues [16] conducted a study aiming to classify the research and define the area of End-User Software Engineering (EUSE). The authors discuss empirical research related to EUSE activities (requirements, design and specification, reuse and testing) and the technologies designed to support them. As results, they identify works that focus on these different activities and also for different platforms.

Tetteroo e Markopoulos [28] presented a larger study comprising a literature review of the EUD field, covering the period from 2004 to 2013. The authors discuss the methods, purposes, and impacts of research in EUD, as well as pointing out trends within the research community and the gaps that need to be addressed. The results point to an increase in interest in EUD researches, although such interest has declined within the HCI community.

Although the above studies present surveys on the state of the art in EUD field, they focus on specific aspects in this research area. Paternó [25], on the other hand, addresses more practice aspects, by discussing and classifying several EUD approaches according to their main characteristics, as well as the technologies and platforms for which they were developed. However, a broader study is lacking, aiming to characterize in a more detailed and systematic way the EUD solutions presented in literature.

Regarding specific EUD solutions, we have identified a number of them, with distinct purposes and for different contexts, as we will show in the Sect. 5, that presents the results of the study conducted in the present work.

4 Systematic Literature Review

Our study followed the methodology proposed by Kitchenham e Charles [14] to conduct the Systematic Literature Review (SLR) which allowed us to characterize the EUD solutions that have been developed in the last years. Such methodology has three main phases: (1) planning the review; (2) conducting the review; and (3) reporting the review. In the phase (1), the goal of the review is identified and a review protocol is developed. Then, in the phase (2), the literature review is conducted by selecting the primary studies, assessing their quality, collecting the data and synthesizing the results. Finally, in the phase (3), findings are reported.

In the following subsections we present the activities carried out in the phases (1) and (2) of the SLR we conducted, while the phase (3) will be described in Sect. 5.

4.1 Research Question

We used the PIO (Population, Intervention and Outcome) strategy to formulate the research question that will guide the study [14], as showed in Table 1.

Table 1. PIO criteria
Full size table

Thus, the following research question (RQ) was formulated for the present study: How are the end-user development solutions characterized? To better answer this research question, it was broken down into the following specific questions (SQ):

  • [SQ1] What contexts have EUD solutions been developed for?

  • [SQ2] What platforms have EUD solutions been developed for?

  • [SQ3] What interaction styles have been used by EUD solutions?

  • [SQ4] What quality-of-use features have been addressed by EUD solutions?

  • [SQ5] What are the limitations of EUD solutions?

Once the specific questions had been identified, the next step was to identify the relevant publications.

4.2 Research Process

Firstly, we elaborated the research string. We did it from the junction of the main terms that compose the research goal. We chose to use each term separately given that different authors use specific terms (e.g. EUP and EUSE) in the context of EUD.

In order to ensure that studies published in national and international conferences and journals were returned, we used keywords in both Portuguese and English language. Therefore, the string showed in Table 2 was defined for this research.

Table 2. Research string
Full size table

The research string was used in electronic search, in the following digital libraries: ACM Digital LibraryFootnote 1, IEEEFootnote 2, Science DirectFootnote 3, SpringerFootnote 4 e a HCIBIBFootnote 5. In addition, we also did manual search in proceedings of Brazilian Symposium on Human Factors in Computing Systems (IHC), for years that were not available at the ACM Digital Library.

The selection of repositories to search for publications was based in their importance for had published the greatest amount of study in the EUD field and that would allow us to undertake a study with high quality and relevance.

We search for studies published from 2007, when the first conference dedicated exclusively to the EUD took place, the International Symposium on End User Development, to 2017, the year when the present research was conducted.

According to [14], studies should be evaluated for their relevance. Therefore, inclusion and exclusion criteria were defined in order to reduce the number of studies that are not relevant to the research question. Tables 3 and 4 exhibit the inclusion (IC) and exclusion (EC) criteria for this research, respectively.

Table 3. Inclusion criteria
Full size table
Table 4. Exclusion criteria
Full size table

Once the inclusion and exclusion criteria have been defined, the search was executed, both manually and automatically, using the research string, in the selected repositories, according to the selection process shown in Fig. 1. The following steps were performed:

  1. 1.

    Reading the title to eliminate irrelevant documents;

  2. 2.

    Reading the abstract and keywords to eliminate studies included in step 1 that were not related to the research questions;

  3. 3.

    Reading the introduction, main topics and conclusion to eliminate studies included in step 2 that were not related to the research questions (skim reading);

  4. 4.

    Complete reading of the selected studies in step 3 and application of the quality criteria, in order to verify their relevance to collect relevant data to the research.

Fig. 1.
figure 1

Studies selection process

Full size image

The above steps were carried out by a researcher, with the assistance of another researcher, who reviewed the whole process. In cases where verification at each step was not sufficient to determine whether or not the study should be included in the next step, it remained on the list so that it could then be analyzed in the next step, until its relevance to the research was confirmed. In steps 1, 2, and 3, the inclusion and exclusion criteria established for the search were applied. The data set found in each of the steps was stored in Excel worksheets.

As we can see in Fig. 1, in the step 1, of 939 articles returned by automatic and manual searches, 187 (20%) were selected for the step 2, after removing duplicates and executing the title reading step. After abstract and keywords reading, 75 papers were selected for step 3, which represents a reduction of 60%. Finally, after skim reading, 55 papers were selected for step 4, when complete reading of them was carried out. Thus, 21 papers were selected for analysis, after the application of the quality criteria shown in Table 5.

Table 5. Quality criteria
Full size table

5 Results

Table 6 presents the 21 studies analyzed and their reference identifier assigned to be referenced in this paper. It also summarizes the data collected during the research. This information will be used in the analysis which is presented in the next section. The analyzed studies were classified according to five dimensions: context of use, platform, interaction style, quality-of-use features and limitations presented by EUD solutions. Context of use refers to the application domain of the EUD solution. The Platform refers to the environment for which the EUD solution was developed, which can be web, desktop or mobile [25]. Interaction style consists of the way in which end users interact with the solution of EUD [23]. Quality of use features is related to characteristics of the EUD solution that qualify the possible interaction through the interface according to certain aspects, such as usability, user experience, accessibility, among others [26]. Limitations consist of problems related to EUD solutions reported in the studies. Finally, Segment refers to where the EUD solution was created, i.e. academia or industry.

6 Analysis

Analyzing the resulting studies from the SLR, it is noticed that the largest number of papers addressing EUD solutions occurred more recently, between 2014 and 2017, corresponding to approximately 66% of the analyzed publications, which characterizes an incipient character of the research involving EUD tools. Regarding publication venue, 43% of papers were published in journals or periodicals, while 57% were published in conferences. Figure 2 shows the publications by year and by publication venue.

Fig. 2.
figure 2

Distribution of publications by year e publication venue.

Full size image

Considering the studies analyzed, 48% of them present EUD solutions for customizing existing applications, while 29% aim to creat new applications and 23% are intended for content creation.

Another aspect observed from the analysis is that most EUD solutions come from academia. Only 5 studies (24%) had collaboration from industry researchers.

Next, in order to answer the research question (RQ) of this study, we will answer the specific questions (SQ), based on the research conducted.

Table 6. References and data collection summary
Full size table

6.1 [SQ1] What Contexts Have EUD Solutions Been Developed For?

Regarding to the context, which refers to the domain to which the analyzed EUD solutions apply, it was possible to classify them as follows:

  • Domain independent: most of EUD solutions analyzed (67%) can be classified in this category, i.e, they allow end-users, even not being technology experts, to be able to create, modify and extend existing content and services in order to get an application that best suits to their needs, whatever the domain which it refers. This is the case, for example, of the tool presented in E20 [10], which can be used by end-users to develop different types of applications.

  • Domain Specific: 33% of the EUD solutions analyzed fit in this category, i.e., they are aimed at to a specific domain, as personal life) (E2 [31]), culture (E3 [3]), health (E7 [29]), smart homes (E10 [9]), businesses (E12 [27]) and sports (E14 [4] and E18 [24]).

6.2 [SQ2] What Platforms Have EUD Solutions Been Developed For?

Most of EUD solutions presented in analyzed studies were developed to web platform (about 65%), followed by desktop (22%) and mobile platform (13%), as shown in Fig. 3. One analyzed solution is multiplatform, contemplating both web and desktop platform.

Fig. 3.
figure 3

Platforms for which EUD solutions were developed

Full size image

6.3 [SQ3] What Interaction Styles Have Been Used by EUD Solutions?

EUD solutions were classfied according to the interaction styles, presented in literature [7], that they provides to end-users. Figure 4 shows the interaction styles provided by solutions analyzed. Visual Programming is the most used between solutions (64%) and only one solution combines more than one interaction style.

Visual programming is a techniques that has become relevant in the research community for reducing barriers for end-users to create content for specific domain and for demonstrating benefits in support to development [20, 24]. Thus, the analyzed EUD solutions use such technique in order to facilitate the interaction of end-users which have no development knowledge. This is the case, for example, of the solution shown by [31], which allows the end-user sets alarms and reminders for daily tasks simply manipulating visual components in its interface.

Fig. 4.
figure 4

Interactions styles in EUD solutions.

Full size image

6.4 [SQ4] What Quality-of-Use Features Have Been Addressed by EUD Solutions?

The majority of EUD solutions analyzed (15 out of 21) presents features associated to quality-of-use, as explained below:

  • Easy of use. This feature is the most present in analyzed solutions (33%) (E2 [31], E4 [11], E9 [2], E11 [21], E14 [4], E20 [10] e E21 [8]). This can be explained by the fact that easy of use i very important for a system can be effectively used by end-users, particularly those which have no extensive experience in technology. Thus, just the fact of not requiring development knowledge, which is an inherent feature of EUD solutions, contributes to ease-of-use. This is the case of SideTalk [8], for example, that has a simple interface and communicates well to users its design intentions.

  • User feedback, concerning the actions he performs in the system. This feature is present in some analyzed studies (19%) and it is important in order to keep the user informed of the results of his actions. Such feature was considered by four analyzed solutions (E1 [12], E3 [3], E10 [9], and E17 [1]). The solution presented in [9], for example, consists of a domestic system that allows users to have an overview of the current status of their smart home.

  • Concealing of non-important informations. Three solutions (E1 [12], E9 [2] e E13 [18]), i.e., 14% of analyzed studies, presented the feature of not show in the interface information that can cause confusion for the user, or even lead to system errors. The Sticklet, for example, shown in [12], hides some JavaScript functionality from users since it can cause failures on system.

  • Adaptation to users needs, in different contexts of use. This feature refers to the flexibility offered by the system, in order to can be tailored to satisfy users needs. Such features present in three of analyzed solutions (E5 [6], E7 [29] and E8 [32]), i.e., 14% of analyzed studies. The EUD solution present in [6], for example, allows users to adopt certain features of the work environment, depending on their experience.

6.5 [SQ5] What Are the Limitations of EUD Solutions?

The absolute majority of analyzed EUD solutions (20 of a total of 21) present some difficulty or limitation in their use. The most of the detected limitations in analyzed solutions consists of aspects which can negatively impact users interaction with the system, as follows:

  • Incompatibility between the language used by the system and the user’s language. This limitation, which is present in 33% of analyzed EUD solutions (seven - E2 [31], E5 [6], E7 [29], E8 [32], E14 [4], E20 [10], and E21 [8]), makes more difficult for user to understand how to interact with the system. Interaction with these solutions requires greater effort on the part of users, as is the case of TagTrainer, presented by [29], where the lack of programming skills and interaction design experience of some therapists inhibit them to create proper exercises for their patients.

  • Lack of control by users. This limitation was reported for 20% of analyzed EUD solutions (six - E3 [3], E4 [11], E6 [5], E9 [2], E10 [9], and E18 [24]). It restricts user interaction with a system in actions that are pre-defined by designer. An example is open.DASH application, presented in [9], which made impossible for users to interact with desirable system features, due to a limited amount of sensors.

  • Lack of help to users. The lack of a help mechanism, which guides the user through his interaction with the system, explaining details on how to use it, was pointed out as limitation in 19% of the studies (four - E1 [12], E9 [2], E20 [10] and E21 [8]). In Stikclet [12], for example, it is not offered to users any assistance to guide them while their interaction.

  • Limitations to specific platforms or formats. 14% of analyzed studies (three - E1 [12], E13 [18], and E16 [19]) address this limitation in EUD solution. This is the case of tool presented in [19], in which it is only applicable information in XML format, but not in other, as JSON or simple text file.

  • Error propensity. In solutions with having such limitation, there is the possibility that user-system interaction is discontinued as a result of failure to perform one or more functionality. 10% of analyzed EUD solutions (two studies - E16 [19] and E20 [10]) indicate the error propensity as a limitation, as is the case of runtime errors shown by EUD solution presented in [10].

  • Other limitations on interaction. In 14% of studies (three - E15 [13], E17 [1], and E19 [30]), it was found the following limitations: not supporting collaboration, lack of feedback to users about the results of their actions e incompatibility between interaction styles used by solution.

In addition to these limitations, one study (E12 [27]) pointed as EUD solution limitation a problem related to integration and scalability. Such issue, different from those presented above, is intrinsic to the system, and not relative to user interaction.

7 Discussion

From the obtained results, it is possible to answer the research question investigated in this study: [RQ]: How are the end-user development solutions characterized?

Initially, we identified that research involving EUD solutions is recent and has intensified from 2014, indicating an increase in the use of the EUD approach in practice as well as in the interest in such research field. In addition, most of the solutions came from the academy, indicating that industry still does not effectively apply the potential of the EUD approach to market opportunities for products and services. This fact confirms, in the present day, what Klann et al. stated in 2006 [15].

The analyzed solutions focus mainly on the tailoring of existing applications, i.e., modifying the appearance or set of attributes of a system by an end user [22]. Such EUD solutions are also predominantly independent of domain, indicating a more general character of the EUD approach. This suggests a tendency to have solutions aimed at meeting the needs of end users in any domain or area of knowledge, instead of being targeted to specialists of specific domains.

The fact of most EUD solutions have been developed for web platform probably occurs because it is currently the most common user interface and can be accessed from any device. This fact may encourage the development of EUD solutions for such platform [25].

Visual programming is the dominant interaction style in EUD solutions. Thus, given the purpose of such technique of making easier the user interaction [7]. This fact indicates that EUD research converges to popularizing the use of its tools, by using the visual programming as a strategy to attract end-users to create their own applications.

The concern with quality of use was present in most of the analyzed EUD solutions, with “ease of use” being the most present feature among them. This fact shows the concern about the use of these solutions by end-users, given that one of the requirements for systems to be widely accepted by their users is that they present properties that bring quality to the possible interactions through their interface [26]. On the other hand, the number of limitations regarding issues that negatively impact the user interaction, present in EUD solutions, exceeds the number of quality-of-use features. This fact indicates an inconsistency and deserves to be further investigated.

This fact suggests a gap in the research on the use of IHC models, methods and techniques in projects and evaluation of EUD tools. Such a situation can be confirmed by the fact that, of the 21 studies analyzed in this study, only one EUD solution was developed and evaluated using HCI models and methods [8].

8 Conclusion and Future Works

This study described how the end-user development approach has been used in the development of end-user support tools. For this, a systematic literature review was conducted in order to identify how these solutions are characterized.

The motivation to conduct this study was initially due to the lack of studies that analyze how research in EUD is reflected in practice.

The results indicated that most of the existing EUD solutions in literature focus on application customization, by using visual programming techniques and thus indicating an initiative to attract users to use them. However, quality-of-use features and limitations identified in EUD solutions point to inconsistencies that indicate the need for research involving HCI models, methods and techniques in the design and evaluation of these tools, in order to achieve a higher quality of use.

In terms of contribution, our results are a basis to research approaching HCI models, methods and techniques in design and evaluation with EUD tools.

To this end, research can be conducted in order to investigate how HCI theories, models, methods and techniques have been used in design and evaluation of EUD solutions and what the impact of this in their quality of use.