Keywords

1 Introduction

Information technology (IT) is a powerful tool for accelerating economic development and websites have evolved to become the core component of any organization, within both the public and private sectors. The use of the Web as a channel for interaction between citizens and government institutions has created new opportunities along with technological development, innovations and new trends.

According to Sandoval-Almazan and Gil-Garcia [1], “The use of information and communication technologies (ICTs) in government has significantly increased in the past decades. Countries around the world are now adopting strategies for better use of these technologies with very different objectives: greater efficiency, deeper transparency, higher service quality, and more engaged citizen participation.” (p. S72). In contrast to this, we also find studies that have negative findings. According to Szkuta et al. [2]: “In the last 15 years the e-government became an important component of the public sector modernisation agenda. Yet, e-government has not lived up to expectations and did not transform the public sector. While significant progress has been made over the last years in terms of e-government service provision, the take-up of those services is still disappointing. As late as 2012, only 43 % of European citizens made use of e-government services, a percentage that drops off to 21 % when one looks at transactional features, i.e. submitting filled forms.” (p. 558).

We find various definitions of eGovernment within the research literature. Most of the definitions concern the use of information and communications technologies, in creation of an efficient dialogue with the citizens. According to Verdegem and Verleye [3]: “One of the objectives of E-government is to make the government and its policies more efficient: providing citizens with quicker and better access to public information and the ability to use services in a more personal and cost-effective manner.” (p. 488). An organization’s website performs a primary function in the provision of services and online information, and acts as the fundamental channel and contributor in communication with the users. Therefore, many organizations invest lot of resources into website development and quality improvements.

Despite this considerable investment, we speculate that most organizations lack knowledge about the measurement of benefits driven by the actual website. Although there are numerous suggestions on methods of assessing the benefits of IT, this particular feature has been an area of concern for both managers and researchers [4]. We also find that this topic is widely discussed among practitioners, and there is debate about the benefits of the websites. Benefits within a public sector setting are not always that tangible and measurable, e.g. in regards to economic performance and converting rate, and therefore we need to identify measures that are appropriate and suitable for this context.

The central government, in many countries, has placed much pressure on public sector organizations with regard to provision of online information and services. In order to focus on the importance and willingness to spend time and money on website quality improvement, various initiatives have been launched during the last 15 years. One example is how Norway and Denmark, from 2000 until recently, have focused on those quality aspects that are of particular importance. Hundreds of public website are evaluated yearly and ranked by quality, the results being made publicly available and the winners highlighted as examples of best practice. The quality of websites has, consequently, received much attention during the last decade. In contrast to this, fewer measures and initiatives have been taken about the assessment of benefits driven by the use of in public sector.

This paper explores the perception of benefits of public sector websites, when almost every organization provides a website and the users are increasingly more knowledgeable and demanding. This study contributes knowledge regarding measurement of benefits perceived from a practitioner’s (webmasters and business managers) point of view. An examination of prior research reveals a lack of specific studies on this topic, which makes this contribution even more relevant in filling a gap in the research literature about the perception and measurement of benefits driven by websites. Regarding practical implications, this study aims to provide the public sector and other agencies with extensive knowledge concerning the benefits of a website and the important aspects to be aware of in ongoing development and quality improvements.

The rest of the paper is structured as follows: Sect. 2 presents a literature review concerning assessment of benefits driven by the public sector website. Section 3 concerns the use of research methods in the present study while Sect. 4 presents the findings. Section 5 provides a discussion of the findings and, finally, Sect. 6 concludes and provides recommendations for subsequent research contributions.

2 Literature Review

While many public organizations invest considerably and devote countless hours of manpower in the process of quality improvement, we speculated that there still exists an overall lack of knowledge with reference to the use of benefits criteria and assessment methods. In order to justify investment in human-computer interaction (HCI), an organization should demonstrate tangible and accountable benefits for the users. In addition, public organizations should demonstrate the discernible internal organizational benefits that a website may create. Lindgaard and Millard [5] stated that investments in HCI can be translated into such benefits as increased productivity, reduced customer support, higher customer satisfaction and brand equity. Although this would be the ideal scenario, it is not the case in many organizations today.

Gupta and Jana [6] distinguish between hard and soft measures in the benefits driven by use of IT. They argue that, due to the constant interaction between public authorities and users, a combination of hard measures (e.g. cost benefits analysis, benchmarking) and soft measures (e.g. qualitative intangible measures) is more appropriate in addressing eGovernment projects.

Flak et al. [7] identified two distinct approaches for measuring benefits of IT: cutting administrative costs and providing added value for the users, while Bannister and Connolly [8] discussed public sector values in a broader sense by focusing on aspects such as respect for the citizen, transparency, integrity and honesty. Heeks [9] is more precise and detailed, classifying benefits into five categories: producing output at lower total cost; increasing output; decreasing output time; higher quality; and new outputs. These measure effectiveness and efficiency, and may generate values for both the organization itself and the users (citizens and businesses).

Because of the increasing focus on eGovernment projects worldwide, there has been increasing research into the measurement of benefits. A number of these studies test models in measuring eGovernment benefit. For example, Edrees and Mahmood [10] provide an empirical test of an adaptation of DeLone and McLean’s information systems success model in the context of eGovernment. Findings showed that all factors are important for understanding eGovernment success. Rana et al. [11] used an integrated information systems success model including such constructs as system quality, information quality, service quality, perceived usefulness, perceived ease of use, perceived satisfaction, perceived risk, and behavioral intention. Results showed a strong correlation between the eight constructs, intention to use and user satisfaction.

From a stakeholder perspective, Osman et al. [12] investigated four main constructs: cost; benefit; risk and opportunity. Results from this study confirm that the framework is a useful approach for evaluating the success of e-government services from the citizens’ perspective. Other studies are more focused in that they investigate a specific area within eGovernment. For example, Xinli [13] investigated the effectiveness of IT in reducing corruption in China; Arendsen et al. [14] evaluated business-to-government systems in use in the Netherlands; and Lamberti et al. [15] explored benefits sought by citizens and channel attitudes for multichannel payment services in Italy.

3 Research Method

This paper draws on both qualitative and quantitative data in order to combine different data sources pertaining to the topic of interest. The strengths of both data types give us a great opportunity to understand a complex and widely discussed issue [16]. The use of methods in this study gives a great opportunity for combining different data sources when investigating the topic of interest. Moreover, if the conclusions from each of the methods are the same, then validity of the study is strengthened [16].

Regarding the qualitative data, ten open-ended interviews were conducted among employees (webmasters and business managers) responsible for public sector websites in Norway. The qualitative interviews took place in the informant’s work place. Notes were taken during the interviews and analyzed immediately afterwards. The researcher preferred not to give the respondents a text-book definition of benefits but opted instead to give the respondents the freedom to express their personal concept and understanding of the term. Consequently, the aim of the interview was to get insights from a practitioner’s perspective without leading them in a certain direction. The researcher strived for a comfortable setting and informants were told the purpose of the interviews and how the data would be treated afterwards.

The informants included in the present study represent different types of public sector organizations providing online information and services to the citizens. What they had in common is that all of them are located close to Norway’s capital city, Oslo, and represent large and well-known organizations with many website users on a daily basis. The questionnaire used during these interviews contained various questions about the organizations’ website presentation. For the purpose of the present study, we have given special attention to the questions/answers concerning benefits of websites in order to identify trends and patterns in the findings, rather than dig into details.

Kvale [17] states that a common critique of interview studies is that the findings are not generalizable because there are too few subjects. However, the aim of this study has not been to challenge or test a theory, rather, to gain insights and knowledge about a relevant topic that, from a literature point of view, lacks references.

In addition to the interviews conducted, the present study also draws on data obtained through an online survey questionnaire involving public sector webmasters in Norway. The survey was inspired by the IS success model from DeLone and McLean [18] in the use of constructs (use of measures). This model is widely cited within information systems (IS) research [19], and for the purpose of collecting data within the public sector. In order to meet this criterion, a pilot-study and interviews was conducted (we went through the questions with experienced webmasters) before the final questionnaire was sent out to all the participants. For the number of respondents included in this paper, N = 308. For each of the questions in the questionnaire, the measurement scale ranged from “very low degree” to “very high degree.” The answer alternative “not relevant” was also included as an option. In order to stick to the topic of the present paper, we have excluded the questions concerning issues other than organizational benefits.

4 Findings

Firstly, through the qualitative interviews we found that user satisfaction is the most important factor in the provision of online information and digital services. Public organizations aim to fulfill the users’ requirements and needs in websites, resulting in ease in finding information and completing tasks. Although the public sector largely holds a monopoly on most information and services, we found that satisfying the users (citizens and businesses) is a vital criterion of success. Website design, content provision, services and technical issues play an important role in this. Facing the fact that the users today are increasingly more demanding, we need to create high quality interactions. Consequently, the website must facilitate great user experiences in the dialogue between citizens and the public sector.

Secondly, on the subject organizational benefits, the provision of online services reduces manual work and increase efficiency and effectiveness, by minimizing help-desk calls and reducing paperwork. Through the organizational effects driven by the website, one of the informants explained that user satisfaction and use of the website decreases the number of inquiries by phone and e-mail: “This saves us time and effort. Answering e-mails is time-consuming, and an object is therefore to reduce the number of inquiries in the future”. Furthermore, the use of digital self-services results in more rapid feedback to the users (citizens), and from an organizational perspective there are also benefits: “The processing time increases and provides the employees with a better overview of the information flow. We spend a lot of time on organizational changes for increasing effectiveness, and we no longer work as we used to. We need to change old routines and create new ones, as well as automating processes where possible and rather spend more time on what we cannot automate”. From this, we see that the use of Web services has a large impact and influence on the organizational performance.

Another point of discussion is whether, because of website efficiency, resources internal to the organizations can be used somewhere else. However, it is not always easy to measures the actual benefits driven by the website performance: We want to become a transparent organization and all the time we are aware of the users. During the process we went through, due to a redesign of the website, we spent a long time discussing how to make it easy for the users. Our focus is to increase the number of digital self-services, but it is difficult to say if this will decrease the number of inquiries by phone. However, this is our goal! While in other cases it might be easier: “Other effects to measure may be decreasing inquiries by phone and mail, as well as the number of downloads of electronic forms and information on the website”.

In addition to effectiveness measures, the findings also reveal that it is beneficial for the employees to provide a website of great quality (including content provision and digital services). When the website appears as an internal knowledge base, the information and services provided can be used by the employees in order to solve their own tasks and by the users to achieve their (and/or the organizations’) goals by visiting the website. Another advantage was pointed out: “When a journalist calls, it is easy to refer to the website for more information”. But, this requires well presented, relevant and updated information.

Thirdly, the findings reveal that the branding effect is an important contributor in the perception of benefits. The websites create a window of opportunity in the provision of online information and services, resulting in a more transparent and informative organization for the users. Providing a website of high quality that fulfills the user’s requirements and expectations will most likely result in great marketing exposure, a good reputation among citizens, and also among employees, who can be proud of the website offered. The website can also be a channel for staff recruitment: “The website is also important in relation to recruit staff to the municipality, for if citizens say that our website is completely hopeless and that you cannot find anything, it will not create a good reputation for the municipality as an organization and place to work”.

Consequently, the findings in the qualitative study reveal that benefits driven by public sector websites range from, on the one hand, the website acting as a brand for the organization, in public relations and for marketing purposes, to, on the other hand, the website contributing to efficiency and effectiveness within the organization and increased productivity for the users.

In contrast to these findings, which show that satisfying the website users is perceived to be benefit number one, and that the organizations aim to provide consistently high quality information and services for the users, there is an absence of HCI activity in most organizations. Although the public sector is highly user-focused and strives to fulfill expectations and needs of citizens, real users are included to a surprisingly low extent in website development and continuous quality improvement. However, there are organizations that are great examples of user inclusion and usability testing, although the main trend among public sector organization reveals great potential for improvement, in both the frequency of testing and the use of methods in such activities.

Table 1 provides the findings from the quantitative study of the extent to which the website contributes benefits for the organization. Findings from Table 1 show that offering information and services without having a personal interaction with the users, and the reduced number of inquiries by phone, e-mail and personal visits, score high, along with organizational benefits related to more time to complete daily tasks within the organization. The website also contributes to an easier way of conveying information and services for the users and is easier to get in touch with, and have contact with, the users. In contrast to this, and to a lower extent, the website contributes to a reduction in organizational costs (e.g. staff reduction, automatic processes).

Table 1. Perception of benefits driven by public sector websites
Full size table

5 Discussion

The aim of combing different empirical components (both qualitative and quantitative data) has been to accomplish a complementary analysis in order to extend our knowledge and understand the topic of interest. Table 2 summarizes the qualitative and the quantitative findings provided.

Table 2. Summarizing the findings from the qualitative and quantitative study
Full size table

Firstly, in our digital community the use of IT has increased during the last decade [1], and happens to be an important contribution to the digitalization of the public sector [2]. Therefore, we need to highlight and discuss the benefits in this regard, and how the website actually contributes. The use of the Web as a channel for communication and interaction with citizens and businesses should also provide the users with better and more efficient access to public information and online services [3]. There are numerous ways to evaluate the benefits of IT [4] and different approaches can be applied. However, we can speculate that many organizations lack empirical evidence of how their website actually performs and, at first, they struggle in identifying the outcome of the benefits.

In assessing user satisfaction, the users of public information and digital services are found to be highly important and seen as the most important benefit perceived from an organizational point of view. In line with this, from the quantitative survey we find that the users are perceived to be satisfied with public sector websites when they can easily access content and digital services. Consequently, we should expect a high degree of HCI activity involved in continuous development and quality improvements, as this is found to be vital in this regard [e.g. 20, 21].

Regarding organizational impacts [see 7, 8], the findings in our study show that measures related to efficiency and effectiveness are frequently mentioned, and these benefits are mainly linked to aspects such as reduced manual work within in the organization, for instance by providing online schemes that the users can fill in online, instead of on paper. This results in more time to complete other tasks and a reduced working load. This is in line with the quantitative findings, which confirm what the website contributes to this. The website can also act as a knowledge base for both employees and external website users (e.g. citizens). This may contribute to a better flow of information (internal and external), in addition to better employee satisfaction.

Concerning the branding effect, the website can additionally stimulate public relations and marketing. Providing a website of great quality may create a good reputation for the organization (and the public sector in general) within our digital society, with increasing online information and digital services.

This study is not without limitations: the qualitative study had only ten open-ended interviews and a high degree of interpretation of the findings. However, findings from both the qualitative and quantitative study show that user satisfaction is found to be most important benefit from the webmasters’ perspectives. This is followed by efficiency and effectiveness, and, finally, the branding effect. The fact that the qualitative and quantitative findings complement each other increases the validity of our study [see 16].

6 Conclusion

This study confirms that websites increase internal efficiency, which is in line with the strategies and goals launched by governments. A website contributes significantly towards facilitating daily tasks, which in turn releases resources which can then be allocated to other duties. This is often viewed as cost efficiency through staff reduction and organizational prioritization of allocation of resources. Additionally, this study highlights the branding and marketing effect as an additional benefit in increasing the reputation among users. A successful implementation of useful public electronic services bears witness to the line of thinking that users should be the prime focus and that it is essential to incorporate a public value perspective.

Upcoming research contributions could advantageously focus on how efficiency and effectiveness are perceived from a user’s point of view, and what aspects of a website are particularly important to them. Moreover, it would be interesting to investigate the differences between public and private sector websites, both from a user’s and an organizational point of view.