Abstract
Purpose
Digital libraries are not only an assortment of information assets yet have turned into a digital community for correspondence, searching and electronic learning. Also, economically the investment in terms of money, time, energy and manpower associated with the development of effective digital library (DL) systems demands high utilization of these resources. This study aims to explore the factors that affect the utilization of digital libraries and may lead to users’ satisfaction and finally high exposure to information systems like digital libraries. However, these factors may work differently in different cultures. Considering this fact, DeLone and McLean’s IS success model (ISSM) is tested and expanded in a local academic context.
Design/methodology/approach
Following the quantitative research design, a total of 355 responses were collected through a questionnaire-based survey. Research scholars of the University of Punjab, Lahore, Pakistan were the reviewed population of this study. A two-stage stratified random sampling method was used to choose the sample. Structural equation modelling is used to find out the nature and extent of the relationship among studied variables.
Findings
The findings confirm that service quality is the strong predictor of DL system use, whereas overall users’ satisfaction mediates the relationship between the predictors (content and service quality) and the outcome variable (use).
Originality/value
This work done is the first main endeavour to use the “Information System Success” theories to intervene and mediate the effect of content, IQ, system quality and service quality on the use of DL in the local context.
Keywords
Citation
Iqbal, M., Rafiq, M. and Soroya, S.H. (2022), "Examining predictors of digital library use: an application of the information system success model", The Electronic Library, Vol. 40 No. 4, pp. 359-375. https://doi.org/10.1108/EL-01-2022-0008
Publisher
:Emerald Publishing Limited
Copyright © 2022, Emerald Publishing Limited
Introduction
Libraries in general are viewed as significant social organizations that assist people with different information sources. American researchers presented the idea of advanced digital libraries during the mid-1990s. A digital library (DL) is an advanced-level library administration that uses information technology (IT) (Li et al., 2019). Alongside the quick advancement in IT, DLs are as of now not simply an assortment of information assets but have turned into a digital community for correspondence, user searching and electronic learning. Therefore, numerous training centres have developed their DLs to upgrade both research and learning (Malapela and De Jager, 2018).
Due to drastic changes in the information industry and DLs accordingly, the information needs and behaviours of users have changed. The emphasis is on easy admittance and access to the information resources, moreover an assumption for association with the information providers (Xu and Du, 2018). Inability to deal with these said necessities and practices will direct DLs to an abuse of resources (Carlock and Perry, 2008).
Simultaneously, these days, DLs as information providers face savage and solid competition (Ross and Sennyey, 2008) and need a closer focus on quality evaluation (Heradio et al., 2012). According to Kiran and Diljit (2011), users might be inclined towards Web search tools including search engines on DLs to access online information to fulfil their required information.
Hence, at this point, DLs do not want to be a digital information centre for their stakeholders and so their loyalty to DLs will significantly drop. Thus, at this stage, DLs must begin agonizing over user loyalty (Keshvari et al., 2015). User loyalty is influenced by user satisfaction (US) directly or in indirect ways (Townley and Boberg, 1997). Generally speaking, the satisfaction that users acquire the subsequent use of DLs is deliberated as an indicator of users’ intent to use the DL services again (Xu and Du, 2018). Hence, users who are satisfied with the DL services can, accordingly, convey viral impacts if they endorse the use of DLs to their family, friends and associates (Reichheld, 2003). Additionally, by and large, US with university DL will immensely affect users’ engagement with DLs (Priporas et al., 2019).
Study context
The Higher Education Commission in Pakistan developed the National DL (HEC-NDL) system in Pakistan in the year 2004 (Said, 2006), which is clearly used for research motives. The HEC-NDL project has admitted over 75,000 digital contents. These contents include full-text digital databases, e-books, e-journals and other digital configurations that are accessed by stakeholders from public/private and non-profit organizations including both research and development entities all around Pakistan. The HEC-NDL program provides access to a huge quantity of scholarly digital content to research scholars from the different recognized data suppliers; for example, Cambridge University Press, Emerald, JStor, McGraw-Hill DL, ScienceDirect, SpringerLink, Taylor and Francis E-Journals (InformaWorld), Wiley Interscience and many others. It also offers 45,000 e-books through various sources, such as Ebrary, McGraw-Hill Collections and Springer E-books [Higher Education Commission-National Digital Library (HEC-NDL), 2021].
Statistics show a huge expansion in the quantity of complete downloaded digital content from the preceding years (Said, 2006) that has had a determined scholastic effect across the country. Subsequently, an assessment of the use of the HEC-NDL information system (IS) is important to justify the financing. The use may be the most robust and the most significant indicator to measure success in any DL IS. Accordingly, the HEC-NDL offers suitable study settings to compute the actual use and mediation effects of users’ satisfaction.
Past studies have used technology adoption theories including the IS success model (ISSM) to examine the impact of certain particular factors on US and loyalty to DLs (Chang, 2013). Nonetheless, inadequate research work has been executed to study DL IS’ use by applying various IS success models and their mediation effects. No examination could be found measuring the impact of the reformulated DeLone and McLean ISS theory with the additional construct of “content” in the DL environment which is considered core for any DL (Carter, 2002).
Locally, a reasonable size of research-based literature on DLs (Iqbal and Warraich, 2012) and databases (Iqbal and Ullah, 2016) are available but the focus is on US and not on the factors affecting the use of DL IS. Although, in the Pakistani context, recently IS related to land records (Malik et al., 2016), enterprise resource planning (ERP) in higher education institutions (Khand and Kalhoro, 2020) and media organizations (Ashraf and Baig, 2021) are studied considering the DeLone and McLean model of ISS which was verified in all mentioned studies, however, no study has been conducted so far to understand the HEC-NDL success predictors from a theoretical lens of the ISSM, even though the ISSM has been applied to study DL use in other parts of the world (Table 2). Recently, a meta-analysis of studies based on the DeLone and McLean ISSM confirmed that the relationships between variables brought different outcomes in different countries with different cultural backgrounds which highlights the need to apply this model in various contexts (Ibrahim et al., 2021). Hence, the current investigation with an additional construct (content), is an effort to expand the ISS theory that may help to identify the influencing factors of DL’s success in the Pakistani context. For this purpose, use is apportioned as a dependent variable and overall US is playing the part as a mediator in the current study. Thus, a significant goal of the current investigation is to uncover the different factors that impact the disinclination/inclination to use a DL in Pakistan. It also validates and expands the DeLone and McLean’s (2003) ISS model concerning the DL systems in Pakistan.
Theoretical framework
The IS department assists the end-user, and service quality aims to capture this notion. This is done by those who want to establish their systems through provisioning services to different entities. Hence, the IS department plays a part both as an information provider and as a service provider (Rosemann and Vessey, 2004). IS are assessed to identify the factors that positively impact the system’s success.
In this regard, different theories are available, particularly DeLone and McLean’s IS success model (ISSM), which was developed in 1992 to measure organizational IS success. It comprises six constructs including information quality (IQ), system quality (SQ), US, individual impact (II), organizational impact (OI) and use (U). In 2002, DeLone and McLean composed a survey paper. This work was considered based on ten years of updates and assessments of their original ISSM. They added another construct called service quality (ser. q) in the new IS success model to fulfil the need for measuring support in IS implementation success. Furthermore, they added the dimension called use rather than intention to use (IU) to evaluate the users’ attitudes. Finally, they combined the constructs of II and OI as another construct by the name of net benefits (NB). As a result, the final ISSM consists of the constructs IQ, SQ, ser. q, U, US and NB. The constructs of this model that has been used in the current study are explained in Table 1.
The study published in 1992 is cited 15,483 times (as cited by Google Scholar, as of 3 January 2022). DeLone and McLean’s ISSM has been used generally to assess technology adoption in various study contexts. These include various settings, such as electronic government (Abdulkareem and Ramli, 2021; At-tamimi and Siregar, 2021), online learning systems (Çelik and Ayaz, 2021; Nuryanti et al., 2022), English as a foreign language course (Hsu, 2021), campus portals (Masrek, 2007) and the banking sector (Purwati et al., 2021).
This model has been successfully used in the academic context as well. Reviewed literature has shown numerous studies from various countries about the use of ISSM to evaluate the ISS of digital libraries (DLs) or e-libraries (Table 2).
The current study is based upon DeLone and McLean’s ISSM and has been executed to check the way in which three distinctive quality dimensions (IQ, SQ and service quality) with the incorporation of another dimension content can impact the user’s satisfaction to result in the impact on the use of DLs.
Hypotheses development and components of the proposed research model
System quality
SQ is a measure to calculate the user-friendliness of the system (Bharati and Berg, 2003; Doll and Torkzadeh, 1988). Normally, SQ is evaluated on behalf of the dimensions including flexibility, functionality, data importance, ease of use, reliability integration and quality (DeLone and McLean, 2003). Generally, researchers use ease of use to address SQ to build an SQ foundation, so, additionally, another facet that contributes to measuring the SQ is also ease of use (Doll and Torkzadeh, 1988; Shin, 2003). A user’s IU a particular system can escalate depending on characteristics of the user interface, such as screen design, terminology and navigation (Ramayah, 2006). As Cheng (2014) has indicated users are conscious of the SQ and researchers rely on that specific SQ and compatibility to prompt US (Lwoga, 2013; Rana et al., 2015). Hence, it is expected that SQ leads to US and behavioural usage intentions (Chen, 2015). There were similar positive results determined in previous studies by Rana et al. (2015) and Lwoga (2013). Consequently, it has been assumed that SQ will impact an individual’s satisfaction and ultimately the IU the DL system as follows:
SQ has a significant positive effect on overall users’ satisfaction with the DL IS.
SQ has a significant positive effect on the use of DL IS.
Information quality
IQ alludes to the system’s output value that may be perceived by the system’s respective users (Negash et al., 2003). On average, IQ is measured concerning consistency, completeness, accuracy, relevance and timeliness. Likewise, IQ has been demonstrated to be a noticeable success factor to examine the overall ISS, especially with regards to Web-based online systems (Schaupp et al., 2006). Literature on IS success determines that IQ has effects on information satisfaction (DeLone and McLean, 2004; Park and Kim, 2008). According to a couple of research studies, IQ is determined as a comprehensive entity of US, not as a different component (Bailey and Pearson, 1983) or the satisfaction of the users is related to the information (Iivari, 1987). To outline the connection between IQ and US or behavioural IU various researchers, such as Shaltoni et al. (2015) and Lwoga (2013), determined that IQ significantly affects the US and a user’s IU the Web-based online learning systems. Therefore, the researchers propose that US and behaviour to use the DL IS both are affected by IQ:
IQ has a significant positive effect on overall users’ satisfaction with the DL IS.
IQ has a significant positive effect on the use of the DL IS.
Service quality
Service quality is related to the user’s emotional evaluation and subjective assessment of the service expectations (Ahn et al., 2004). It very well may be successfully evaluated in terms of reliability, follow-up service, empathy, confidence, competence and responsiveness. DeLone and McLean (2002) added the service quality to measure the IS success; thus, a lot of consideration has been given to the updated ISSM (Stylianou and Kumar, 2000). Masrek (2007) found that using correlation analysis shows strong relationships exist between service quality and US. Comparable evidence was determined in studies by Lwoga (2013) and Shaltoni et al. (2015). Studies have also tracked down the significant connection with the system use, therefore the following hypotheses are proposed:
Service quality has a significant positive effect on the overall users’ satisfaction with the DL IS.
Service quality has a significant positive effect on the use of DL IS.
Content
From a research perspective, digital libraries are content collected and organized entities on the behalf of user communities (Borgman, 2000). Content defines the intellectual content of the document which means that content characterizes the scholarly substance of the document or, in other words, what the document is all about. In a DL, content may be diverse and heterogeneous in nature and other aspects. Typical DL collections are these information contents that might be commercially acquired content as well as locally created/published resources. Thus, content is an important element of a DL (Carter, 2002). Therefore, it was assumed that:
Content positively affects the overall users’ satisfaction with the DL IS.
Content positively affects the use of DL IS.
Overall user satisfaction
US is viewed as one of the significant aspects to measure the IS’s success. As indicated by DeLone and McLean (1992) US and system use (U) are associated. A widely implemented US measurement tool named end-user computing satisfaction (EUCS) was created by Doll and Torkzadeh (1988). Another comprehensive US model for small business users was established by Palvia (1996) through the implementation of IT. He counted the IQ, service quality, security and hardware adequacy as antecedents of a user’s satisfaction. DeLone and McLean (1992) recommended a comprehensive order of six significant IS success dimensions. Among these six dimensions, the US was the most broadly used particular success dimension. Seddon and Kiew (1994) defined US as “pleasant or unpleasant user feelings’ related to the benefits which [an] individual likes to accomplish on account of collaborating with a specific information system”. On a comparative note, US was defined by Doll and Torkzadeh (1988) as to how users feel concerning using specific computer applications. These feelings are firmly connected to how accommodating the system was intended to address user’s needs and requirements (Seddon and Kiew, 1994). Chen and Chengalur Smith (2015) found that in educational settings when using an objective, the system brings about a great user experience and, hence, the user feels satisfied. Thus, the user is urged to reuse the system. In previous studies, a comparative relationship was found by Cheng (2014), Ismail et al. (2012) and Lwoga (2013). Finally, overall satisfaction comes from content, SQ, IQ and service quality is likely to increase the use of the DL IS. However, the constructed hypothesis is as follows:
The overall US positively influences the use of a DL.
The graphical representation of the proposed model is available in Figure 1.
Research design and methodology
This explanatory research has used the quantitative approach to test the developed hypotheses. The IS success including DL mainly depends upon the continuous use by the users (Rezvani et al., 2017). Therefore, all the research students (MPhil and PhD) in the academic year of 2018 from the University of Punjab comprise the research population. The University of Punjab is ranked overall second among the top ten universities in the country (HEC, 2016), and it is one of the oldest universities in Pakistan (established in 1982). Therefore, it was decided to study the University of the Punjab research scholars HEC-NDL usage patterns through the lens of ISSM. These research scholars were the users of HEC-NDL. It was assumed that all the students were good users of the DL since they are research scholars. A two-stage stratified random sampling technique was used. There were 13 faculties at the University of the Punjab, Lahore, Pakistan. At the first stage, four strata including the faculty of Behavioral and Social Sciences, Arts and Humanities, Life Sciences, Economics and Management Sciences departments were selected purposively as these faculties were assumed to represent almost all major disciplines. In the second stage, respondents were selected on a random basis. In total, 230 PhD and 865 MPhil students were enrolled in the institutes/departments/colleges of the selected faculties. The sample size was calculated by using Yamane’s (1967, p. 99) formula separately for PhD and MPhil students. As a result, the PhD sample size was 146 and the MPhil-based sample size was 274. This sample was chosen randomly proportionally based on faculty strength and then based on each department’s strengths. Finally, a random table was used to select the respondents from each department. The total sample size was 420, whereas the total useable responses were 355 and the response rate was 85%.
The measurement scale (Appendix) was developed based on DeLone and McLean’s (2003), Joshi’s (1990) and Doll and Torkzadeh’s (1988). Only researchers were well aware of the category of each factor of the scale and their arrangement. Each item was rated on the scale of five-point Likert type scale, one to five, respectively, correspond to the “strongly disagree” to “strongly agree”. The instrument was pilot tested before final data collection based on 50 responses. The Cronbach’s alpha (CA) values remained between 0.7 and 0.8 for all the constructs separately (Table 3).
Data collection procedure
Data were collected when the COVID-19 lockdown was over. Permission letters were taken from the concerned bodies to smoothly execute the procedure. The researchers personally visited the units (departments/institutes/centres/schools) of the selected faculties. With the prior permission of concerned authorities, the respondents who were not available on campus were approached through e-mail. There were 355 usable questionnaires and the response rate was 85%.
Collected data were analysed with the help of ADANCO (2.1.0). Partial least squares (PLS) and structural equation modelling (SEM) were used to validate the proposed model.
Results
Model measurement
The study was based on reflective constructs; therefore, ADANCO Model A was applied for the analysis. To measure the reliability of reflective constructs with various indicators, the ADANCO 2.0.1 suggests three reliability coefficients; that is, CA (Cronbach, 1951), Dijkstra–Henseler’s rho (PA) (Dijkstra and Henseler, 2015) and composite reliability (CR) (Werts et al., 1978), whereas values > 0.7 are acceptable for CR and CA (Chin, 2010; Hair et al., 2017). However, rho A coefficient values > 0.6 are within an acceptable range (Dijkstra and Henseler, 2015; Schuberth et al., 2018). Data presented in Table 4 confirm that CA, CA and rho A are higher than 0.7; therefore, the data was further analysed for PLS and SEM.
Convergent validity
The psychometric quality of the scale was assessed through convergent validity measures. The average variance extracted (AVE) greater than 0.5 is acceptable. The data in Table 5 confirms that all the constructs measured were measuring the same phenomena, as AVE values were above the threshold value (i.e. > 0.05). Factor loadings for all the constructs remained satisfactory (i.e. > 0.7); however, two statements, one for IQ (statement H3) and the other related to use (statement H4) were dropped due to having values lower than the threshold (i.e. ≥ 0.7).
Discriminant validity
HTMT 0.85 is considered a reliable measure to ensure that all the constructs are statistically different from each other. Data presented in Table 6 confirm that all values were below the threshold value of 0.85 confirming the convergent validity of the scale.
Hypotheses testing
The results presented in Table 7 confirm that five out of the nine hypotheses were accepted at p ≤ 0.01. A detailed overview of the results revealed that although content (β = 0.07, t = 0.15, p = 0.15) and SQ (β = −0.01, t = −0.28, p = 0.78) have no significant impact on the use of IS, however, IQ (β = 0.17, t = 3.16, p = 0.00) and service quality (β = 0.14, t = 2.79, p = 0.01) significantly enhance the use of IS.
On the other side, content (β = 0.31, t = 5.78, p = 0.00) and service quality (β = 0.15, t = 2.59, p = 0.01) have a significant positive impact on the overall US and overall US enhances the use of IS (β = 0.53, t = 11.44, p = 0.00). Thus, it mediates the relationship between content, service quality and use of IS. Whereas SQ (β = −0.05, t = −0.73, p = 0.47) and IQ (β = 0.10, t= −1.61, p = 0.11) have no direct impact on overall US. The graphical presentation of the relationships among the variables is presented in Figure 2.
Discussion
The study offered interesting results, as five out of the nine hypotheses are upheld. At first, IQ and service quality profoundly improve the use of digital library information systems (DLIS). Also, DL content and service quality decidedly affect the overall US (OUS), and OUS, at last, improves the use of DLIS. Thus, OUS mediates the relationship among content, service quality and use of DLIS. However, SQ remained insignificant in this regard. Similarly, in this way unmistakably service quality has an impact on DL use. The outcomes are reliable with past study findings in various settings including mobile websites and mobile commerce (Vance et al., 2008; Zhou, 2011). The service quality has an impact on the use of DL systems both in direct and indirect ways.
Therefore, library professionals and information service providers simultaneously need to organize the DL services and exercises, such as face-to-face direction, information retrieval contests, training about the use of DLs to assist with diminishing users’ efforts and time to retrieve the required information. Information service providers and librarians’ co-ops must give ingenious, classy and well-designed DLs, having an unmistakable layout, reliable system and successful navigation to upgrade the DLIS’s quality. In the meantime, they must work on the nature and quality of DL services provided to university students, including research scholars, to assist them to enhance their satisfaction with the further and continuous use of the DL. Moreover, they need to guarantee the reliability of DL websites.
Furthermore, information professionals and librarians must fortify correspondences with DL users and get users’ experience information to further develop the DL service quality. For instance, librarians might use online and real-time tools, such as Facebook, WhatsApp, WeChat, Twitter and Ask-A-Librarian, to provide in-time responses to the DL users. What is more, library and information professionals may deliver customized and personalized services to DL users to enhance its use and apparent value by leveraging users’ preferences, location and cookies data. By and by, library administrators ought to acquire earlier consent from the DL users to keep away from any possible infringement of the user’s privacy.
Additionally, library professionals and information services providers need to give proficient and customized information resources to their users since the distinction between an effective and an inadequate DL is the quality of its services provided to its users and the quality of information resources available to its users through it.
Hence, overall, information professionals must improve DL use by creating reliable, clear and effective DL websites, providing time, speedily customized and proficient DL services and justify by introducing exceptional, precise and comprehensive information resources. It is necessary to work for the progress and effectiveness of the DL in a way that student and research scholars will be ensured that a particular DLIS can assist them with working proficiently in their life and work through the availability of quality information effortlessly. Similarly, information professionals should deliver the services in a manner with the availability of DLIS use training material on the DL websites so that users can feel that it is not difficult to figure out how to use the DLIS. Summarizing, library information professionals and specialist co-ops should make every effort within reach to further develop the overall US so users will make use of the advanced DLIS continually.
Implications for research and practices
Theoretical implications
Users’ continuous use of a DL depends upon their level of satisfaction with the DL (Gefen, 2002; Lin and Wang, 2006). The consequences of this research work can assist analysts and researchers with advancing their comprehension of the variables impacting digital libraries’ US, ultimately enhancing the use of a DLIS. Second, this study has upgraded the information on US in the field of advanced DLIS. Extensive work done is available in regards to DL US mainly based upon IT reception theories, such as the IS success theory by DeLone and McLean (2003), Joshi’s (1990) overall US model and the EUCS model by Doll and Torlzadeh (1988). In the current study, a blend of these ISS theories is developed to recognize the principal factors influencing overall US that affect the use of a DLIS. It might be helpful for researchers to better understand the connections and relationships among these constructs.
Additionally, this study is the first endeavour to use the described theories to intervene and mediate the impact of content, SQ, IQ and service quality on DL use. Overall US can better explain and fully mediate the impacts of content and service quality on the use of a DLIS.
Implication for practices
This work gives some directions to the library administrators and specialists to pay attention to building US and DL use. The current research exploration demonstrates that service quality has a huge impact on DL use through US. It means service quality applies both in direct and indirect ways even though content and IQ are likewise important to enhance the use of a DLIS.
Library and information specialists must ensure the users’ insight about using digital libraries is captured. In the current study, DL user IQ, content and service quality are quite helpful and significant to further develop the overall US to improve the continuous use of a DL.
For instance, information professionals dealing with digital libraries in universities can foster a literature searching plan as a part of the students’ course project or research plans and propose information retrieval instructional classes and training programs. This kind of service quality and administrations will not just empower the university students, especially research scholars, to figure out how to use a DLIS. However, it would assist them with understanding the usefulness, and the significance of digital libraries. It will ensure them the convenience that a successful DLIS will bring them in their research and learning process.
Furthermore, library administrators might invest some energy into working on simplifying the DL’s interface and making DL services more appealing and simpler to use accordingly to stimulate the users to constantly use the DL services. It will steadily develop US and it will ultimately boost the use of the DL. This research work offers guidelines for local and other developing countries’ academic institutions to plan and devise effective DLs.
Conclusion and future work
To recognize the factors influencing users’ satisfaction to enhance the DL use, IS success (ISS) theories-based survey research was conducted. Given the study outcomes, it can be claimed that the ISS theory by DeLone and McLean, Joshi’s (1990) overall US model and the EUCS model of Doll and Torlzadeh (1988) can serve as a consolidated, combined, valuable and useful theoretical basis to predict and comprehend or understand the factors that influence users’ satisfaction and DL use. Even though the results of this research work are substantial, the outcomes might be applied carefully for the following reasons.
Initially, this work is executed in Pakistan where DLs are growing and developing quickly yet in their essential and primary stage. Simultaneously, there may be an absence of cross-national and cultural diversity. The users are presumably impacted by their cultures and societies. Along these lines, the study outcomes should be generalized, to sum up, different nations with developed DLs and diverse cultural societies.
Secondly, the explained variance of DL use is 36%. In this way, there might be factors other than US impacting the use of DLs. These may be a peer and social effect, trust, self-efficacy and privacy concerns which ought to be measured in a study later on in the future.
Finally, this study has analysed just the factors decidedly adding to increase the US to increase the DL use, yet it did not investigate negative factors responsible for reducing US and devotion that ultimately lessen DL use. Therefore, because of these limits, there is the indication of a need and a requirement for a further study: a study that will report both positive and negative factors.
Figures
Description of DeLone & McLean’s ISSM constructs used in the current study
| Sr. # | Constructs | Construct description |
|---|---|---|
| 1 | Information quality | This is the quality of a particular system that a system can store, deliver or produce |
| 2 | System quality | Usability features of the system |
| 3 | Service quality | A user’s subjective assessment that the service they are getting from the portal is the service they expect |
| 5 | Use | This construct determines the utilization of IS through its users |
| 6 | User satisfaction | This construct determines the user’s level of overall satisfaction with a particular system |
Implementation of DeLone and McLean’s ISSM in an educational set-up
| Authors/Year | Country | Sample size | Type of educational information system |
|---|---|---|---|
| Gao (2020) | China | 232 | Digital library |
| Alzahrani et al. (2019) | Malaysia | 978 | Digital library |
| Aldholay et al. (2018) | Yemeni | 448 | Digital library |
| Chen et al. (2016) | Taiwan | 264 | Digital library |
| Shaltoni et al. (2015) | Jordan | 374 | Online portal |
| Chen and Chengalur Smith (2015) | USA | 376 | Online Web portal |
| Samadi et al. (2014) | Iran | 425 | Digital library and virtual communities |
| Cheng (2014) | Taiwan | 378 | E-learning |
| Lwoga (2013) | Tanzania | 408 | Digital library |
Reliability
| Factors | Items | α |
|---|---|---|
| Information quality | 5 | 0.75 |
| System quality | 5 | 0.75 |
| Service quality | 3 | 0.81 |
| Content | 4 | 0.74 |
| Overall user satisfaction | 3 | 0.73 |
| System usage | 4 | 0.74 |
Reliability measures
| Constructs | Dijkstra–Henseler's rho (ρA) | Jöreskog’s rho (ρc) | Cronbach’s alpha (α) |
|---|---|---|---|
| Content | 0.90 | 0.91 | 0.87 |
| Information quality | 0.85 | 0.89 | 0.84 |
| System quality | 0.90 | 0.91 | 0.87 |
| Service quality | 0.82 | 0.89 | 0.82 |
| System usage | 0.77 | 0.87 | 0.77 |
| User satisfaction | 0.82 | 0.88 | 0.82 |
Convergent validity
| Indicator | Loadings | AVE |
|---|---|---|
| Content | 0.72 | |
| Con1 | 0.82 | |
| Con2 | 0.84 | |
| Con3 | 0.87 | |
| Con4 | 0.86 | |
| Information quality | 0.61 | |
| IQ1 | 0.74 | |
| IQ2 | 0.77 | |
| IQ3 | 0.76 | |
| IQ4 | 0.78 | |
| IQ5 | 0.84 | |
| System quality | 0.71 | |
| SQ1 | 0.85 | |
| SQ2 | 0.88 | |
| SQ4 | 0.83 | |
| SQ5 | 0.80 | |
| Service quality | 0.73 | |
| SerQ1 | 0.87 | |
| SerQ2 | 0.85 | |
| SerQ3 | 0.85 | |
| Use | 0.68 | |
| UIU1 | 0.84 | |
| UIU2 | 0.80 | |
| UIU3 | 0.84 | |
| Users’ satisfaction | 0.65 | |
| OUS1 | 0.81 | |
| OUS2 | 0.81 | |
| OUS3 | 0.81 | |
| OUS4 | 0.79 |
Discriminant validity
| Construct | Content | Information quality | System quality | Service quality | System usage | Overall user satisfaction |
|---|---|---|---|---|---|---|
| Content | ||||||
| Information quality | 0.11 | |||||
| System quality | 0.18 | 0.32 | ||||
| Service quality | 0.28 | 0.14 | 0.06 | |||
| System usage | 0.33 | 0.12 | 0.04 | 0.33 | ||
| Overall user satisfaction | 0.36 | 0.13 | 0.03 | 0.27 | 0.71 |
Summary
| Effect | Original coefficient | |||
|---|---|---|---|---|
| t-value | p-value (two-sided) | Remarks | ||
| Content → System usage | 0.07 | 1.46 | 0.15 | Not supported |
| Content → Overall user satisfaction | 0.31 | 5.78 | 0.00 | Supported |
| Information quality → System usage | 0.17 | 3.16 | 0.00 | Supported |
| Information quality → Overall user satisfaction | −0.10 | −1.61 | 0.11 | Not supported |
| System quality → System usage | −0.01 | −0.28 | 0.78 | Not supported |
| System quality → Overall user satisfaction | −0.05 | −0.73 | 0.47 | Not supported |
| Service quality → System usage | 0.14 | 2.79 | 0.01 | Supported |
| Service quality → Overall user satisfaction | 0.15 | 2.59 | 0.01 | Supported |
| Overall user satisfaction → System usage | 0.53 | 11.44 | 0.00 | Supported |
Research instrument
| Code | Scale factors | Adapted from |
|---|---|---|
| Information quality | ||
| 1 | The digital library provides complete information | DeLone and McLean’s (2003) reformulated information system success model (ISSM) |
| 2 | Information on the digital library is easy to understand | |
| 3 | A digital library meets someone’s information requirements | |
| 4 | Digital library provides relevant information | |
| 5 | Digital library provides secure (safe) information | |
| Service quality | ||
| 1 | I am satisfied with the easy way to use the digital library | DeLone and McLean’s (2003) reformulated information system success model (ISSM) |
| 2 | I have empathy (not enjoyable and comfortable) to use the digital library | |
| 3 | I am satisfied with the responsiveness of digital library support | |
| System quality | ||
| 1 | Digital library is easily adaptable to me | DeLone and McLean’s (2003) reformulated information system success model (ISSM) |
| 2 | The digital library on campus is always available to me | |
| 4 | I am satisfied with the digital library response time | |
| 5 | Digital library is usable to me | |
| Use (system usage) | ||
| 1 | I use the digital library for my classwork | DeLone and McLean’s (2003) reformulated information system success model (ISSM) |
| 2 | I navigate the digital library to find the information | |
| 3 | I can visit different sites to find information within the digital library | |
| 4 | I download information from the digital library on every visit | |
| Content | ||
| 1 | Digital library provides the exact information necessary | End-user computing satisfaction (EUCS) model by Doll and Torkzadeh (1988) |
| 2 | Information on the digital library itself satisfies my needs | |
| 3 | The search result on the digital library is almost exactly the thing I need | |
| 4 | Digital library provides plenty of information | |
| Overall user satisfaction | ||
| 1 | I feel the digital library is able to meet my information needs in my area of interest | Joshi’s (1990) overall user satisfaction (OUS) model |
| 2 | I feel the digital library is able to meet the requirements of all the users it serves | |
| 3 | I feel the digital library is efficient | |
| 4 | I feel the digital library is effective | |
Appendix
References
Abdulkareem, A.K. and Ramli, R.M. (2021), “Does digital literacy predict e-government performance? An extension of DeLone and McLean information system success model”, Electronic Government, an International Journal, Vol. 17 No. 4, pp. 466-493.
Ahn, T., Ryu, S. and Han, I. (2004), “The impact of the online and offline features on the user acceptance of internet shopping malls”, Electronic Commerce Research and Applications, Vol. 3 No. 4, pp. 405-420.
Aldholay, A.H., Isaac, O., Abdullah, Z. and Ramayah, T. (2018), “The role of transformational leadership as a mediating variable in DeLone and McLean information system success model: the context of online learning usage in Yemen”, Telematics and Informatics, Vol. 35 No. 5, pp. 1421-1437.
Alzahrani, A.I., Mahmud, I., Ramayah, T., Alfarraj, O. and Alalwan, N. (2019), “Modelling digital library success using the DeLone and McLean information system success model”, Journal of Librarianship and Information Science, Vol. 51 No. 2, pp. 291-306.
Ashraf, A. and Baig, F. (2021), “Evaluating the effectiveness of information systems in media organization: a study of Pakistan news TV”, Journal of Media Studies, Vol. 36 No. 2, pp. 1-34.
At-Tamimi, R.M.R. and Siregar, T.E. (2021), “Measurement of successful implementation of institution level financial application system (SAKTI) web full module with DeLone and McLean information system success model approach”, Budapest International Research and Critics Institute: Humanities and Social Sciences, Vol. 4 No. 4, pp. 10098-10107.
Bailey, J.E. and Pearson, S.W. (1983), “Development of a tool for measuring and analyzing computer user satisfaction”, Management Science, Vol. 29 No. 5, pp. 530-545.
Bharati, P. and Berg, D. (2003), “Managing information systems for service quality: a study from the other side”, Information Technology and People, Vol. 16 No. 2, pp. 183-202.
Borgman, C.L. (2000), “Digital libraries and the continuum of scholarly communication”, Journal of Documentation, Vol. 56 No. 4, pp. 412-430.
Carlock, D.M. and Perry, A.M. (2008), “Exploring faculty experiences with e-books: a focus group”, Library Hi Tech, Vol. 26 No. 2, pp. 244-254.
Carter, D.S. (2002), “Elements of digital libraries, present and future”, DL Journal, Vol. 22 No. March, available at: www.dl.slis.tsukuba.ac.jp/DLjournal/No_22/1-superman/1-superman.html (accessed 30 December 2021).
Çelik, K. and Ayaz, A. (2021), “Validation of the DeLone and McLean information systems success model: a study on student information system”, Education and Information Technologies, Vol. 27 No. 4, pp. 4709-4727.
Chang, C.C. (2013), “Exploring the determinants of e-learning systems continuance intention in academic libraries”, Library Management, Vol. 34 Nos 1/2, pp. 40-55.
Chen, Y.H. (2015), “Testing the impact of an information literacy course: Undergraduates’ perceptions and use of the university libraries’ web portal”, Library and Information Science Research, Vol. 37 No. 3, pp. 263-274.
Chen, Y.H. and Chengalur Smith, I. (2015), “Factors influencing students’ use of a library web portal: Applying course integrated information literacy instruction as an intervention”, The Internet and Higher Education, Vol. 26, pp. 42-55.
Chen, J.F., Chang, J.F., Kao, C.W. and Huang, Y.M. (2016), “Integrating ISSM into TAM to enhance digital library services: a case study of the Taiwan digital Meta-Library”, The Electronic Library, Vol. 34 No. 1, pp. 58-73.
Cheng, Y.M. (2014), “Extending the expectation-confirmation model with quality and flow to explore nurses’ continued blended e-learning intention”, Information Technology and People, Vol. 27 No. 3, pp. 230-258.
Chin, W.W. (2010), “Bootstrap cross-validation indices for PLS path model assessment”, In Handbook of Partial Least Squares, Springer, Berlin, Heidelberg, pp. 83-97.
Cronbach, L.J. (1951), “Coefficient alpha and the internal structure of tests”, Psychometrika, Vol. 16 No. 3, pp. 297-334.
DeLone, W.H. and McLean, E.R. (1992), “Information systems success: the quest for the dependent variable”, Information Systems Research, Vol. 3 No. 1, pp. 60-95.
DeLone, W.H. and McLean, E.R. (2002), “Information systems success revisited”, in 35th Annual HI International Conference on System Sciences (HICSS ‘02), Big Island, HI, 7-10 January, IEEE, pp. 2966-2976.
DeLone, W.H. and McLean, E.R. (2003), “The DeLone and McLean model of information systems success: a ten-year update”, Journal of Management Information Systems, Vol. 19 No. 4, pp. 9-30.
DeLone, W.H. and McLean, E.R. (2004), “Measuring e-commerce success: applying the DeLone and McLean information systems success model”, International Journal of Electronic Commerce, Vol. 9 No. 1, pp. 31-47.
Dijkstra, T.K. and Henseler, J. (2015), “Consistent partial least squares path modeling”, MIS Quarterly, Vol. 39 No. 2, pp. 297-316.
Doll, W.J. and Torkzadeh, G. (1988), “The measurement of end-user computing satisfaction”, MIS Quarterly, Vol. 12 No. 2, pp. 259-274.
Gao, H.L. (2020), “An empirical assessment of digital library service: perspective of DeLone and McLean information system success model”, International Journal of Humanities and Social Science, Vol. 10 No. 4, pp. 70-77.
Gefen, D. (2002), “Customer loyalty in e-commerce”, Journal of the Association for Information Systems, Vol. 3 No. 1, pp. 27-51.
Hair, J., Hollingsworth, C.L., Randolph, A.B. and Chong, A.Y.L. (2017), “An updated and expanded assessment of PLS-SEM in information systems research”, Industrial Management and Data Systems, Vol. 117 No. 3, pp. 442-458.
Heradio, R., Fernández-Amorós, D., Cabrerizo, F.J. and Herrera-Viedma, E. (2012), “A review of quality evaluation of digital libraries based on users’ perceptions”, Journal of Information Science, Vol. 38 No. 3, pp. 269-283.
Higher Education Commission (HEC) (2016), “5th ranking of Pakistani higher education institutions (HEIs) 2015”, available at: https://hec.gov.pk/english/universities/Documents/Ranking_Doc%20(2015).pdf
Higher Education Commission-National Digital Library (HEC-NDL) (2021), available at: www.digitallibrary.edu.pk (accessed 15 February 2021).
Hsu, L. (2021), “What makes good LMOOCs for EFL learners? Learners’ personal characteristics and information system success model”, Computer Assisted Language Learning, doi: 10.1080/09588221.2021.1899243
Ibrahim, B., Glood, S., Bahloos, S. and Abd, S. (2021), “A Meta-analysis review of DeLone and McLean information system success model”, in Proceedings of First International Conference on Mathematical Modeling and Computational Science, Springer, Singapore, pp. 195-208.
Iivari, J. (1987), “User information satisfaction (UIS) reconsidered: an information system as the antecedent of UIS”, in Eighth International Conference on Information Systems, Pittsburgh, PA, pp. 57-73.
Iqbal, M. and Ullah, A. (2016), “Qualitative usability evaluation of databases: a case study”, Library Hi Tech News, Vol. 33 No. 3, pp. 8-10.
Iqbal, M. and Warraich, N.F. (2012), “Usability evaluation of an academic library website: a case of the university of the Punjab”, Pakistan Journal of Information Management and Libraries, Vol. 13, pp. 1-11.
Ismail, N.Z., Razak, M.R., Zakariah, Z., Alias, N. and Aziz, M.N.A. (2012), “E-learning continuance intention among higher learning institution students’ in Malaysia”, Procedia – Social and Behavioral Sciences, Vol. 67, pp. 409-415.
Keshvari, M., Zare Farashbandi, F. and Geraei, E. (2015), “Modelling influential factors on customer loyalty in public libraries: a study of west Iran”, The Electronic Library, Vol. 33 No. 4, pp. 810-823.
Khand, Z.H. and Kalhoro, M.R. (2020), “Testing and validating DeLone and McLean is model: ERP system success in higher education institutions of Pakistan”, Engineering, Technology and Applied Science Research, Vol. 10 No. 5, pp. 6242-6248.
Kiran, K. and Diljit, S. (2011), “Antecedents of customer loyalty: does service quality suffice?”, Malaysian Journal of Library and Information Science, Vol. 16 No. 2, pp. 95-113.
Li, S., Jiao, F., Zhang, Y. and Xu, X. (2019), “Problems and changes in digital libraries in the age of big data from the perspective of user services”, The Journal of Academic Librarianship, Vol. 45 No. 1, pp. 22-30.
Lin, H.H. and Wang, Y.S. (2006), “An examination of the determinants of customer loyalty in mobile commerce contexts”, Information and Management, Vol. 43 No. 3, pp. 271-282.
Lwoga, T.E. (2013), “Measuring the success of library 2.0 technologies in the African context: the suitability of the DeLone and McLean’s model”, Campus-Wide Information Systems, Vol. 30 No. 4, pp. 288-307.
Malapela, T. and De Jager, K. (2018), “Theories of value and demonstrating their practical implementation in academic library services”, The Journal of Academic Librarianship, Vol. 44 No. 6, pp. 775-780.
Malik, B.H., Shuqin, C., Qamar, S. and Mattiullah, B. (2016), “Examining success of land record information systems (LRMIS) in Pakistan: Validating an incorporated is success model”, European Scientific Journal, ESJ, Vol. 12 No. 2, pp. 258-289.
Masrek, M.N.B. (2007), “Measuring campus portal effectiveness and the contributing factors”, Campus-Wide Information Systems, Vol. 24 No. 5, pp. 342-354.
Negash, S., Ryan, T. and Igbaria, M. (2003), “Quality and effectiveness in web-based customer support system”, Information and Management, Vol. 40 No. 8, pp. 757-768.
Nuryanti, Y., Asbari, M., Nadeak, M., Jainuri, J. and Amri, L.H.A. (2022), “Peran kepemimpinan transformasional dalam information system success model: Analisis praktik e-Learning di perguruan tinggi”, Edukatif : Jurnal Ilmu Pendidikan, Vol. 4 No. 3, pp. 3681-3690.
Palvia, P.C. (1996), “A model and instrument for measuring small business user satisfaction with information technology”, Information and Management, Vol. 31 No. 3, pp. 151-163.
Park, C. and Kim, Y. (2008), “The effect of information satisfaction and relational benefit on consumer’s online shopping site commitment”, Web Technologies for Commerce and Services Online, Vol. 1, pp. 149-169.
Priporas, C.V., Stylos, N. and Kamenidou, I. (2019), “City image, city brand personality and generation Z residents’ life satisfaction under economic crisis: predictors of city-related social media engagement”, Journal of Business Research, Vol. 119, pp. 453-463.
Purwati, A.A., Mustafa, Z. and Deli, M.M. (2021), “Management information system in evaluation of BCA mobile banking using DeLone and McLean model”, Journal of Applied Engineering and Technological Science (Jaets), Vol. 2 No. 2, pp. 70-77.
Ramayah, T. (2006), “Interface characteristics, perceived ease of use and intention to use a digital library in Malaysia”, Information Development, Vol. 22 No. 2, pp. 123-133.
Rana, N.P., Dwivedi, Y.K., Williams, M.D. and Weerakkody, V. (2015), “Investigating success of an e-government initiative: validation of an integrated is success model”, Information Systems Frontiers, Vol. 17 No. 1, pp. 127-142.
Reichheld, F.F. (2003), “The one number you need to grow”, Harvard Business Review, Vol. 81 No. 12, pp. 46-55.
Rezvani, A., Khosravi, P. and Dong, L. (2017), “Motivating users toward continued usage of information systems: self-determination theory perspective”, Computers in Human Behavior, Vol. 76, pp. 263-275.
Rosemann, M. and Vessey, I. (2004), “On the applicability of the Bunge-Wand-Weber ontology to enterprise systems requirements”, in 15th Australasian Conference on Information Systems, pp. 232-235.
Ross, L. and Sennyey, P. (2008), “The library is dead, long live the library! the practice of academic librarianship and the digital revolution”, The Journal of Academic Librarianship, Vol. 34 No. 2, pp. 145-152.
Said, A. (2006), “Accessing electronic information: a study of Pakistan’s digital library”, INSAP, Oxford, available at: www.inasp.info/sites/default/files/2018-04/accessing_electronic_information.pdf
Samadi, I., Masrek, M.N. and Yatin, S.F.B.M. (2014), “The effect of individual characteristics and digital library characteristics on digital library effectiveness: a survey at university of Tehran”, World Applied Sciences Journal, Vol. 30, pp. 214-220.
Schaupp, L.C., Fan, W. and Belanger, F. (2006), “Determining success for different website goals”, in 39th Annual HI International Conference on System Sciences (HICSS ‘06), Kauia, HI, 4-7 January, Vol. 6, IEEE, p. 107b.
Schuberth, F., Henseler, J. and Dijkstra, T.K. (2018), “Partial least squares path modeling using ordinal categorical indicators”, Quality and Quantity, Vol. 52 No. 1, pp. 9-35.
Seddon, P.B. and Kiew, M.Y. (1994), “A partial test and development of the DeLone and McLean model of is success”, in 15th International Conference on Information Systems (ICIS ‘94), Vancouver, Canada, Association for Information Systems, pp. 99-110.
Shaltoni, A.M., Khraim, H., Abuhamad, A. and Amer, M. (2015), “Exploring students’ satisfaction with universities’ portals in developing countries: a cultural perspective”, The International Journal of Information and Learning Technology, Vol. 32 No. 2, pp. 82-93.
Shin, B. (2003), “An exploratory investigation of system success factors in data warehousing”, Journal of the Association for Information Systems, Vol. 4 No. 1, pp. 141-170.
Stylianou, A.C. and Kumar, R.L. (2000), “An integrative framework for is quality management”, Communications of the ACM, Vol. 43 No. 9, pp. 99-104.
Townley, C.T. and Boberg, K. (1997), “The changing role of technical university libraries: 1983-1996”, in Proceedings of the IATUL Conferences, NM State University, Las Cruces, NM.
Vance, A., Elie-Dit-Cosaque, C. and Straub, D.W. (2008), “Examining trust in information technology artifacts: the effects of system quality and culture”, Journal of Management Information Systems, Vol. 24 No. 4, pp. 73-100.
Werts, C.E., Rock, D.R., Linn, R.L. and Jöreskog, K.G. (1978), “A general method of estimating the reliability of a composite”, Educational and Psychological Measurement, Vol. 38 No. 4, pp. 933-938.
Xu, F. and Du, J.T. (2018), “Factors influencing users’ satisfaction and loyalty to digital libraries in chinese universities”, Computers in Human Behavior, Vol. 83, pp. 64-72.
Yamane, T. (1967), Problems to Accompany Statistics, an Introductory Analysis, Harper and Row.
Zhou, T. (2011), “Examining the critical success factors of mobile website adoption”, Online Information Review, Vol. 35 No. 4, pp. 636-652.
Further reading
Alzahrani, A.I., Mahmud, I., Ramayah, T., Alfarraj, O. and Alalwan, N. (2017), “Extending the theory of planned behavior (TPB) to explain online game playing among Malaysian undergraduate students”, Telematics and Informatics, Vol. 34 No. 4, pp. 239-251.
Wu, J.H. and Wang, Y.M. (2006), “Measuring KMS success: a respecification of the DeLone and McLean’s model”, Information and Management, Vol. 43 No. 6, pp. 728-739.
Yun, H., Kettinger, W.J. and Lee, C.C. (2012), “A new open door: the smartphone’s impact on work-to-life conflict, stress, and resistance”, International Journal of Electronic Commerce, Vol. 16 No. 4, pp. 121-152.
Zhang, L., Ye, P. and Liu, Q. (2011), “A survey of the use of electronic resources at seven universities in Wuhan, China”, Program, Vol. 45 No. 1, pp. 67-77.
Acknowledgements
All authors contributed equally to this paper.