Skip to main content
SpringerLink
Log in

An experimental investigation comparing individual and collaborative work productivity when using desktop and cloud modeling tools

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Successful modeling tools need to effectively support individual as well as team-based work (collaboration) within colocated and virtual environments. In the past, achieving this has been challenging, since traditional modeling tools are desktop-based and thus suitable for individual and colocated work only. However, with the rise of web-based architectures and the cloud paradigm, desktop modeling tools now have rivals in their web-based counterparts that are especially suited for online collaboration (e-collaboration). The objective of our research was to probe the question as to ‘which type of modeling tools (desktop or cloud-based) performs better in cases of individual work and e-collaboration’, and to obtain insights into the sources of the strengths and weaknesses regarding both types of modeling tools. A controlled experiment was performed in which we addressed two types of modeling tools—desktop and cloud-based, in respect to two types of work—individual and e-collaboration. Within these treatments, we observed the productivity of 129 undergraduate IT students, who performed different types of modeling activities. The experimental participants reported no statistical significant differences between self-reported expertise with the investigated tools as well as their overall characteristics. However, they did finish individual and e-collaborative activities faster when using cloud modeling tool, where significant differences in favor of the cloud modeling tool were detected during e-collaboration. If we aggregate the results, we can argue that cloud modeling tools are comparable with desktop modeling tools during individual activities and outperform them during e-collaboration. Our findings also correlate with the related research, stating that with the use of state-of-the-art Web technologies, cloud-based applications can achieve the user experience of desktop applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Alharbi ST (2012) Users’ acceptance of cloud computing in Saudi Arabia: an extension of technology acceptance model. Int J Cloud Appl Comput 2:1–11. doi:10.4018/ijcac.2012040101

    Google Scholar 

  • Basili VR, Caldiera G, Rombach HD (1994) Goal question metric paradigm. Encyclopedia of software engineering. Wiley-Interscience, pp 528–532

  • Benlian A, Hess T (2011) Opportunities and risks of software-as-a-service: findings from a survey of IT executives. Decis Support Syst 52:232–246. doi:10.1016/j.dss.2011.07.007

    Article  Google Scholar 

  • Bibi S, Katsaros D, Bozanis P (2012) Business application acquisition: on-premise or SaaS-based solutions? IEEE Softw 29:86–93. doi:10.1109/MS.2011.119

    Article  Google Scholar 

  • Bland JM, Altman DG (1995) Multiple significance tests: the Bonferroni method. BMJ 310:170

    Article  Google Scholar 

  • Briand LC, Differding CM, Rombach HD (1996) Practical guidelines for measurement-based process improvement

  • Chebrolu SB (2012) How do cloud capabilities impact various aspects of IT effectiveness? 2012 I.E. 5th International Conference on Cloud Computing (CLOUD). pp 932–940

  • Chieu TC, Mohindra A, Karve AA, Segal A (2009) Dynamic scaling of web applications in a virtualized cloud computing environment. IEEE 281–286

  • Chinosi M, Trombetta A (2011) BPMN: an introduction to the standard. Comput Stand Interfaces. doi:10.1016/j.csi.2011.06.002

  • Clason D, Dormody T (1994) Analyzing data measured by individual Likert-type items. J Agric Educ 35:31–35

    Article  Google Scholar 

  • Dale V (2012) Usability for desktop apps. http://help.utest.com/testers/crash-courses/usability/usability-for-desktop-apps. Accessed 25 Jul 2012

  • De Valck K, van Bruggen GH, Wierenga B (2009) Virtual communities: a marketing perspective. Decis Support Syst 47:185–203. doi:10.1016/j.dss.2009.02.008

    Article  Google Scholar 

  • Desisto RP, Pring B (2011) Essential SaaS overview and 2011 guide to SaaS research. 15

  • Dillon T, Wu C, Chang E (2010) Cloud computing: issues and challenges. IEEE 27–33

  • Dixon W, Yuen K (1974) Trimming and winsorization: a review. Stat Pap 15:157–170. doi:10.1007/BF02922904

    MathSciNet  MATH  Google Scholar 

  • Du J, Lu J, Wu D et al (2013) User acceptance of software as a service: evidence from customers of China’s leading e-commerce company Alibaba. J Syst Softw. doi:10.1016/j.jss.2013.03.012

  • Dyba T, Kitchenham BA, Jorgensen M (2005) Evidence-based software engineering for practitioners. IEEE Softw 22:58–65. doi:10.1109/MS.2005.6

    Article  Google Scholar 

  • Ellis C, Wainer J (1994) A Conceptual model of groupware. Proceedings of the 1994 ACM conference on Computer supported cooperative work. pp 79–88

  • Ellis CA, Gibbs SJ, Rein G (1991) Groupware: some issues and experiences. Commun ACM 34:39–58. doi:10.1145/99977.99987

    Article  Google Scholar 

  • Ferreira A, Antunes P (2007) A technique for evaluating shared workspaces efficiency. Proceedings of the 10th international conference on Computer supported cooperative work in design III. Springer, Berlin, pp 82–91

    Google Scholar 

  • Fuks H, Raposo AB, Gerosa MA, Lucena CJP (2005) Applying the 3C model to groupware development. Int J Cooperative Inf Syst 299–328

  • García-Magariño I, Fuentes-Fernández R, Gómez-Sanz JJ (2010) A framework for the definition of metamodels for computer-aided software engineering tools. Inf Softw Technol 52:422–435. doi:10.1016/j.infsof.2009.10.008

    Article  Google Scholar 

  • Gerosa MA, Fuks H, Lucena C (2003) Analysis and design of awareness elements in collaborative digital environments: a case study in the AulaNet learning environment. J Interact Learn Res 14:315–332

    Google Scholar 

  • Godse M, Mulik S (2009) An approach for selecting Software-as-a-Service (SaaS) product. IEEE 155–158

  • Holden RJ, Karsh B-T (2010) The technology acceptance model: its past and its future in health care. J Biomed Inform 43:159–172. doi:10.1016/j.jbi.2009.07.002

    Article  Google Scholar 

  • Holzinger A, Mayr S, Slany W, Debevc M (2010) The influence of AJAX on Web usability. e-Business (ICE-B), Proceedings of the 2010 International Conference on. Athens, pp 124–127

  • Iosup A, Ostermann S, Yigitbasi MN et al (2011) Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans Parallel and Distrib Syst 22:931–945. doi:10.1109/TPDS.2011.66

    Article  Google Scholar 

  • ISO 9126–4 (2004) ISO/IEC TR 9126-4-Software engineering, product quality, quality in use metrics. International Organization for Standardization

  • Ju J, Wang Y, Fu J et al (2010) Research on key technology in SaaS. IEEE 384–387

  • Katzmarzik A (2011) Product differentiation for Software-as-a-Service providers. Bus Inf Syst Eng 3:19–31. doi:10.1007/s12599-010-0142-4

    Article  Google Scholar 

  • Keller A, Hüsig S (2009) Ex ante identification of disruptive innovations in the software industry applied to web applications: the case of Microsoft’s vs. Google’s office applications. Technol Forecast Soc Chang 76:1044–1054. doi:10.1016/j.techfore.2009.03.005

    Article  Google Scholar 

  • Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. 65

  • Kitchenham BA, Pfleeger SL, Pickard LM et al (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28:721–734. doi:10.1109/TSE.2002.1027796

    Article  Google Scholar 

  • Kuhn DL (1989) Selecting and effectively using a computer aided software engineering tool. Annual Westinghouse computer symposium, Pittsburgh, PA (USA)

  • Legris P, Ingham J, Collerette P (2003) Why do people use information technology? A critical review of the technology acceptance model. Inf Manag 40:191–204

    Article  Google Scholar 

  • Mader S (2007) Wikipatterns, 1st ed. Wiley

  • Mamčenko J (2004) Introduction to lotus notes collaborative software

  • Mann J (2011) Cloud computing means new opportunities and decisions for collaboration. 4

  • Marston S, Li Z, Bandyopadhyay S et al (2011) Cloud computing—the business perspective. Decis Support Syst 51:176–189. doi:10.1016/j.dss.2010.12.006

    Article  Google Scholar 

  • Martin WE, Bridgmon KD (2012) Quantitative and statistical research methods: from hypothesis to results, 1st ed. Jossey-Bass

  • Melao N, Pidd M (2000) A conceptual framework for understanding business processes and business process modelling. Inform Syst J 10:105–129. doi:10.1046/j.1365-2575.2000.00075.x

    Article  Google Scholar 

  • Mell P, Grance T (2011) The NIST definition of cloud computing (Draft). 7

  • Mili H, Tremblay G, Jaoude GB et al (2010) Business process modeling languages: sorting through the alphabet soup. ACM Comput Surv 43:4:1–4:56. doi:10.1145/1824795.1824799

    Article  Google Scholar 

  • Min D, Koo S, Chung Y-H, Kim B (1999) Distributed GOMS: an extension of GOMS to group task. 1999 I.E. International Conference on Systems, Man, and Cybernetics, 1999. IEEE SMC’99 Conference Proceedings. IEEE, pp 720–725 vol.5

  • Mohagheghi P (2010) An approach for empirical evaluation of model-driven engineering in multiple dimensions—SINTEF. Paris, France, pp 6–17

  • Neuman WL (2005) Social research methods: qualitative and quantitative approaches, 5th ed. Allyn & Bacon

  • Opitz N, Langkau TF, Schmidt NH, Kolbe LM (2012) Technology acceptance of cloud computing: empirical evidence from German IT Departments. 2012 45th Hawaii International Conference on System Science (HICSS). pp 1593–1602

  • Oz E (2005) Information technology productivity: in search of a definite observation. Inf Manag 42:789–798. doi:10.1016/j.im.2004.08.003

    Article  Google Scholar 

  • Patel H, Pettitt M, Wilson JR (2012) Factors of collaborative working: a framework for a collaboration model. Appl Ergon 43:1–26. doi:10.1016/j.apergo.2011.04.009

    Article  Google Scholar 

  • Quinn LS (2010) Comparing online vs. traditional office software. In: TechSoup. http://www.techsoup.org/learningcenter/software/page11852.cfm. Accessed 27 Jul 2012

  • Roca JC, Chiu C-M, Martínez FJ (2006) Understanding e-learning continuance intention: an extension of the technology acceptance model. Int J Hum Comput Stud 64:683–696. doi:10.1016/j.ijhcs.2006.01.003

    Article  Google Scholar 

  • Schmidt K (1991) Riding a tiger, or computer supported cooperative work. ECSCW’91 Proceedings of The Second European Conference on Computer-Supported Cooperative Work 1–16

  • Schmietendorf A (2008) Assessment of business process modeling tools under consideration of business process management activities. IWSM/Metrikon/Mensura. pp 141–154

  • Schuman S (2006) Creating a culture of collaboration: The International Association of Facilitators Handbook, 1st ed. Jossey-Bass

  • Serçe FC, Swigger K, Alpaslan FN et al (2011) Online collaboration: collaborative behavior patterns and factors affecting globally distributed team performance. Comput Hum Behav 27:490–503. doi:10.1016/j.chb.2010.09.017

    Article  Google Scholar 

  • Shapiro R, White SA, Bock C, et al. (2011) BPMN 2.0 handbook second edition: methods, concepts, case studies and standards in business process modeling notation. Future strategies, incorporated

  • Sheriff PD (2002) Designing for web or desktop? In: http://msdn.microsoft.com. http://msdn.microsoft.com/en-us/library/ms973831.aspx. Accessed 29 Mar 2012

  • SPRING Singapore (2011) A guide to productivity measurement. SPRING Singapore, Singapore

    Google Scholar 

  • Sun A (2013) Enabling collaborative decision-making in watershed management using cloud-computing services. Environ Model Softw 41:93–97. doi:10.1016/j.envsoft.2012.11.008

    Article  Google Scholar 

  • Thong JYL, Hong W, Tam K-Y (2002) Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context, and individual differences? Int J Hum Comput Stud 57:215–242. doi:10.1016/S1071-5819(02)91024-4

    Article  Google Scholar 

  • Torchiano M, Ricca F, Marchetto A (2010) Are web applications more defect-prone than desktop applications? Int J Softw Tools Technol Transfer 13:151–166. doi:10.1007/s10009-010-0182-6

    Article  Google Scholar 

  • Trochim W, Donnelly JP (2006) The research methods knowledge base, 3rd ed. Atomic dog

  • van Ommeren E, Duivestein S, deVadoss J et al (2009) Collaboration in the cloud: how cross-boundary collaboration is transforming business. Microsoft and Sogeti, Groningen

    Google Scholar 

  • VMware (2009) Solving the desktop dilemma with user-centric desktop virtualization for the enterprise. 6

  • Wang Y (2006) E-collaboration—a literature analysis. Intelligent production machines and systems. Elsevier, pp 132–137

  • Wang L, Tao J, Kunze M, et al. (2008) Scientific cloud computing: early definition and experience. IEEE 825–830

  • Winter JCF, Dodou D (2010) Five-point Likert items: t test versus Mann–Whitney-Wilcoxon. Pract Assess Res Eval 15:16

    Google Scholar 

  • Wu W-W (2011a) Mining significant factors affecting the adoption of SaaS using the rough set approach. J Syst Softw 84:435–441. doi:10.1016/j.jss.2010.11.890

    Article  Google Scholar 

  • Wu W-W (2011b) Developing an explorative model for SaaS adoption. Expert Syst Appl 38:15057–15064. doi:10.1016/j.eswa.2011.05.039

    Article  Google Scholar 

  • Xu X (2012) From cloud computing to cloud manufacturing. Robot Comput Integr Manuf 28:75–86. doi:10.1016/j.rcim.2011.07.002

    Article  Google Scholar 

  • Yan Z, Reijers HA, Dijkman RM (2010) An evaluation of BPMN modeling tools. In: Mendling J, Weidlich M, Weske M (eds) Business process modeling notation. Springer, Berlin, pp 121–128

    Chapter  Google Scholar 

  • Yang S, Yoo B, Jahng J (2010) Does the SaaS model really increase customer benefits? Asia Pac J Inf Syst 20:87–101

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, University of Maribor, Maribor, Slovenia

    Gregor Polančič, Gregor Jošt & Marjan Heričko

Authors
  1. Gregor Polančič
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregor Jošt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marjan Heričko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gregor Polančič.

Additional information

Communicated by: Natalia Juristo

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 231 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Polančič, G., Jošt, G. & Heričko, M. An experimental investigation comparing individual and collaborative work productivity when using desktop and cloud modeling tools. Empir Software Eng 20, 142–175 (2015). https://doi.org/10.1007/s10664-013-9280-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-013-9280-x

Keywords

Search

Navigation