Abstract
This article discusses the role of formal experiments in empirical software engineering. I take the view that the role of experiments has been overemphasised. Laboratory experiments are not representative of industrial software engineering tasks, so do not provide us with a reliable assessment of the effect of our techniques and tools. I suggest we need to concentrate a larger proportion of our research effort on industrial quasi-experiments and case studies. Methodologies for these empirical methods are well-understood in the social science and would appear to be appropriate mechanisms for investigating many software engineering research questions. In addition, I believe we need to make the results of empirical software engineering more visible and relevant to practitioners. To influence practitioners I suggest that we need to produce evidence-based text books and evidence-based software engineering standards.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jørgensen, M., Carelius, G.J.: An Empirical Study of Software Project Bidding. IEEE Transactions of Software Engineering 30(12), 953–969 (2004)
Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences. A practical Guide. Blackwell Publishing, Oxford (2006)
Shaddish, W.R., Cook, T.D., Campbell, D.T.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, Boston (2002)
El Emam, K., Birk, A.: Validating ISO/IEC 15504 measures of software development process capability. The Journal of Systems and Software 51, 119–149 (2000a)
El Emam, K., Birk, A.: Validating ISO/IEC 15504 measures of software Requirements Analysis Process Capability. IEEE Transactions on Software Engineering 26(6), 541–566 (2000b)
Dion, R.: Process Improvement and the Corporate Balance Sheet. IEEE Software 10(4), 28–35 (1993)
Steen, H.U.: Reporting framework-based software process improvement. A quantitative and qualitative review of 71 experience reports of CMM-based SPI. Master Thesis, Simula Research Laboratory & Department of Informatics University of Oslo, 29th October (2004)
McGarry, F., Burke, S., Decker, B.: Measuring the Impacts Individual process Maturity Attributes have on Software Products. In: Proceedings Fifth International Software Metrics Symposium, IEEE Computer Society, pp. 52–60. IEEE Computer Society Press, Los Alamitos (1998)
Yin, R.K.: Case Design and Methods, 3rd edn. Sage Publications, Thousand Oaks (2004)
Kitchenham, B., Budgen, D., Breton, P., Turner, M., Charters, S., Linkman, S.: Large Scale Software Engineering Questions – Expert Opinion or Empirical Evidence? Experience and Methods from Integrating Evidence-based Software Engineering into Education. In: Proceedings 4th International Workshop WSESE2003, Fraunhofer IESE-Report No., 068.06/E (2006)
Rainer, A., Jagielska, D., Hall, T.: Software Practice versus evidence-based software engineering research. In: Proceedings of the Workshop on Realising Evidence-based Software Engineering, ICSE-2005 (2005), http:/cfm.portal.acm.org/dl
Endres, A., Rombach, H.D.: Empirical Software and Systems Engineering: A Handbook of Observations, Laws and Theories. Addison Wesley, Reading (2003)
Kitchenham, B.: Procedures for Undertaking Systematic Reviews. Joint Technical Report, Computer Science Department, Keele University (TR/SE-0401) and National ICT Australia Ltd. (0400011T.1) (2004)
ISO/IEC 25000. International Standard. Software Engineering – Software product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE (2005-08-01)
Koyani, S.J., Bailey, R.W., Nall, J.R.: Research based web design and usability guidelines. National Cancer Institute (2003), Available for download at http://usability.gov/pdfs/guidelines.html
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Kitchenham, B. (2007). Empirical Paradigm – The Role of Experiments. In: Basili, V.R., Rombach, D., Schneider, K., Kitchenham, B., Pfahl, D., Selby, R.W. (eds) Empirical Software Engineering Issues. Critical Assessment and Future Directions. Lecture Notes in Computer Science, vol 4336. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71301-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-71301-2_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71300-5
Online ISBN: 978-3-540-71301-2
eBook Packages: Computer ScienceComputer Science (R0)