Skip to main content
Log in

Are Individual Differences in Software Development Performance Possible to Capture Using a Quantitative Survey?

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software engineering is human intensive. Thus, it is important to understand and evaluate the value of different types of experiences, and their relation to the quality of the developed software. Many job advertisements focus on requiring knowledge of, for example, specific programming languages. This may seem sensible at first sight, but is it really possible to capture software development performance using this kind of simple measure? On the other hand, maybe it is sufficient to have general knowledge in programming and then it is enough to learn a specific language within the new job. Two key questions are (1) whether prior knowledge of a specific language actually does improve software quality and (2) whether it is possible to capture performance using simple quantitative measures? This paper presents an empirical study where the experience, for example with respect to a specific programming language, of students is assessed using a quantitative survey at the beginning of a course on the personal software process (PSP), and the outcome of the course is evaluated, for example, using the number of defects and development time. Statistical tests are used to analyze the relationship between experience/background and the performance of the students in terms of software quality. The results are mostly unexpected, for example, we are unable to show any significant relation between experience in the programming language used and the number of defects detected.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Boehm, B. W., 1981. Software Engineering Economics. Englewood, NJ, USA: Prentice-Hall.

    Google Scholar 

  • Briand, L. C., El Emam, K., and Morasca, S. 1996. On the application of measurement theory in software engineering. Empirical Software Engineering: An International Journal 1(1): 61–88.

    Google Scholar 

  • Brooks, F. 1980. Studying programmer behavior experimentally: The problems of proper methodology. Communications of the ACM 23(4): 207–213.

    Google Scholar 

  • Brooks, F. 1987. No silver bullet: Essence and accidents of software engineering. IEEE Computer 20(4): 10–20.

    Google Scholar 

  • Curtis, B. 1980. Measurement and experimentation in software engineering. Proceedings of IEEE 68(9), 1144–1157.

    Google Scholar 

  • Ferguson, P., Humphrey, W. S., Khajenoori, S., Macke, S. and Matvya, A. 1997. Results of applying the personal software process. IEEE Computer 30(5): 24–31.

    Google Scholar 

  • Höst, M., Regnell, B., and Wohlin, C. 2000. Using students as subjects-a comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering: An International Journal 5(3): 201–214.

    Google Scholar 

  • Humphrey, W. S. 1995. A Discipline for Software Engineering. Reading, MA: Addison Wesley.

    Google Scholar 

  • Humphrey, W. S. 1997. Introduction to the Personal Software Process. Reading, MA: Addison Wesley.

    Google Scholar 

  • Humphrey, W. S. 1996. Using a defined and measured personal software process. IEEE Software, 77-88, May.

  • Kachigan, S. K. 1986. Statistical Analysis: An Introduction to Univariate &; Multivariate Methods. New York: Radius Press.

    Google Scholar 

  • Montgomery, D. C. 1997. Design and Analysis of Experiments. 4th edition, New York, NY: John Wiley &; Sons.

    Google Scholar 

  • Robson, C. 1993Real World Research: A Resource for Social Scientists and Practitioners-Researchers. Blackwell.

  • Wesslén, A. 2000. A replicated empirical study of the impact of the methods in the PSP on individual engineers. Empirical Software Engineering: An International Journal 5(2): 93–123.

    Google Scholar 

  • Wohlin, C. 1998. The personal software process as a context for empirical studies. IEEE TCSE Software Process Newsletter 12, pp. 7–12, Spring.

    Google Scholar 

  • Wohlin, C., and Wesslén, A. 1998. Understanding software defect detection in the personal software process. Proceedings IEEE 8th International Symposium on Software Reliability Engineering, pp. 49-58, Paderborn, Germany.

  • Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. Experimentation in Software Engineering?An Introduction. Boston, USA: Kluwer Academic Publishers.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wohlin, C. Are Individual Differences in Software Development Performance Possible to Capture Using a Quantitative Survey?. Empirical Software Engineering 9, 211–228 (2004). https://doi.org/10.1023/B:EMSE.0000027780.08194.b0

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:EMSE.0000027780.08194.b0

Navigation