Skip to main content

Advertisement

Log in

Onboarding inexperienced developers: struggles and perceptions regarding automated testing

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Previous research found that inexperienced software engineers may tend to view automatic testing as a waste of time and as an activity completely separate from programming. This could have a negative impact on their later careers and could be a sign that improvements in software engineering education are needed when it comes to testing. At the same time, this stance could negatively influence the perception that practitioners have of recent university graduates. To explore this issue, we conducted a qualitative study and surveyed 170 and interviewed 22 practitioners about their experiences with recent graduates, focusing on software testing skills. We find that practitioners do recognize a skill gap between university graduates and industry expectations and that this perception could be engrained deeply enough already to influence hiring practices. Practitioners use different and at times costly strategies to alleviate this skill gap, such as training and mentoring efforts. We validated core findings in a survey with 698 professional software developers. Our qualitative insights can help industry, research, and educational institutions guide in-depth studies that explore the severity of the effects we have found. The coping strategies we have found can provide valuable starting points that can inform changes in how we educate the software engineers of the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com

  2. The statements not depicted are:

    (2) In hiring, we’re having trouble finding software developers that meet our needs.

    (10) Testing is an integral part of software engineering.

    (16) Testing is something that is separate from actual software development.

  3. For example, technologies like Adobe Flash go in and out of fashion, but one would not want a whole generation of software engineers being trained solely in how to work with Adobe Flash.

  4. The compilation of a comprehensive list of quality attributes is left to future work.

  5. Sometimes, companies make their products available as open source in order to show off their high-quality standards and attract capable contributors (Pham et al. 2013).

  6. The company Airbnb shared their guidelines for newcomers about using javascript and how to test it, https://github.com/airbnb/javascript#testing.

References

  • Begel, A., & Simon, B. (2008a). Novice software developers, all over again. In Proceedings of the fourth international workshop on computing education research (pp. 3–14). ACM.

  • Begel, A., & Simon, B. (2008b). Struggles of new college graduates in their first software development job. In ACM SIGCSE Bulletin (Vol. 40, pp. 226–230). ACM.

  • Boud, D., & Feletti, G. (1998). The challenge of problem-based learning. Hove: Psychology Press.

    Google Scholar 

  • Brechner, E. (2003). Things they would not teach me of in college: What microsoft developers learn later. In Companion of the 18th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications (pp. 134–136). ACM.

  • Bruegge, B. (1992). Teaching an industry-oriented software engineering course. In C. Sledge (Ed.), Software engineering education, lecture notes in computer science (Vol. 640, pp. 63–87). Berlin: Springer. doi:10.1007/3-540-55963-9_40.

  • Carrington, D., & Kim, S. K. (2003). Teaching software design with open source software. In FIE 2003 33rd annual frontiers in education, 2003 (Vol. 3, p. S1C-9). IEEE.

  • Dagenais, B., Ossher, H., Bellamy, R. K., Robillard, M. P., & De Vries, J. P. (2010). Moving into a new software project landscape. In Proceedings of the 32nd ACM/IEEE international conference on software engineering (Vol. 1, pp. 275–284). ACM.

  • Deak, A., & Sindre, G. (2014). Analyzing the importance of teaching about testing from alumni survey data. Norsk informatikkonferanse (NIK) 2013.

  • DeMarco, T., & Lister, T. (1987). Peopleware: Productive people and teams. Oxford: Dorset House.

    Google Scholar 

  • Edwards, S. H. (2003a). Rethinking computer science education from a test-first perspective. In Companion of the 18th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications (pp. 148–155). ACM.

  • Edwards, S. H. (2003b). Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In Proceedings of the international conference on education and information systems: technologies and applications EISTA (Vol. 3).

  • Elbaum, S., Person, S., Dokulil, J., & Jorde, M., (2007). Bug hunt: Making early software testing lessons engaging and affordable. In 29th international conference on software engineering, 2007. ICSE 2007 (pp. 688–697). IEEE.

  • Fagerholm, F., Guinea, A.S., Münch, J., & Borenstein, J. (2014). The role of mentoring and project characteristics for onboarding in open source software projects. In Proceedings of the 8th ACM/IEEE international symposium on empirical software engineering and measurement (p. 55). ACM.

  • Fagerholm, F., Johnson, P., Guinea, A. S., Borenstein, J., & Munch, J. (2013). Onboarding in open source software projects: A preliminary analysis. In 2013 IEEE 8th international conference on global software engineering workshops (ICGSEW) (pp. 5–10). IEEE.

  • Garousi, V., & Felderer, M. (2016). Developing, verifying, and maintaining high-quality automated test scripts. IEEE Software, 33(3), 68–75.

    Article  Google Scholar 

  • Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research (Vol. 3). Chicago: Aldine de Gruyter.

    Google Scholar 

  • Glass, R. L., Collard, R., Bertolino, A., Bach, J., & Kaner, C. (2006). Software testing and industry needs. IEEE Software, 23(4), 55–57.

    Article  Google Scholar 

  • Janzen, D. S., & Saiedian, H. (2006). Test-driven learning: Intrinsic integration of testing into the CS/SE curriculum. In In ACM SIGCSE bulletin (Vol. 38, pp. 254–258). ACM.

  • Johnson, M., & Senges, M. (2010). Learning to be a programmer in a complex organization: A case study on practice-based learning during the onboarding process at google. Journal of Workplace Learning, 22(3), 180–194.

    Article  Google Scholar 

  • Jones, E. L. (2001). An experiential approach to incorporating software testing into the computer science curriculum. In 31st annual frontiers in education conference, 2001 (Vol. 2, p. F3D-7). IEEE.

  • Jones, E. L., & Chatmon, C. L. (2001). A perspective on teaching software testing. In Journal of computing sciences in colleges, consortium for computing sciences in colleges (Vol. 16, pp. 92–100).

  • Kaner, C., & Padmanabhan, S. (2007). Practice and transfer of learning in the teaching of software testing. In 20th conference on software engineering education and training, 2007. CSEET’07 (pp. 157–166). IEEE.

  • Lübke, D., Flohr, T., & Schneider, K. (2004). Serious insights through fun software-projects. In T. Dingsøyr (Ed.), Software process improvement, lecture notes in computer science (Vol. 3281, pp. 57–68). Berlin: Springer. doi:10.1007/978-3-540-30181-3_6.

  • Lustosa Neto, V., Coelho, R., Leite, L., Guerrero, D. S., & Mendonça, A. P. (2013). Popt: A problem-oriented programming and testing approach for novice students. In 2013 35th international conference on software engineering (ICSE) (pp. 1099–1108). IEEE.

  • Marrero, W., & Settle, A. (2005). Testing first: Emphasizing testing in early programming courses. In ACM SIGCSE bulletin (Vol. 37, pp. 4–8). ACM.

  • Ng, S., Murnane, T., Reed, K., Grant, D., & Chen, T. (2004). A preliminary survey on software testing practices in Australia. In Software engineering conference, 2004. Proceedings. 2004 Australian (pp. 116–125). IEEE.

  • Pham, R., Kiesling, S., Liskin, O., Singer, L., & Schneider, K. (2014). Enablers, inhibitors, and perceptions of testing in novice software teams. In 22th international symposium on the foundations of software engineering (FSE 2014), Hong Kong, China. (to appear).

  • Pham, R., Mörschbach, J., & Schneider, K. (2015a). Communicating software testing culture through visualizing testing activity. In 7th international workshop on social software engineering (SSE 2015). ACM, SSE 2015. doi:10.1145/2804381.2804382.

  • Pham, R., Singer, L., Liskin, O., Figueira Filho, F., & Schneider, K. (2013). Creating a shared understanding of testing culture on a social coding site. In International conference on software engineering (ICSE) (pp. 112–121). IEEE Press.

  • Pham, R., Stoliar, Y., & Schneider, K. (2015b). Automatically recommending test code examples to inexperienced developers. In Proceedings of the 10th joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (ESEC/FSE 2015). ACM, ESEC/FSE 2015. doi:10.1145/2786805.2803202.

  • Radermacher, A., Walia, G., & Knudson, D. (2014). Investigating the skill gap between graduating students and industry expectations. In Companion Proceedings of the 36th international conference on software engineering (pp. 291–300) ACM.

  • Rajlich, V. (2013). Teaching developer skills in the first software engineering course. In 2013 35th international conference on software engineering (ICSE) (pp. 1109–1116). IEEE.

  • Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

    Google Scholar 

  • Shepard, T., Lamb, M., & Kelly, D. (2001). More testing should be taught. Communications of the ACM, 44(6), 103–108.

    Article  Google Scholar 

  • Sim, S. E., & Holt, R. C. (1998). The ramp-up problem in software projects: A case study of how software immigrants naturalize. In Proceedings of the international conference on software engineering, 1998 (pp. 361–370). IEEE.

  • Singer, L., Figueira Filho, F., Cleary, B., Treude, C., Storey, M. A., & Schneider, K. (2013). Mutual assessment in the social programmer ecosystem: An empirical investigation of developer profile aggregators. In Proceedings of the 2013 conference on computer supported cooperative work. CSCW ’13 (pp. 103–116). NY: ACM. doi:10.1145/2441776.2441791.

  • Sowe, S. K., Stamelos, I., & Deligiannis, I. (2006). A framework for teaching software testing using f/oss methodology. Berlin: Springer. (In Open Source Systems).

    Book  Google Scholar 

  • Spacco, J., & Pugh, W. (2006). Helping students appreciate test-driven development (TDD). In Companion to the 21st ACM SIGPLAN symposium on object-oriented programming systems, languages, and applications (pp. 907–913). ACM.

  • Steinmacher, I., Wiese, I. S., Conte, T., Gerosa, M. A., & Redmiles, D. (2014). The hard life of open source software project newcomers. In Proceedings of the 7th international workshop on cooperative and human aspects of software engineering (pp. 72–78). ACM.

  • Stuart, H. C., Dabbish, L., Kiesler, S., Kinnaird, P., & Kang, R. (2012a). Social transparency in networked information exchange: A theoretical framework. In Proceedings of the ACM 2012 conference on computer supported cooperative work. CSCW ’12 (pp. 451–460). New York, NY: ACM. doi:10.1145/2145204.2145275.

  • Stuart, H. C., Dabbish, L., Kiesler, S., Kinnaird, P., & Kang, R. (2012b). Social transparency in networked information exchange: A theoretical framework. doi:10.1145/2145204.2145275.

    Google Scholar 

  • van Deursen, A., Nederlof, A., & Bouwers, E. (2013). Teaching software architecture: With GitHub!. Retrieved August 24, 2015, from http://avandeursen.com/2013/12/30/teaching-software-architecture-with-github/.

  • Whittaker, J. A., Arbon, J., & Carollo, J. (2012). How Google tests software. Boston: Addison-Wesley.

    Google Scholar 

  • Zagalsky, A., Feliciano, J., Storey, M. D., Zhao, Y., & Wang, W. (2015). The emergence of github as a collaborative platform for education. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, CSCW 2015, Vancouver, BC, Canada (pp. 1906–1917), March 14–18, 2015. doi:10.1145/2675133.2675284.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Raphael Pham.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham, R., Kiesling, S., Singer, L. et al. Onboarding inexperienced developers: struggles and perceptions regarding automated testing. Software Qual J 25, 1239–1268 (2017). https://doi.org/10.1007/s11219-016-9333-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-016-9333-7

Keywords

Navigation