Skip to main content
SpringerLink
Log in

Students’ Systems Thinking Competency Level Detection through Software Cost Estimation Concept Modeling

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Systems Thinking Competencies have become extremely important and widely studied due to increasing complexity of technical and societal systems. Because of this, when they are taught, it is extremely useful to identify whether or not students demonstrate Systems Thinking Competencies so that effective teaching strategies can be designed. Through the use of the Adapted Holistic Scoring Method we seek to assess Concept Maps developed by postgraduate and undergraduate engineering students in order to determine their knowledge of Systems Thinking Competencies. Firstly, their background in cost estimation drivers was identified, and secondly, their Systems Thinking Competence proficiency level. We assessed how a short cost estimation training improved specific Systems Thinking Competencies. This research contributes to the body of knowledge because it provides empirical evidence on systems thinking competencies among engineering students in the context of cost estimation. The research was organized in two phases. First, Students demonstrated knowledge of cost estimation drivers, and were asked about specific levels of Systems Thinking Competencies. Second, after students were taught systems thinking competencies, their knowledge of the connection between cost drivers and Systems Thinking Competencies was obtained. A Mann-Whitney U-test was applied in order to identify if there were significant differences between Phase 1 and Phase 2 in both Estimation Cost Drivers included and Systems Thinking Competence Level Demonstrated Results show improvement in a specific Systems Thinking competency – ability to see relationships – among the student population studied (N = 45).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.
Fig. 15.
Fig. 16.
Fig. 17.
Fig. 18.
Fig. 19.
Fig. 20.

Similar content being viewed by others

REFERENCES

  1. Amissah, M., Gannon, T., and Monat, J., What is systems thinking? Expert perspectives from the WPI systems thinking colloquium of 2 October 2019, Systems, 2020, vol. 8, no. 1, p. 6.

    Article  Google Scholar 

  2. Assaraf, O. and Orion, N., Development of system thinking skills in the context of Earth system education, J. Res. Sci. Teach., 2005, vol. 42, pp. 518–560. https://doi.org/10.1002/tea.20061

    Article  Google Scholar 

  3. Assaraf, O. and Orion, N., System Thinking skills at the elementary school, J. Res. Sci. Teach., 2009, vol. 47, pp. 540–563. https://doi.org/10.1002/tea.20351

    Article  Google Scholar 

  4. Ateskan, A. and Lane, J.F., Assessing teachers’ Systems Thinking skills during a professional development program in Turkey, J. Cleaner Prod., 2016, vol. 172, no. 20, pp. 4348–4356. https://doi.org/10.1016/j.jclepro.2017.05.094

    Article  Google Scholar 

  5. Azevedo, R., Moos, D.C., Greene, J.A., Winters, F.I., and Cromley, J.C., Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia?, Edu. Technol. Res. Develop., 2008, vol. 56, pp. 45–72. https://doi.org/10.1007/s11423-007-9067-0

    Article  Google Scholar 

  6. Boehm, B.W., Abts, C., Winsor-Brown, A., Chulani, S., Clark, B.K., Horowitz, E., Madachy, R., Reifer, D.J., and Steece, B., Software Cost Estimation with COCOMO II, Prentice Hall Press, 2009.

    Google Scholar 

  7. Bayram, S., The effectiveness of concept and software mapping for representing student data and process schema in science, Master Thesis, Pittsburgh: Univ. of Pittsburgh, 1995.

  8. Ben-Zvi-Assaraf, O. and Orion, B., Four case studies, six years later: developing system thinking skills in junior high school and sustaining them over time, J. Res. Sci. Teach., 2010, vol. 47, no. 10, pp. 1253–1280.

    Article  Google Scholar 

  9. Brandstädter, K., Harms, U., and GroBschedl, J., Assessing system thinking through different concept-mapping practices, Int. J. Sci. Edu., 2012, vol. 34, no. 14, pp. 2147–2170.

    Article  Google Scholar 

  10. Cañas, A.J. and Novak, J.D., Freedom VS. restriction of content and structure during CAMping-possibilities and limitations for construction and assessment. Concept maps: theory, methodology, technology, Proc. 5th Int. Conf. on Concept Mapping, Valetta, 2012, vol. 2.

  11. Crawley, E., Cameron, B., and Selva, D., Systems Architecture: Strategy and Product Development for Complex Systems, Hoboken, NJ: Pearson Higher Education, 2015.

    Google Scholar 

  12. Darmofal, D.L., Soderholm, D.H., and Brodeur, D.R., Using concept maps and concept questions to enhance conceptual understanding, Proc. Frontiers in Education Conf., Boston, MA, Nov. 6–9, 2002.

  13. Davies, M., Concept mapping, mind mapping and argument mapping: what are the differences and do they matter?, Higher Edu., 2011, vol. 62, no. 3, pp. 279–301. https://doi.org/10.1007/s10734-010-9387-6

    Article  Google Scholar 

  14. Duarte, E., Loureiro, A.C., and Sukowsky-Tavares, C., Challenges and weaknesses in the use of Concept Maps as a learning strategy in undergraduate health programs, Knowl. Manag. E-Learn., 2017, vol. 9, no. 3, pp. 153–169.

    Google Scholar 

  15. Duncan, R.G. and Reiser, B.J., Reasoning across ontologically distinct levels: students’ understandings of molecular genetics, J. Res. Sci. Teach., 2007, vol. 44, no. 7, pp. 938–959. https://doi.org/10.1002/tea.20186

    Article  Google Scholar 

  16. Goldsmith, T.E. and Davenport, D.M., Assessing structural similarity of graphs, in Pathfinder Associative Networks: Studies in Knowledge Organization, Schvaneveldt, R.W., Ed., Norwood, NJ: Ablex, 1989, pp. 75–87.

    Google Scholar 

  17. Goldstone, R. and Wilensky, U., Promoting transfer by grounding complex systems principles. journal of the learning sciences, J. Learn. Sci., 2008, vol. 17, no. 4, pp. 465–516. .https://doi.org/10.1080/10508400802394898

    Article  Google Scholar 

  18. ECTS User’s Guide, 2009. https://doi.org/10.2766/88064

  19. Hebel, M., Light bulbs and change: systems thinking and organizational learning of new ventures, Learn. Org., 2007, vol 14, no. 6, pp. 499–509. https://doi.org/10.1108/09696470710825114

    Article  Google Scholar 

  20. Henderson, C., Yerushalmix, E., Heller, K., Heller, P., and, Kuo, V.H., Multi-Layered Concept Maps for the Analysis of Complex Interview Data, 2003.

  21. Hmelo-Silver, C.E., Holton, D.L., and Kolodner, J.L., Designing to learn about complex systems, J. Learn. Sci., 2000, vol. 9, no. 3, pp. 247–298. https://doi.org/10.1207/S15327809JLS0903_2

    Article  Google Scholar 

  22. Hu, M. and Shealy, T., Methods for measuring systems thinking: differences between student self-assessment, concept maps scores, and cortical activation during tasks about sustainability, Proc. ASEE Annu. Conf. & Exposition, Salt Lake City, UT, 2018.

  23. Kennedy, D., Hyland, A., and Ryan, N., Learning outcomes and competencies, in Introducing Bologna Objectives and Tools, 2019, B 2.3–3.

  24. Lavi, R. and Dori, Y.J., Systems Thinking of pre- and in-service science and engineering teachers, Int. J. Sci. Edu., 2018, vol. 41, no. 2, pp. 248–279. https://doi.org/10.1080/09500693.2018.1548788

    Article  Google Scholar 

  25. Markow, P.G. and Lonning, R.A., Usefulness of concept maps in college chemistry laboratories: students’ perceptions and effects on achievement, J. Res. Sci. Teach., 1998, vol. 35, no. 9, pp. 1015–1029. https://doi.org/10.1002/(SICI)1098-2736(199811)35:9<1015::AID-TEA4>3.0.CO;2-G

  26. McClure, J.R., Sonak, B., and Suen, H.K., Concept map assessment of classroom learning: reliability, validity, and logistical practicality, J. Res. Sci. Teach., 1999, vol. 36, no. 4, pp. 475–492.

    Article  Google Scholar 

  27. Meadows, D.H., Thinking in Systems, Earthscan, 2009.

    Google Scholar 

  28. Mehrjerdi, Y.Z., Quality function deployment and its profitability engagement: a Systems Thinking perspective, Int. J. Quality Reliab. Manag., 2011, vol. 28, no. 9, pp. 910–928.

    Article  Google Scholar 

  29. Miertschin, S.L. and Willis, C.L., Using Concept Maps to navigate complex learning environments, Proc. 8th Conf. on Information Technology Education, SIGITE’07, Destin, FL, 2007. https://doi.org/10.1145/1324302.1324340

  30. Mintzes, J.J., Wandersee, J.H., and Novak, J.D., Teaching Science for Understanding: a Human Constructivist View, San Diego, CA: Acad. Press, 1998.

    Google Scholar 

  31. National Research Council, NRC. National Science Education Standards, National Academy, Washington, 1996.

    Google Scholar 

  32. Novak, J. and Gowin, D., Learning How to Learn, New York: Cambridge Univ. Press, 1984.

    Book  Google Scholar 

  33. Novak, J.D. and Musonda, D., A twelve-year longitudinal study of science concept learning, Am. Edu., 1991, vol. 28, no. 1, pp. 117–153. https://doi.org/10.3102/00028312028001117

    Article  Google Scholar 

  34. Odom, A.L. and Kelly, P.V., Integrating concept mapping and the learning cycle to teach diffusion and osmosis concepts to high school biology students, Sci. Edu., 2000, vol. 85, no. 6, pp. 615–635. https://doi.org/10.1002/sce.1029

    Article  Google Scholar 

  35. Patton, M.Q., Qualitative Evaluation and Research Methods, Newbury Park, CA: Sage, 2002.

    Google Scholar 

  36. Raved, L. and Yarden, A., Developing seventh grade students’ Systems Thinking skills in the context of the human circulatory system, Front. Health, 2014, vol. 2, pp. 1–11. https://doi.org/10.3389/fpubh.2014.00260

    Article  Google Scholar 

  37. Richmond, B., Systems thinking/systems dynamics: let’s just get on with it, Proc. Int. Systems Dynamics Conf., Sterling, 1994.

  38. Richmond, B., Systems Thinking: critical Thinking Skills for the 1990s and beyond, Syst. Dyn. Rev., 1993, vol. 9, no. 2, pp. 113–133.

    Article  Google Scholar 

  39. Rye, J.A. and Rubba, P.A., Scoring Concept Maps: an expert map-based scheme weighted for relationships, School Sci. Math., 2010, vol. 102, no. 1, pp. 33–44. https://doi.org/10.1111/j.1949-8594.2002.tb18194.x

    Article  Google Scholar 

  40. Sankaran, S., Tay, B.H., and Orr, M., Managing organizational change by using soft systems thinking in action research projects, Int. J. Manag. Projects Bus., 2009, vol. 2, no. 2, pp. 179–197. https://doi.org/10.1108/175383709109449257

    Article  Google Scholar 

  41. Schwendimann, B., Concept Mapping, in Encyclopedia of Science Education, Gunstone, R., Ed., Dordrecht, Heidelberg, New York, London: Springer, 2014. https://doi.org/10.1007/978-94-007-2150-0_40

    Book  Google Scholar 

  42. Senge, P., The Fifth Discipline: the Art and Practice of the Learning Organization, Doubleday, NY, 1990.

    Google Scholar 

  43. Songer, C.J. and Mintzes, J.J., Understanding cellular respiration: an analysis of conceptual change in college biology, Sci. Teach., 1994, vol. 31, no. 6, pp. 621–637. https://doi.org/10.1002/tea.3660310605

    Article  Google Scholar 

  44. Stewart, M., Joined up thinking? Evaluating the use of concept-mapping to develop complex system learning, Assess. Eval. Higher Edu., 2012, vol. 37, no. 3, pp. 349–368. https://doi.org/10.1080/02602938.2010.534764

    Article  Google Scholar 

  45. Trendowicz, A. and Jeffery, R., Constructive Cost Model-COCOMO. Software Project Effort Estimation: Foundations and Best Practice Guidelines for Success, Springer, 2014, chapter 9.

    Book  Google Scholar 

  46. Tripto, J., Assaraf, O.B-Z., and Amit, M., Mapping what they know: concept maps as an effective tool for assessing students’ Systems Thinking, Am. J. Oper. Res., 2013, vol. 3, pp. 245–258. https://doi.org/10.4236/ajor.2013.31A022

    Article  Google Scholar 

  47. Trochim, W.M., Cabrera, D.A., Milstein, B., Gallagher, R.S., and Leischow, S.J., Practical challenges of systems thinking and modeling in public health, Am. J. Public Health, 2006, vol. 96, no. 3, pp. 538–546. https://doi.org/10.2105/AJPH.2005.066001

    Article  Google Scholar 

  48. Turns, J., Atman, C.J., and Adams, R., Concept Maps for engineering education: a cognitively motivated tool supporting varied assessment functions, IEEE Trans. Edu., 2000, no. 2, pp. 164–173. https://doi.org/10.1109/13.8480699

  49. Valerdi, R. and Rouse, W.B., When systems thinking is not a natural act, Proc. IEEE Int. Systems Conf., San Diego, CA, 2010, pp. 184–189. https://doi.org/10.1109/SYSTEMS.2010.5482446

  50. Verhoeff, R.P., Waarlo, A.J., and Boersma, K.T., Systems modeling and the development of coherent understanding of cell biology, Int. J. Sci. Edu., 2008, vol. 30, no. 4, pp. 543–568. https://doi.org/10.1080/09500690701237780

    Article  Google Scholar 

  51. Viacheslav, M., ST and collective problem solving practices, Proc. Conf. System Analysis in Economics, Moscow, 2018. https://doi.org/10.33278/SAE-2018.eng.269-272

  52. Watson, M.K., Pelkey, J., Noyes, C.R., and Rodgers, M.O., Assessing conceptual knowledge using three concept map scoring methods, J. Eng. Edu., 2015, no. 1, pp. 118–146.

  53. Valdés-Souto, F. and Naranjo-Albarrán, L., Improving the software estimation models based on functional size through validation of the assumptions behind the linear regression and the use of the confidence intervals when the reference database presents a wedge-shape form, Progr. Comput. Software, 2021, vol. 105, no. 1, pp. 118–146.

    Google Scholar 

  54. Durán, M., Juárez-Ramírez, R., Jiménez, S., and Tona, C., User story estimation based on the complexity decomposition using Bayesian networks, Progr. Comput. Software, 2020, vol. 46, pp. 569–583. https://doi.org/10.1134/S0361768820080095

    Article  Google Scholar 

  55. White, R. and Gunstone, R., Gunstone, Probing Understanding, London: Falmer, 1992.

    Google Scholar 

  56. York, S., Lavi, R., Dori, Y.J., and Orgill, M., Applications of systems thinking in STEM education, J. Chem. Edu., 2019, vol. 96, no. 12, pp. 2742–2751. https://doi.org/10.1021/acs.jchemed.9b00261

Download references

Author information

Authors and Affiliations

  1. UPAEP, Department of Engineering, Calle 21 Sur 1103, Barrio de Santiago, 72410, Puebla, Pue., México

    J.R. Aguilar-Cisneros

  2. University of Arizona, Department of Systems and Industrial Engineering, 85721, Tucson, AZ, USA

    R. Valerdi

  3. University of Twente, Department of Design, Production and Management, Enschede, the Netherlands

    B. P. Sullivan

Authors
  1. J.R. Aguilar-Cisneros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Valerdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. P. Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to J.R. Aguilar-Cisneros, R. Valerdi or B. P. Sullivan.

Ethics declarations

The authors declare that they have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aguilar-Cisneros, J., Valerdi, R. & Sullivan, B.P. Students’ Systems Thinking Competency Level Detection through Software Cost Estimation Concept Modeling. Program Comput Soft 48, 499–512 (2022). https://doi.org/10.1134/S0361768822080060

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768822080060

Search

Navigation