Skip to main content
Springer Nature Link
Log in

Can we do better? a classification of algorithm run-time-complexity improvement using the SOLO taxonomy

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Improving code while preserving its functionality is a common task in the hi-tech industry. Yet students have difficulties improving an algorithm’s run-time complexity by an order of magnitude. The paper focuses on assessing students’ abilities in this area. We designed a Structure of the Observed Learning Outcome (SOLO) taxonomy, using software quality factors, to assess students’ cognitive ability while improving complexity. The research was conducted with college students studying for their bachelor’s degree in engineering. We established a classification based on their solutions for a given task. Later, we used the same task to validate our classification with another group of engineering students. We then compared the previous average grades of the second group of students with their SOLO levels. The results show that the higher the students’ previous average grades, the greater the probability that their solutions would be classified at higher taxonomic levels. These results indicate that our SOLO classification is indeed accurate. The paper presents our novel SOLO taxonomic levels for tasks improving run-time complexity and offers several suggestions to assist students and enhance the teaching process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Code availability

Code is available in the paper.

References

  • Adams, K. M. (2015). Non-functional requirements in systems analysis and design. Topics in Safety, Risk, Reliability and Quality, Vol. 28. Springer.

  • Ajami, S., Woodbridge, Y., & Feitelson, D. G. (2017). Syntax, predicates, idioms: What really affects code complexity? In Proceedings of the 25th International Conference on Program Comprehension (pp. 66–76). Piscataway, NJ: IEEE Press.

  • Beck, K., & Gamma, E. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley Professional.

    Google Scholar 

  • Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed learning outcome). New York: Academic Press.

    Google Scholar 

  • Castro, F. E. V., & Fisler, K. (2017). Designing a multi-faceted SOLO taxonomy to track program design skills through an entire course. In Proceedings of the 17th Koli Calling International Conference on Computing Education Research (pp. 10–19). New York: ACM.

  • Chan, C. C., Tsui, M. S., Chan, M. Y., & Hong, J. H. (2002). Applying the structure of the observed learning outcomes (SOLO) taxonomy on student’s learning outcomes: An empirical study. Assessment & Evaluation in Higher Education, 27(6), 511–527. https://doi.org/10.1080/0260293022000020282

    Article  Google Scholar 

  • Clear, T., Whalley, J. L., Lister, R., Carbone, A., Hu, M., Sheard, J., & Thompson, E. (2008). Reliably classifying novice programmer exam responses using the SOLO taxonomy. 21st National Advisory Committee on Computing Qualifications. (pp. 23–30). NACCQ: Hamilton, NZ.

    Google Scholar 

  • Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2009). Introduction to algorithms. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  • Duran, R., Sorva, J., & Leite, S. (2018). Towards an analysis of program complexity from a cognitive perspective. In Proceedings of the 2018 ACM Conference on International Computing Education Research, ICER ’18 (pp. 21–30). New York: ACM.

  • Enström, E., & Kann, V. (2017). Iteratively intervening with the “most difficult” topics of an algorithms and complexity course. ACM Transactions on Computing Education (TOCE), 17(1), 1–38. https://doi.org/10.1145/3018109

    Article  Google Scholar 

  • Fokides, E. (2017). Students learning to program by developing games: Results of a year-long project in primary school settings. Journal of Information Technology Education: Research, 16, 475–505. ‏https://doi.org/10.28945/3893

  • Fowler, M. (1999). Refactoring: Improving the design of existing code. Reading, MA: Addison-Wesley.

    MATH  Google Scholar 

  • Freeman, S., & Pryce, N. (2009). Growing object-oriented software, guided by tests. Upper Saddle River, NJ: Pearson Education.

    Google Scholar 

  • Gal-Ezer, J., Vilner, T., & Zur, E. (2004). Teaching algorithm efficiency at CS1 level: A different approach. Computer Science Education, 14(3), 235–248. https://doi.org/10.1080/0899340042000302736

    Article  Google Scholar 

  • Ginat, D., & Menashe, E. (2015). SOLO taxonomy for assessing novices’ algorithmic design. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 452–457). New York: ACM.

  • Hattie, J., & Purdie, N. (1998). The SOLO model: Addressing fundamental measurement issues. In B. Dart & G. Boulton-Lewis (Eds.), Teaching and Learning in Higher Education. (pp. 145–176). Melbourne, Australia: Australian Council for Educational Research.

    Google Scholar 

  • ISO (International Organization for Standardization). (2010). Systems and software engineering—vocabulary. ISO/IEC/IEEE 24765: 2010. Geneva, Switzerland: ISO.

  • Izu, C., Weerasinghe, A., & Pope, C. (2016). A study of code design skills in novice programmers using the SOLO taxonomy. In Proceedings of the 2016 ACM Conference on International Computing Education Research (pp. 251–259). New York: ACM.

  • Keane, T., Keane, W. F., & Blicblau, A. S. (2016). Beyond traditional literacy: Learning and transformative practices using ICT. Education and Information Technologies, 21(4), 769–781.

    Article  Google Scholar 

  • Leou, M., Abder, P., Riordan, M., & Zoller, U. (2006). Using ‘HOCS-centered learning’ as a pathway to promote science teachers’ metacognitive development. Research in Science Education, 36(1–2), 69–84. https://doi.org/10.1007/s11165-005-3916-9

    Article  Google Scholar 

  • Lister, R., Clear, T., Simon, B., Bouvier, D. J., Carter, P., Eckerdal, A., & Thompson, E. (2010). Naturally occurring data as research instrument: Analyzing examination responses to study the novice programmer. ACM SIGCSE Bulletin, 41, 156–173. https://doi.org/10.1145/1709424.1709460

    Article  Google Scholar 

  • Lister, R., Simon, B., Thompson, E., Whalley, J. L., & Prasad, C. (2006). Not seeing the forest for the trees: Novice programmers and the SOLO taxonomy. ACM SIGCSE Bulletin, 38, 118–122. https://doi.org/10.1145/1140124.1140157

    Article  Google Scholar 

  • Mayer, R. E. (1981). The psychology of how novices learn computer programming. ACM Computing Surveys (CSUR), 13(1), 121–141. https://doi.org/10.1145/356835.356841

    Article  Google Scholar 

  • Papadimitriou, C. H. (2003). Computational complexity. Encyclopedia of Computer Science. (pp. 260–265). Chichester, UK: John Wiley & Sons.

    Google Scholar 

  • Qahmash, A., Joy, M., Boddison, A., & Special Educational Needs. (2017). Investigating high-achieving students’ code-writing abilities through the SOLO taxonomy. In PPIG (pp. 17–27). Psychology of Programming Interest Group.

  • Sheard, J. I., Carbone, A., Lister, R., Simon, B., Thompson, E., & Whalley, J. L. (2008). Going SOLO to assess novice programmers. ACM SIGCSE Bulletin, 40, 209–213. https://doi.org/10.1145/1384271.1384328

    Article  Google Scholar 

  • Shuhidan, S., Hamilton, M., & D’Souza, D. (2009). A taxonomic study of novice programming summative assessment. In Proceedings of the Eleventh Australasian Conference on Computing Education (Vol. 95, pp. 147–156). Darlinghurst, Australia: Australian Computer Society.

  • Sirkiä, T., & Sorva, J. (2012). Exploring programming misconceptions: An analysis of student mistakes in visual program simulation exercises. In Proceedings of the 12th Koli Calling International Conference on Computing Education Research (pp. 19–28).

  • Statter, D., & Armoni, M. (2020). Teaching abstraction in computer science to 7th grade students. ACM Transactions on Computing Education (TOCE), 20(1), 1–37. https://doi.org/10.1145/3372143

    Article  Google Scholar 

  • Stavrinoudis, D., & Xenos, M. (2008). Comparing internal and external software quality measurements. In Proceedings of the 2008 Conference on Knowledge-Based Software Engineering: Proceedings of the Eighth Joint Conference on Knowledge-Based Software Engineering (pp. 115–124). Amsterdam, Netherlands: IOS Press.

  • Stoecklin, S., Smith, S., & Serino, C. (2007). Teaching students to build well-formed object-oriented methods through refactoring. ACM SIGCSE Bulletin, 39, 145–149. ACM. https://doi.org/10.1145/1227310.1227364

  • Stuurman, S., Passier, H., & Barendsen, E. (2016). Analyzing students’ software redesign strategies. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research (pp. 110–119). New York: ACM.

  • Teague, D., & Lister, R. (2014). Programming: Reading, writing and reversing. In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (pp. 285–290).

  • Uzumcu, O., Bay, E. (2020) The effect of computational thinking skill program design developed according to interest driven creator theory on prospective teachers Education and Information Technologies 1–19. https://doi.org/10.1007/s10639-020-10268-3

  • Whalley, J. (2006). CSEd research instrument design: The localisation problem. In S. Mann & N. Bridgeman (Eds.), Proceedings of The Nineteenth Annual NACCQ Conference (pp. 307–312). Hamilton, NZ: NACCQ.

  • Whalley, J. L., Lister, R., Thompson, E., Clear, T., Robbins, P., Kumar, P. K., & Prasad, C. (2006, January). An Australasian study of reading and comprehension skills in novice programmers, using the bloom and SOLO taxonomies. In Proceedings of the 8th Australasian Conference on Computing Education-Volume 52 (pp. 243–252). Australian Computer Society, Inc.

  • Whalley, J., Clear, T., Robbins, P., & Thompson, E. (2011). Salient elements in novice solutions to code writing problems. In Proceedings of the Thirteenth Australasian Computing Education Conference (Vol. 114, pp. 37–46). Darlinghurst, Australia: Australian Computer Society.

  • Whalley, J., & Kasto, N. (2014). How difficult are novice code writing tasks? A software metrics approach. In Proceedings of the Sixteenth Australasian Computing Education Conference (Vol. 148, pp. 105–112). Darlinghurst, Australia: Australian Computer Society.

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, Shamoon College of Engineering, Ashdod, Israel

    Lior Aronshtam & Tammar Shrot

  2. Department of Industrial Engineering and Management, Shamoon College of Engineering, Ashdod, Israel

    Ronit Shmallo

Authors
  1. Lior Aronshtam
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Tammar Shrot
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ronit Shmallo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ronit Shmallo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aronshtam, L., Shrot, T. & Shmallo, R. Can we do better? a classification of algorithm run-time-complexity improvement using the SOLO taxonomy. Educ Inf Technol 26, 5851–5872 (2021). https://doi.org/10.1007/s10639-021-10532-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-021-10532-0

Keywords