Skip to main content
SpringerLink
Log in

Grading Programs Based on Hybrid Analysis

  • Conference paper
  • First Online:
Web Information Systems and Applications (WISA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11817))

Included in the following conference series:

Abstract

Grading programming assignments often take a lot of time and effort, so there is a growing need for automatic scoring systems. The existing program scoring system gives scores mainly by executing test cases. It cannot identify homework plagiarism or give grades according to steps. Therefore, this paper proposes an automatic scoring technique for Python programming assignments based on hybrid program analysis. Our scoring tool Paprika combines dynamic analysis and static analysis methods. Based on the Python abstract tree, it uses static analysis to realize the detection of clones and code style assessment and uses dynamic analysis to score with running test cases. It has a good tolerance for various formats of output of the code. The experiment shows that on the one hand, Paprika can reduce the burden of teachers and teaching assistants; on the other hand, it can accurately and objectively give grades and detailed feedback, which is helpful for students to improve learning efficiency and programming level.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://coursemarker.software.informer.com.

  2. 2.

    https://web-cat.cs.vt.edu/Web-CAT/WebObjects/Web-CAT.woa.

  3. 3.

    https://scholarworks.sjsu.edu/etd_projects/51/.

  4. 4.

    http://checkstyle.sourceforge.net.

  5. 5.

    https://github.com/skyhover/Deckard.

  6. 6.

    https://google.github.io/styleguide/pyguide.html.

  7. 7.

    https://github.com/YiBinRanMian/assignments.

  8. 8.

    https://leetcode.com/problems/two-sum/.

  9. 9.

    https://leetcode.com/problems/move-zeroes/.

  10. 10.

    https://leetcode.com/problems/maximum-subarray/.

  11. 11.

    http://findbugs.sourceforge.net.

  12. 12.

    https://github.com/juqian/Slithice.

  13. 13.

    http://pmd.github.io/.

  14. 14.

    https://www.pylint.org/.

References

  1. Hollingsworth, J.K.: Automatic graders for programming classes. Commun. ACM 3(10), 528–529 (1960)

    Article  Google Scholar 

  2. Spacco, J., Hovemeyer, D., Pugh, W., Emad, F., Hollingsworth, J.K., Padua-Perez, N.: Experiences with marmoset: designing and using an advanced submission and testing system for programming courses. ACM SIGCSE Bull. 38, 13–17 (2006)

    Article  Google Scholar 

  3. Edwards, S.H., Perezquinones, M.A.: Web-CAT: automatically grading programming assignments. In: Technical Symposium on Computer Science Education, vol. 40, no. 3, p. 328 (2008)

    Article  Google Scholar 

  4. Rodriguez-del-Pino, J.: A Virtual Programming Lab for Moodle with automatic assessment and anti-plagiarism features (2012)

    Google Scholar 

  5. Patil, A.: Automatic Grading of Programming Assignments. Master’s Projects. 51 (2010). https://scholarworks.sjsu.edu/etd_projects/51

  6. Higgins, C., Hegazy, T., Symeonidis, P., Tsintsifas, A.: The CourseMarker CBA system: improvements over Ceilidh. Educ. Inf. Technol. 8(3), 287–304 (2003)

    Article  Google Scholar 

  7. Wang, T., Su, X., Ma, P., et al.: Ability-training-oriented automated assessment in introductory programming course. Comput. Educ. 56(1), 220–226 (2011)

    Article  Google Scholar 

  8. Queirós, R.A.P., Leal, J.P.: PETCHA: a programming exercises teaching assistant. In: Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education, pp. 192–197 (2012)

    Google Scholar 

  9. Jiang, L., Misherghi, G., Su, Z., Glondu, S.: DECKARD: scalable and accurate tree-based detection of code clones. In: ICSE, pp. 96–105 (2007)

    Google Scholar 

  10. Hopcroft, J.E., Ullman, J.D.: Chapter 4: Context-free grammars, pp. 77–106; Chapter 6: Properties of context-free languages. In: Introduction to Automata Theory, Languages, and Computation, pp. 125–137. Addison-Wesley (1979)

    Google Scholar 

  11. Linz, P.: An Introduction to Formal Languages and Automata, p. 291. Jones & Bartlett Publishers, Sudbury (2011). ISBN 978-1-4496-1552-9

    Google Scholar 

  12. Meduna, A.: Automata and Languages: Theory and Applications, p. 730. Springer, London (2000). ISBN 978-1-85233-074-3

    Book  Google Scholar 

  13. Davis, M., Sigal, R., Weyuker, E.J.: Computability, Complexity, and Languages: Fundamentals of Theoretical Computer Science, 2nd ed., p. 189. Morgan Kaufmann (1994). ISBN 978-0-08-050246-5

    Chapter  Google Scholar 

  14. Martin, J.C.: Introduction to Languages and the Theory of Computation, 4th edn, p. 277. McGraw-Hill, New York (2010). ISBN 9780073191461

    Google Scholar 

  15. Kamiya, T., Kusumoto, S., Inoue, K.: CCFinder: a multilinguistic token-based code clone detection system for large scale source code. TSE 28(7), 654–670 (2002)

    Google Scholar 

  16. Zhang, J., Xu, L., Li, Y.: Classifying Python code comments based on supervised learning. In: Meng, X., Li, R., Wang, K., Niu, B., Wang, X., Zhao, G. (eds.) WISA 2018. LNCS, vol. 11242, pp. 39–47. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02934-0_4

    Chapter  Google Scholar 

  17. Ihantola, P., Ahoniemi, T., Karavirta, V., Seppälä, O.: Review of recent systems for automatic assessment of programming assignments. In: Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pp. 86–93. ACM (2010)

    Google Scholar 

  18. Li, L., Feng, H., Zhuang, W., Meng, N., Ryder, B.: CCLearner: a deep learning-based clone detection approach (2017). https://doi.org/10.1109/icsme.2017.46

  19. Li, Z., Lu, S., Myagmar, S., Zhou, Y.: CP-Miner: a tool for finding copy-paste and related bugs in operating system code. In: OSDI, pp. 289–302 (2004)

    Google Scholar 

Download references

Acknowledgment

The work is supported by National Key R&D Program of China (2018YFB1003901), National Natural Science Foundation of China (61832009, 61728203).

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Nanjing University, Nanjing, 210023, China

    Zhikai Wang & Lei Xu

  2. Department of Computer Science and Technology, Sichuan University, Chengdu, 610211, China

    Zhikai Wang

Authors
  1. Zhikai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Xu .

Editor information

Editors and Affiliations

  1. Southeast University, Nanjing, China

    Weiwei Ni

  2. Tianjin University, Tianjin, China

    Xin Wang

  3. Wuhan University, Wuhan, China

    Wei Song

  4. Tianjin University of Technology, Tianjin, China

    Yukun Li

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Z., Xu, L. (2019). Grading Programs Based on Hybrid Analysis. In: Ni, W., Wang, X., Song, W., Li, Y. (eds) Web Information Systems and Applications. WISA 2019. Lecture Notes in Computer Science(), vol 11817. Springer, Cham. https://doi.org/10.1007/978-3-030-30952-7_63

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30952-7_63

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30951-0

  • Online ISBN: 978-3-030-30952-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation