Skip to main content

Advertisement

SpringerLink
Log in

Towards a fictional collective programming scenario: an approach based on the EIF loop

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

In this paper, we base our research on a fictional collective programming scenario: A group of physically-distributed programmers try to collaboratively solve a programming problem in a web-based development environment, through a continually executing loop, consisting of three concurrent activities: exploration, integration, and feedback. In exploration, a programmer is freely to submit a sequence of gradually improved solutions until achieving a correct one. All programmers’ latest submissions are integrated into a collective artifact through integration. And through feedback, any programmer who hasn’t achieved a correct solution is continuously pushed with personalized feedback information from the collective artifact, to help the programmer improve her/his submission. In order to facilitate the realization of this fictional scenario, we narrow the target problems to those introductory programming problems, design a genetic algorithm to integrate a set of syntax-correct programs into a collective program dependence graph (CPDG), and propose an automatic feedback generation method based on the CPDG and a programmer’s latest submission. The key idea is to generate feedback from mutual inspiration: Any programmer’s submission (even not correct) may possess information that could provide inspiration for others. We evaluate the proposed approach through a set of simulated experiments, as well as a set of real experiments. The results show that our approach has a precision of 90% and a recall of 80% in randomly generated data sets on average, and a precision of 69% and a recall of 77% in real student submissions on average.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. Available at https://pypi.org/project/pycparser/

  2. Available at https://www-m9.ma.tum.de/graph-algorithms/matchings-hungarian-method/index_en.html

  3. Available at https://en.wikipedia.org/wiki/F1_score

  4. Programming Grid is a computer-aided education system for programming courses based on PKU Judge Online, an online judge system for ACM/ICPC in Peking University (http://acm.pku.edu.cn/JudgeOnline)

  5. Appendix A shows more details of the 3 tasks.

References

  • Adam A, Laurent J (1980) LAURA, a system to debug student programs. Artif Intell 15(1):75–122

    Article  Google Scholar 

  • Boyer KE, Dwight AA, Fondren RT, Vouk MA, Lester JC (2008) A development environment for distributed synchronous collaborative programming. In: Conference on Innovation & Technology in Computer Science Education, pp 158–162

  • Debroy V, Wong WE (2010) Using mutation to automatically suggest fixes for faulty programs. In: the 3rd IEEE International Conference on Software Testing, Verification, and Validation (ICST), pp 65–74

  • Dan S (2013) Evolutionary optimization algorithms. Wiley

  • Forrest S, Nguyen T, Weimer W, Goues CL (2009) A genetic programming approach to automated software repair. In: 18th International Conference on Genetic Algorithm and the 14th Annual Genetic Programming Conference evolutionary (GECCO), pp 947–954

  • Genovese V, Odetti L, Magni R, Dario P (1992) Self organizing behavior and swarm intelligence in a cellular robotic system. In: IEEE International Symposium on Intelligent Control, pp 243–248

  • Girgis M R (1992) An experimental evaluation of a symbolic execution system. Softw Eng J 7(4):285–290

    Article  Google Scholar 

  • Gulwani S, Radicek I, Zuleger F (2018) Automated clustering and program repair for introductory programming assignments. Programming Language Design and Implementation 53(4):465–480

    Google Scholar 

  • Gopinath D, Malik MZ, Khurshid S (2011) Specification-based program repair using SAT. In: International Conference on Tools and Algorithms for Construction and Analysis of Systems, pp 173–188

  • Gartner VC, Pinto SC (2012) IdDE: A collaborative environment for distributed programming in pairs. In: Conferencia Latinoamericana EN Informatica, pp 1–10

  • Horwitz S, Reps T, Binkley D (2004) Inter-procedural slicing using dependence graphs. Acm Sigplan Notices 4(39):229–243

    Article  Google Scholar 

  • Harrold MJ, Malloy BA, Rothermel G (1993) Efficient construction of program dependence graphs. International Symposium on Software Testing and Analysis 18(3):160–170

    Google Scholar 

  • Kurniawan A, Kurniawan A, Soesanto C, Wijaya J E C (2015) Coder: real-time code editor application for collaborative programming. Procedia Computer Science 59:510–519

    Article  Google Scholar 

  • Könighofer R, Bloem R (2011) Automated error localization and correction for imperative programs. In: Formal Methods in Computer-Aided Design, pp 91–100

  • Kidwell P (1996) The mythical man-month: Essays on software engineering. IEEE Annals of the History of Computing 18(4):71

    Article  Google Scholar 

  • Lin Y, Wu C, Chiu C (2018) The use of wiki in teaching programming: effects upon achievement, attitudes, and collaborative programming behaviors. International Journal of Distance Education Technologies 16 (3):18–45

    Article  Google Scholar 

  • Maclaren MD (1969) The art of computer programming—volume 1: fundamental algorithms. Siam Review 11(1):89–91

    Article  Google Scholar 

  • Mcdowell C, Werner L, Bullock HE, Fernald J (2002) The effects of pair-programming on performance in an introductory programming course. Technical Symposium on Computer Science Education 34(1):38–42

    Google Scholar 

  • Qi YH, Mao XG, Lei Y, Dai ZY, Wang CS (2013) Does genetic programming work well on automated program repair. In: The 2013 International Conference on Computational and Information Sciences (ICCIS), pp 1875-1878

  • Rolim R, Soares G, Dantoni L, Polozov O, Gulwani S, Gheyi R, Suzuki R, Hartmann B (2017) Learning syntactic program transformations from examples. In: International Conference on Software Engineering, pp 404–415

  • Susi T, Ziemke T (2001) Social cognition, artefacts, and stigmergy: a comparative analysis of theoretical frameworks for the understanding of artefact-mediated collaborative activity. Cognitive Systems Research 4(2):273–290

    Article  Google Scholar 

  • Shannon CE (1950) The mathematical theory of communication. Bell Labs Technical Journal 9(3):31–32

    Google Scholar 

  • Saraph V, Milenkovic T (2014) MAGNA: Maximizing accuracy in global network alignment. Bioinformatics 30(20):2931–2940

    Article  Google Scholar 

  • Singh R, Gulwani S, Solarlezama A (2013) Automated feedback generation for introductory programming assignments. Programming Language Design and Implementation 48(6):15–26

    Google Scholar 

  • Shi JL, Shah A (2018) Pyrus: A collaborative programming game to support problem-solving. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

  • Wang K, Singh R, Su Z (2018) Search, align, and repair: Data-driven feedback generation for introductory programming exercises. Programming Language Design and Implementation 53(4):481–495

    Google Scholar 

  • Watson C, Li FWB, Godwin JL (2012) BlueFix: Using crowd-sourced feedback to support programming students in error diagnosis and repair. In: 11th International Conference on Advances in Web-Based Learning, pp 228–239

  • Xu S W, Chee Y S (2003) Transformation-based diagnosis of student programs for programming tutoring systems. IEEE Transactions on Software Engineering 29(4):360–384

    Article  Google Scholar 

  • Yi J, Ahmed UZ, Karkare A, Tan SH, Roychoudhury A (2017) A feasibility study of using automated program repair for introductory programming assignments. In: 11th Joint Meeting of the European Software Engineering Conference and the ACM Sigsoft Symposium on the Foundations of Software Engineering, pp 740–751

  • Yashiro T, Harada Y, Mukaiyama K (2017) Plugramming: A tangible programming tool for children’s collaborative learning. In: 19th International Conference on Human-Computer Interaction, pp 398–409

  • Yang G, Jeong Y, Min K, Lee J., Lee B (2018) Applying genetic programming with similarBug fix information to automatic fault repair. Symmetry 10:1–92

    Google Scholar 

  • Zhang W, Mei H (2020) A constructive model for collective intelligence. National Science Review 7(8):1273–1277

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for precious feedback that allowed us to significantly improve this paper. This research is supported by the National Natural Science Foundation of China under Grant Nos. 61690200, 61620106007 and 61751210.

Author information

Authors and Affiliations

  1. Key Laboratory of High Confidence Software Technology (MoE), Peking University, Beijing, China

    Chunhui Wang, Wei Zhang, Haiyan Zhao & Zhi Jin

  2. Institute of Software, School of EECS, Peking University, Beijing, China

    Chunhui Wang, Wei Zhang, Haiyan Zhao & Zhi Jin

  3. College of Computer Science and Technology, Inner Mongolia Normal University, Hohhot, China

    Chunhui Wang

Authors
  1. Chunhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haiyan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Zhang or Zhi Jin.

Additional information

Communicated by: Xin Xia, Hui Liu, David Lo

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Software Applications (NASAC)

Appendix A Real Data Description

Appendix A Real Data Description

figure e
figure f
figure g
figure h

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Zhang, W., Zhao, H. et al. Towards a fictional collective programming scenario: an approach based on the EIF loop. Empir Software Eng 25, 3671–3710 (2020). https://doi.org/10.1007/s10664-020-09850-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-020-09850-7

Keywords

Search

Navigation