An extended educational system for programming and its evaluation
Introduction
In programming exercises students make various source codes that may be incorrect. Even correct codes for one specification may be various. It is difficult for novice students to judge whether their codes are correct and proper, when their codes differ from answers in textbooks. They need judgments and advice inside and outside of classrooms.
Many kinds of interactive programming tutoring systems for novice students have been developed. Some systems teach students the way to correct codes interactively. These aid mainly the process of programming. Anderson and Skwarecki [1] developed a Lisp intelligent tutor that generates effective tutorial dialogues and serves as syntax editor. It monitors students’ coding from the first and guides students when they go astray from desired codes. Bridge [2] is a Pascal intelligent tutorial environment that teaches programming in three phases—a natural language ‘program’, a ‘plan program’ and a Pascal program. Programs are checked at each phase. Memo-ii [3] is an environment that guides students in building computational model of a problem. In the environment the corresponding code is automatically generated in a given programming language. These systems guide students for only prepared codes and do not allow other variations.
Some systems evaluate students’ codes. PAT [4] is an on-line Pascal tutoring system that complies, links and executes students’ codes and advises the students. It does not analyze codes directly, so it cannot judge an incorrect code that outputs correctly for given input data. Whereas the following three systems analyze codes directly, Laura [5] is a debugging system for Fortran codes. It compares an input code with a given model code. To reduce the search space, it partially standardizes codes and tries to match the two codes using control-flow-oriented graphs. Proust [6], [7] is an on-line analysis and understanding system for Pascal codes written by novice programmers. It maps parts of source codes to given requirements. The mapping is directly affected by variations of codes. C-Tutor [8], [9] is an intelligent tutoring system for novice C programmers. Using flame-structured knowledge, it extracts goals and plans from a model code made by a lecturer and generates problem description. It standardizes syntactically various codes made by a student regarding constructs of if-statements, assignments in condition parentheses et al. It analyzes both results and gives intention-based diagnosis. These three systems standardize codes about only a few kinds of variations. Particularly they do not standardize variations related to data flow, because they represent data flow indirectly.
In programming exercises students may make countless variations even for correct codes. Programming is something creative, so accepting such variations is important for education, but the aforementioned systems accept only a few variations. Moreover systems may be extended to accept codes of new specifications in the future, but such extension of the aforementioned system is difficult, because they use huge data for each specification program.
In recent years many educational systems in various areas have introduced World Wide Web (WWW) as an inexpensive powerful tool. Students can easily access such systems using Web browsers inside and outside of classrooms. Karsten and Kaparthi developed a WWW system that delivers visual explanation programming resources to students [10]. They used WWW as ‘off-the-shelf’ software to minimize development time and cost. Reed and Afjeh have developed interactive gas turbine simulator for engineering education on WWW with Java [11].
We have developed an educational system of C and Pascal programming for novice students. It judges their codes and advises on them. On behalf of education, we regard that acceptance of various codes and easiness of access are the most important. For acceptance of various codes, the system standardizes partially students’ codes and model codes by representation method and rewriting, and it compares both codes using a variable-name-free method. These reduce the number of model codes that should be stored in the program database. Moreover model codes are automatically translated into the format of the program database, so adding codes of new variations and new specifications to the database is easy. For easiness of access, the system is connected to the Internet, so students can access it anywhere and anytime using a Web browser.
This article presents extension to more practical usage for our previous system [12], [13], [14] and also the evaluation of new system. The previous system could judge only Pascal codes handling simple variables. The new system was extended to judge C codes handling array data and arithmetic function using enriched database. Section 2states the source codes that can be judged by the system. The structure of system and the process of judgement are described in Section 3. The evaluation of the system is summarized in Section 4.
Section snippets
Source code
The new system was extended to judge basic C codes in addition to Pascal codes. As for C control structures, it can handle if, if–else, ?–:, for, while, do–while, continue, break and goto. It handles arithmetic functions call such as ‘ sin’. It cannot handle user-defined functions and pointers. It handles corresponding Pascal syntax also. As for data types, it was extended to handle array variables in addition to simple variables. It cannot handle user-defined data types. The codes are
System
Fig. 3 shows the structure of the system. Students access the system with a Web browser. The system is connected to the Internet and gets a students’ code through CGI (Common Gateway Interface) of the Web server. The system is made of four subsystems: translator, variation remover, database and comparator. The translator converts the code into a data-flow-based representation and standardizes it as three kinds of variations. The variation remover standardizes it by rewriting as 15 kinds of
Evaluation
The main forte of this system is acceptance of various codes, so we have evaluated it by its ability to judge various codes and amount of codes that the database should store.
For the evaluation we collected C codes made by students who had not made the codes in the database and they had not received any lecture about these codes. First we classified the collected codes into three categories—correct ones, ones with simple bugs and ones with serious bugs. The codes with simple bugs are codes of
Conclusion
We have developed the self-teaching system for programming. The system is for novice students inside and outside of classroom. We regard that acceptance of various codes and ease of access are the most important for the educational system. For acceptance of various codes, the system standardizes partially students’ codes and model codes; firstly three kinds of variations are standardized by the data-flow-based representation; secondly 15 kinds of variations are removed by rewriting; finally
References (14)
- et al.
Software tools for the learning of programming: a proposal
Computers and Education
(1994) - et al.
A system to debug student program
Artificial Intelligence
(1980) An intelligent tutoring system for introductory C language course
Computers and Education
(1997)- et al.
Using dynamic explanations to enhance novice programmer instruction via the WWW
Computers and Education
(1998) - et al.
Developing interactive educational engineering software for the world wide web with Java
Computers and Education
(1998) - et al.
A method to remove variations in source codes
Information and Software Technology
(1996) - et al.
The automated tutoring of introductory computer programming
Communications of the ACM
(1986)
Cited by (6)
Experiments with a socratic intelligent tutoring system for source code understanding
2020, Proceedings of the 33rd International Florida Artificial Intelligence Research Society Conference, FLAIRS 2020PAAA: An implementation of programming assignments automatic assessing system
2019, ACM International Conference Proceeding SeriesAn analysis of novice compilation behavior using verificator
2011, Proceedings of the International Conference on Information Technology Interfaces, ITIFunction inlining algorithm for program analysis
2009, Proceedings - 2009 International Conference on Computational Intelligence and Software Engineering, CiSE 2009Program normalization for removing code variations
2008, Proceedings - International Conference on Computer Science and Software Engineering, CSSE 2008Normalization for removing code variations in modular programs
2008, Proceedings - International Symposium on Computer Science and Computational Technology, ISCSCT 2008