Skip to main content
SpringerLink
Log in

The automatic assessment of formal specification coursework

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

THE PURPOSE OF THIS PAPER is to describe a system that is able to grade the quality of a formal specification written in the Z language. It is intended to assess the quality of specifications written by students as coursework for a course in formal specification. The quality is measured by considering two factors: correctness and maintainability.

There are two stages in measuring the correctness of a specification. First, the syntax and semantics of the specification must be checked. Second, the level to which the specification satisfies the customer requirements must be determined. In this paper, we animate the specification against various test states of the system. Marks for animation are produced according to the extent to which the specification fulfills the customer requirements.

Specification maintainability is divided into two categories. First the typographic arrangement metric derived from the specification will be compared against a range of absolute values. Second, the complexity metrics derived from the specification will be compared against those for a model specification.

The performance of the system (on introductory exercises) has been compared with a human marker. The result shows that (at this introductory level) the system and a human marker broadly agreed on the marks given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Austin, S., & Parkin G.I. (1993).Formal methods: A survey report. The National Physical Lab. Middlesex.

  • Balzer, R., & Goldman, N. (1979). Principles of good software specification and their implications for specification languages.Proceedings of IEEE Conference on Specifications of Reliable Software, 58–67.

  • Barden, R., Stepney, S., & Cooper, D. (1994).Z in practice. Hertfordshire, UK: Prentice Hall.

    Google Scholar 

  • Basili, V.R. (1980). Tutorial on models and metrics for software management and engineeering.IEEE Catalog No. EH0-167-7. IEEE Computer Society. Los Alamitos, CA.

    Google Scholar 

  • Basili, V.R., Selby, R.W. Jr., & Philips, T. (1983). Metric analysis and validation across Fortran projects.IEEE Transaction Software Engineerng SE-9, 652–663.

    Article  Google Scholar 

  • Burke, E., & Foxley, E. (1996).Logic and its applications. Hertfordshire, UK: Prentice Hall Europe.

    Google Scholar 

  • Coleman, M., & Pratt, S. (1986).Software engineering for students. Kent, UK: Chartwell-Bratt Ltd.

    Google Scholar 

  • Curtis, W. (1980). Measurement and experimentation in software engineering.Proceedings of the IEEE, 68(9), 1114–1157.

    Article  Google Scholar 

  • Dean, C.N., & Hinchey M.G. (1995). Introducing formal methods through role play.ACM SIGCSE Bulletin, 27(1), 302–306.

    Article  Google Scholar 

  • Dijkstra, E.W. (1996).Forward In CN Dean & M.G. Hinshey (Eds.).Teaching and Learning Formal Methods. San Diego, CA: Academic Press International Series in Formal Methods.

    Google Scholar 

  • Fields, B., & Elovang-Goransson, M. (1992). A VDM case study in mural.IEEE Transactions Software Engineering, 18(4), 279–295.

    Article  Google Scholar 

  • Finney, K. (1996). Mathematical notation in formal specification: Too difficult for the massess?.IEEE Transactions on Software Engineering, 22(2), 158–159.

    Article  Google Scholar 

  • Foxley, E., & Zin, A.M. (1990).Zpp — A Troff Preprocessor for Typesetting Z Specifications. Nottingham, England: Nottingham University Computer Science.

    Google Scholar 

  • Garlan, D. (1995). Making formal methods education effective for professional software engineers.Information and Software Technology, 37(5–6), 261–268.

    Article  Google Scholar 

  • Goguen, J.A. (1984). Parameterized Programming.IEEE Transactions Software Engineering, 10(5), 528–543.

    Article  Google Scholar 

  • Hall, P.A.V. (1988). Towards testing with respect to formal specifications.Proceedings of Second IEE/BCS Conference: Software Engineering 88 (pp. 159–163). London.

    Google Scholar 

  • Halstead, M. (1977).Elements of software science. Amsterdam, The Netherlands: Elsevier Scientific Publishing Co.

    Google Scholar 

  • Helke, S., Neustupny T., & Santen T. (1997). Automating test case generation from Z specifications with Isabelle.Proceedings of the 10th International Conference of ZUM ’97. (pp 52–71). Reading, UK

  • Henry, S., & Kafura, D. (1984). The evaluation of software system’s structure using quatitative software metrics.Software-Practice and Experience, 14(6), 561–573.

    Article  Google Scholar 

  • Jalote, P. (1989). Testing the completeness of specifications.IEEE Transactions on Software Engineering, 15(5), 526–531.

    Article  Google Scholar 

  • Kearney, J.K., Sedlemeyer, R.L., Thompson, W.B., & Gray, M.A. (1986). Software complexity measurement.Communications of the ACM, 29(11), 1044–1050.

    Article  Google Scholar 

  • Kemmerer, R.A. (1985). Testing formal specification to detect design errors.IEEE Transactions on Software Engineering, 11(1), 32–43.

    Article  Google Scholar 

  • Kernighen, B.W., & Plauger, P.J. (1974).The Elements of programming style. New York, NY: McGraw-Hill Book Co.

    Google Scholar 

  • Leite, J.C.S.P., & Freeman, P.A. (1991). Requirements validation through viewpoint resolution.IEEE Transactions on Software Engineering, 17(12), 1253–1269.

    Article  Google Scholar 

  • Oman, P.W., & Cook, C.R. (1988). A paradigm for programming style research.SIGPLAN Notices, 23(12), 69–79.

    Article  Google Scholar 

  • Parker, C. (1991).Z tools catalogue. ZIP/BAe/90/020. Warton, UK: Software Technology Dept, British Aerospace.

    Google Scholar 

  • Pressman, R. (1992).Software engineering. Berkshire, UK: McGraw-Hill Company Europe.

    Google Scholar 

  • Redish, K.A., & Smyth, W.F. (1986). Program style analysis: A natural by-product of program compilation.Communications of the ACM, 29(2), 126–133.

    Article  Google Scholar 

  • Rees, M.J. (1982). Automatic assessment aid for pascal programs.SIGPLAN Notices, 17(10), 33–42.

    Article  Google Scholar 

  • Shukur, Z. (1997). Automating Z specification assessment process.Technical Report. University of Nottingham.

  • Shukur, Z. (1998). Applying Z coursework on-line.Proceedings of Association for Learning Technology Conference ’98 (p. 35). Oxford, UK.

  • Spivey, J.M. (1988a).The Z notation: Reference Manual. Hertfordshire, UK: Prentice Hall.

    Google Scholar 

  • Spivey, J.M. (1988b)Understanding Z: A specification language and its formal semantics. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Whitty, R. (1989). Structural metrics for Z specifications.Fourth Annual ZUM. Oxford, UK: Rewley House.

    Google Scholar 

  • Wordsworth, J.B. (1987). Education in formal methods for software engineering.Information and Software Technology, 29(1), 27–32.

    Article  Google Scholar 

  • Wordsworth, J.B. (1996). An industrial perspective on educational issues relating to formal methods. In C.N. Dean & M.G. Hinchey, (Ed.)Teaching and Learning Formal Methods (pp 1–9). San Diego, CA: Academic Press International Series in Formal Methods.

    Google Scholar 

  • Zin, A.M., & Foxley, E. (1994a). Analyse: An automatic program assessment system.Malaysian Journal of Computer Science, 7, 123–142.

    Google Scholar 

  • Zin, A.M. (1994b). A formal development support system based on the liberal approach. Ph.D. Theses, University of Nottingham, UK.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Nottingham, NG7 2RD, University Park, Nottingham, UK

    Zarina Shukur, Edmund Burke & Eric Foxley

Authors
  1. Zarina Shukur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edmund Burke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Foxley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edmund Burke.

Additional information

Zarina Shukur is studying for a Ph.D. degree in the Department of Computer Science at the University of Nottingham. She belongs to the Learning Technology Research Group and works under the supervision of Dr. E. Burke and Dr. E. Foxley.

Edmund Burke is a Senior Lecturer in the Department of Computer Science at the University of Nottingham. He is a member of the Learning Technology Research Group and is a leader of the Automated Scheduling and Planning Research Group at the university.

Eric Foxley is an Associate Senior Lecturer at the Department of Computer Science. He is a member of the Learning Technology Research Group and is responsible for supporting the Ceilidh system.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shukur, Z., Burke, E. & Foxley, E. The automatic assessment of formal specification coursework. J. Comput. High. Educ. 11, 86–119 (1999). https://doi.org/10.1007/BF02940843

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02940843

Keywords

Search

Navigation