Skip to main content
SpringerLink
Log in

Guidelines for reporting results of computational experiments. Report of the ad hoc committee

  • Published:
Mathematical Programming Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. J.C.P. Bus, “Performance measures for evaluating mathematical programming software,”COAL Newsletter (Jan. 1980) 8.

  2. H.P. Crowder, R.S. Dembo and J.M. Mulvey, “On reporting computational experiments with mathematical software,”ACM Transactions on Mathematical Software 5 (1979) 193–203.

    Google Scholar 

  3. H.P. Crowder and P.B. Saunders, “Results of a survey on MP performance indicators,”COAL Newsletter (Jan. 1980) 2–6.

  4. J.J. Dongarra, J.R. Bunch, C.B. Moler and G.W. Stewart,Linpack Users' Guide (SIAM Publications, Philadelphia, PA, 1979).

    Google Scholar 

  5. J.J. Elam and D. Klingman, “NETGEN II: A system for generating structured network-based mathematical programming test problems,” in: J.M. Mulvey, ed.,Evaluating Mathematical Programming Techniques, Lecture Notes in Economics and Mathematical Systems No. 199 (Springer-Verlag, New York, 1982) pp. 16–23.

    Google Scholar 

  6. D.M. Gay, “Algorithm 611. Subroutines for unconstrained minimization using a model/trust-region approach,”ACM Transactions on Mathematical Software 9 (1983) 503–524.

    Google Scholar 

  7. D.M. Gay, “Electronic mail distribution of linear programming test problems,”COAL Newsletter 13 (Dec. 1985) 10–12.

    Google Scholar 

  8. J. Gilsinn, K. Hoffman, R. Jackson, E. Leyendecker, P. Saunders and D. Shier, “Methodology and analysis for comparing discrete linear L1 approximation codes,”Communications in Statistics 136 (1977) 399–413.

    Google Scholar 

  9. K.L. Hiebert, “An evaluation of mathematical software that solves nonlinear least squares problems,”ACM Transactions on Mathematical Software 7 (1981) 1–16.

    Google Scholar 

  10. K.L. Hiebert, “An evaluation of mathematical software that solves systems of nonlinear equations,”ACM Transactions on Mathematical Software 8 (1982) 5–20.

    Google Scholar 

  11. J.K. Ho and E. Loute, “Computational experience with advanced implementation of decomposition algorithms for linear programming,”Mathematical Programming 27 (1983) 283–290.

    Google Scholar 

  12. D.C. Hoaglin, V.C. Klema and S.C. Peters, “Exploratory data analysis in a study of the performance of nonlinear optimization routines,”ACM Transactions on Mathematical Software 8 (1982) 145–162.

    Google Scholar 

  13. W. Hoch and K. Schittkowski,Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems No. 187 (Springer-Verlag, New York, 1981).

    Google Scholar 

  14. R. Hockney and C. Jesshope,Parallel Computers: Architecture, Programming and Algorithms (Adam Hilger, Bristol, 1981).

    Google Scholar 

  15. A. Hoffman, M. Mannos, D. Sakolowsky and N. Wieymann, “Computational experience in solving linear programs,”Journal of SIAM 1 (1953) 17–33.

    Google Scholar 

  16. K.L. Hoffman and D.R. Shier, “A test problem generator for discrete linear L1 approximation problems,”ACM Transactions on Mathematical Software 6 (1980) 587–593.

    Google Scholar 

  17. K.L. Hoffman and R.H.F. Jackson, “In pursuit of a methodology for testing mathematical programming software,” in: J.M. Mulvey, ed.,Evaluating Mathematical Programming Techniques, Lecture Notes in Economics and Mathematical Systems No. 199 (Springer-Verlag, New York, 1982) pp. 177–199.

    Google Scholar 

  18. R.H.F. Jackson and J.M. Mulvey, “A critical review of comparisons of mathematical programming algorithms and software (1953–1977),”Journal of Research of the National Bureau of Standards 83 (1978) 563–583.

    Google Scholar 

  19. D. Klingman, A. Napier and F. Stutz, “NETGEN: A program for generalizing large scale assignment, transportation, and minimum cost flow network problems,”Management Science 20 (1974) 814–821.

    Google Scholar 

  20. G. Lidor, “Construction of nonlinear programming test problems with known solution characteristics,” in: J.M. Mulvey, ed.,Evaluating Mathematical Programming Techniques, Lecture Notes in Economics and Mathematical Systems No. 199 (Springer-Verlag, New York, 1982) pp. 25–35.

    Google Scholar 

  21. I. Lustig, “An analysis of an available set of linear programming test problems,”Computers and Operations Research 16 (1989) 173–184.

    Google Scholar 

  22. J.N. Lyness, “Performance profiles and software evaluation,” Argonne National Laboratories TM 343 (Argonne, IL, 1979).

    Google Scholar 

  23. J. May and R. Smith, “The definition and generalization of geometrically random linear constraint sets,” in: J.M. Mulvey, ed.,Evaluating Mathematical Programming Techniques, Lecture Notes in Economics and Mathematical Systems No. 199 (Springer-Verlag, New York, 1982) pp. 24–34.

    Google Scholar 

  24. W.M. Michaels and R.P. O'Neill, “A mathematical programming generator: MPGENR,”ACM Transactions on Mathematical Software 6 (1980) 31–40.

    Google Scholar 

  25. A. Miele and S. Gonzales, “On the comparative evaluation of algorithms for mathematical programming problems,” in: O. Mangasarian, R. Meyer, and S. Robinson, eds.,Nonlinear Programming Vol. 3 (New York, 1978) pp. 337–359.

  26. J.J. More, B.S. Garbow and K.E. Hillstrom, “Testing unconstrained optimization software,”ACM Transactions on Mathematical Software 7 (1981) 17–41.

    Google Scholar 

  27. R.P. O'Neill, “A comparison of real-world linear programs and their randomly generated analogs,” in: J.M. Mulvey, ed.,Evaluating Mathematical Programming Techniques, Lecture Notes in Economics and Mathematical Systems No. 199 (Springer-Verlag, New York, 1982) pp. 44–59.

    Google Scholar 

  28. P.M. Pardalos, “Generation of large-scale quadratic programs for use as global optimization test problems,”ACM Transactions on Mathematical Software 13 (1987) 133–137.

    Google Scholar 

  29. J.B. Rosen, “Global minimization of a linearly constrained concave function by partition of the feasible region (Appendix B),” Technical Report 81-11, Computer Science Department, University of Minnesota (Minneapolis, MN, 1981).

    Google Scholar 

  30. K. Schittkowski, “A numerical comparison of optimization software using randomly generated test problems,” Rechenzentrum Preprint No. 43, University of Würzburg (Würzburg, Germany, 1978).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Manufacturing Engineering, National Institute of Standards and Technology, 20899, Gaithersburg, MD, USA

    Richard H. F. Jackson (Chair)

  2. Center for Computing and Applied Mathematics, National Institute of Standards and Technology, 20899, Gaithersburg, MD, USA

    Paul T. Boggs

  3. Operations Research and Applied Statistics Department, George Mason University, 22030, Fairfax, VA, USA

    Stephen G. Nash

  4. London School of Economics and Political Science, WC2A 2AE, London, United Kingdom

    Susan Powell

Authors
  1. Richard H. F. Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul T. Boggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen G. Nash
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susan Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jackson, R.H.F., Boggs, P.T., Nash, S.G. et al. Guidelines for reporting results of computational experiments. Report of the ad hoc committee. Mathematical Programming 49, 413–425 (1990). https://doi.org/10.1007/BF01588801

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01588801

Keywords

Search

Navigation