research-article

The business case for automated software engineering

Authors:

Oussama Elrawas,

Martin Feather,

Barry BoehmAuthors Info & Claims

ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering

Pages 303 - 312

https://doi.org/10.1145/1321631.1321676

Published: 05 November 2007 Publication History

Abstract

Adoption of advanced automated SE (ASE) tools would be favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the "local tuning" problem. Normally, predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable.

This paper shows that assessing the relative merits of different SE methods need not require precise local tunings. STAR1 is a simulated annealer plus a Bayesian post-processor that explores the space of possible local tunings within software prediction models. STAR1 ranks project decisions by their effects on effort and defects and threats. In experiments with two NASA systems, STAR1 found that ASE tools were necessary to minimize effort/ defect/ threats.

References

[1]

B. Boehm. Software Engineering Economics. Prentice Hall, 1981.

Digital Library

[2]

B. Boehm. Safe and simple software cost analysis. IEEE Software, pages 14--17, September/October 2000. Available from http://www.computer.org/ certification/beta/Boehm_Safe.pdf.

Digital Library

[3]

B. Boehm, E. Horowitz, R. Madachy, D. Reifer, B. K. Clark, B. Steece, A. W. Brown, S. Chulani, and C. Abts. Software Cost Estimation with Cocomo II. Prentice Hall, 2000.

Digital Library

[4]

S. Chulani, B. Boehm, and B. Steece. Bayesian analysis of empirical software engineering cost models. IEEE Transaction on Software Engineerining, 2 (4), July/August 1999.

Digital Library

[5]

J. Clark, J. J. Dolado, M. Harman, R. M. Hierons, B. Jones, M. Lumkin, B. Mitchell, S. Mancoridis, K. Rees, M. Roper, and M. Shepperd. Reformulating software engineering as a search problem. IEE Proceedings on Software, 150(3):161--175, 2003. Available from http://www.brunel.ac.uk/~csstrmh/papers/sbse.ps.

[6]

R. Clark. Faster treatment learning, Computer Science, Portland State University. Master's thesis, 2005.

[7]

S. L. Cornford, M. S. Feather, J. Dunphy, J. Salcedo, and T. Menzies. Optimizing spacecraft design optimization engine development: Progress and plans. In Proceedings of the IEEE Aerospace Conference, Big Sky, Montana, 2003. Available from http://menzies.us/pdf/03aero.pdf.

[8]

T. DeMarco and T. Lister. Peopleware: productive projects and teams. Dorset House Publishing Co., Inc., New York, NY, USA, 1987.

Digital Library

[9]

S. Devnani-Chulani. Bayesian Analysis of Software Cost and Quality Models. PhD thesis, 1999. Available on-line at http://citeseer.ist.psu.edu/devnani-chulani99bayesian.html.

Digital Library

[10]

P. Domingos and M. J. Pazzani. On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning, 29(2--3):103--130, 1997.

Digital Library

[11]

M. Feather and T. Menzies. Converging on the optimal attainment of requirements. In IEEE Joint Conference On Requirements Engineering ICRE'02 and RE'02, 9-13th September, University of Essen, Germany, 2002. Available from http://menzies.us/pdf/02re02.pdf.

Digital Library

[12]

M. Hall and G. Holmes. Benchmarking attribute selection techniques for discrete class data mining. IEEE Transactions On Knowledge And Data Engineering, 15(6):1437--1447, 2003. Available from http://www.cs.waikato.ac.nz/~mhall/HallHolmesTKDE.pdf.

Digital Library

[13]

M. Harman and J. Wegener. Getting results from search-based approaches to software engineering. In ICSE '04: Proceedings of the 26th International Conference on Software Engineering, pages 728--729, Washington, DC, USA, 2004. IEEE Computer Society.

Digital Library

[14]

K. Havelund and T. Pressburger. Model checking java programs using java pathfinder. International Journal on Software Tools for Technology Transfer, (4), April 2000. Available from http://ase.arc.nasa.gov/visser/jpf/jpf1.ps.gz.

[15]

G. Holzmann. The model checker SPIN. IEEE Transactions on Software Engineering, 23(5):279--295, May 1997.

Digital Library

[16]

IEEE-1012. IEEE standard 1012--2004 for software verification and validation, 1998.

[17]

B. Jackson, J. Griggs, K. Costello, and D. Solomon. Systems level definition of iv&v, 2006. NASA document IVV 09-1, last revised March 16, 2006, available on-line at http://www.nasa.gov/centers/ivv/pdf/170825main_IVV_09-1.pdf.

[18]

J. Josephson, B. Chandrasekaran, M. Carroll, N. Iyer, B. Wasacz, and G. Rizzoni. Exploration of large design spaces: an architecture and preliminary results. In AAAI '98, 1998. Available from http://www.cis.ohio-state.edu/~jj/Explore.ps.

[19]

J. King and M. Diaz. How CMM impacts quality, productivity, rework, and the bottom line. CROSSTALK, March 2002. Avaiable from http://www.stsc.hill. af.mil/crosstalk/2002/03/diaz.html.

[20]

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, Number 4598, 13 May 1983, 220, 4598:671--680, 1983.

[21]

R. Kohavi and G. H. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1--2):273--324, 1997.

Digital Library

[22]

T. Menzies, Z. Chen, J. Hihn, and K. Lum. Selecting best practices for effort estimation. IEEE Transactions on Software Engineering, November 2006. Available from http://menzies.us/pdf/06coseekmo.pdf.

Digital Library

[23]

T. Menzies and Y. Hu. Data mining for very busy people. In IEEE Computer, November 2003. Available from http://menzies.us/pdf/03tar2.pdf.

Digital Library

[24]

T. Menzies, D. Raffo, S. on Setamanit, Y. Hu, and S. Tootoonian. Model-based tests of truisms. In Proceedings of IEEE ASE 2002, 2002. Available from http://menzies.us/pdf/02truisms.pdf.

Digital Library

[25]

T. Menzies and J. Richardson. Xomo: Understanding development options for autonomy. In COCOMO forum, 2005, 2005. Available from http://menzies.us/pdf/05xomo_cocomo_forum.pdf. For more details, see also the longer technical report http://menzies.us/pdf/05xomo101.pdf.

[26]

T. Menzies and E. Sinsel. Practical large scale what-if queries: Case studies with software risk assessment. In Proceedings ASE 2000, 2000. Available from http://menzies.us/pdf/00ase.pdf.

Digital Library

[27]

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller. J. Chem. Phys, 21:1087--1092, 1953.

[28]

R. Rubinstein and D. Kroese. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer-Verlag, 2004.

Digital Library

[29]

D. Wallace and R. Fujii. Software verification and validation: An overview. IEEE Software, pages 10--17, May 1989.

Digital Library

Cited By

Meem FSmith JJohnson BRoychoudhury APaiva AAbreu RStorey M(2024)Exploring Experiences with Automated Program Repair in PracticeProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639182(1-11)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639182
Jing XChen HXu BJing XChen HXu B(2024)IntroductionIntelligent Software Defect Prediction10.1007/978-981-99-2842-2_1(1-11)Online publication date: 18-Jan-2024
https://doi.org/10.1007/978-981-99-2842-2_1
Shu RXia TChen JWilliams LMenzies T(2021)How to Better Distinguish Security Bug Reports (Using Dual Hyperparameter Optimization)Empirical Software Engineering10.1007/s10664-020-09906-826:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1007/s10664-020-09906-8
Show More Cited By

Index Terms

The business case for automated software engineering

Recommendations

How Much Software Quality Investment Is Enough: A Value-Based Approach

A classical problem facing many software projects is determining when to stop testing and release the product for use. Risk analysis helps address this issue by balancing the risk exposure of doing too little with the risk exposure of doing too much. A ...
Impact and contributions of MBASE on software engineering graduate courses

As the founding Director of the Center for Software Engineering, Professor Barry Boehm developed courses that have greatly impacted the education of software engineering students. Through the use of the MBASE framework and complementary tools, students ...
The influence of COCOMO on software engineering education and training

As the discipline of software engineering has matured, COCOMO (constructive cost model) has evolved, both in response to and as a leading indicator of changes in software engineering methods and techniques. This paper traces the evolution of the COCOMO ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering

November 2007

590 pages

ISBN:9781595938824

DOI:10.1145/1321631

General Chair:
Kurt Stirewalt
Michigan State University, USA
,
Program Chairs:
Alexander Egyed
Teknowledge Corporation, USA
,
Bernd Fischer
University of Southampton, UK

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASE07

Sponsor:

ASE07: International Conference on Automated Software Engineering 2007

November 5 - 9, 2007

Georgia, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
735
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Meem FSmith JJohnson BRoychoudhury APaiva AAbreu RStorey M(2024)Exploring Experiences with Automated Program Repair in PracticeProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639182(1-11)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639182
Jing XChen HXu BJing XChen HXu B(2024)IntroductionIntelligent Software Defect Prediction10.1007/978-981-99-2842-2_1(1-11)Online publication date: 18-Jan-2024
https://doi.org/10.1007/978-981-99-2842-2_1
Shu RXia TChen JWilliams LMenzies T(2021)How to Better Distinguish Security Bug Reports (Using Dual Hyperparameter Optimization)Empirical Software Engineering10.1007/s10664-020-09906-826:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1007/s10664-020-09906-8
Krishna RMenzies T(2020)Learning actionable analytics from multiple software projectsEmpirical Software Engineering10.1007/s10664-020-09843-625:5(3468-3500)Online publication date: 24-Jul-2020
https://dl.acm.org/doi/10.1007/s10664-020-09843-6
Agrawal AMenzies TMinku LWagner MYu Z(2020)Better software analytics via “DUO”: Data mining algorithms using/used-by optimizersEmpirical Software Engineering10.1007/s10664-020-09808-925:3(2099-2136)Online publication date: 22-Apr-2020
https://dl.acm.org/doi/10.1007/s10664-020-09808-9
Nair VAgrawal AChen JFu WMathew GMenzies TMinku LWagner MYu ZZaidman AKamei YHill E(2018)Data-driven search-based software engineeringProceedings of the 15th International Conference on Mining Software Repositories10.1145/3196398.3196442(341-352)Online publication date: 28-May-2018
https://dl.acm.org/doi/10.1145/3196398.3196442
Mathew GMenzies TErnst NKlein J(2017)“SHORT”er Reasoning About Larger Requirements Models2017 IEEE 25th International Requirements Engineering Conference (RE)10.1109/RE.2017.31(154-163)Online publication date: Sep-2017
https://doi.org/10.1109/RE.2017.31
Fu WMenzies TShen X(2016)Tuning for software analyticsInformation and Software Technology10.1016/j.infsof.2016.04.01776:C(135-146)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1016/j.infsof.2016.04.017
Nair VMenzies TChen J(2016)An (Accidental) Exploration of Alternatives to Evolutionary Algorithms for SBSESearch Based Software Engineering10.1007/978-3-319-47106-8_7(96-111)Online publication date: 24-Sep-2016
https://doi.org/10.1007/978-3-319-47106-8_7
Peters FMenzies TLayman LBertolino ACanfora GElbaum S(2015)LACE2Proceedings of the 37th International Conference on Software Engineering - Volume 110.5555/2818754.2818851(801-811)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2818754.2818851
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten