skip to main content
10.1145/2972958.2972959acmotherconferencesArticle/Chapter ViewAbstractPublication PagespromiseConference Proceedingsconference-collections
research-article

Estimating Story Points from Issue Reports

Published: 09 September 2016 Publication History

Abstract

Estimating the effort of software engineering tasks is notoriously hard but essential for project planning. The agile community often adopts issue reports to describe tasks, and story points to estimate task effort. In this paper, we propose a machine learning classifier for estimating the story points required to address an issue. Through empirical evaluation on one industrial project and eight open source projects, we demonstrate that such classifier is feasible. We show that ---after an initial training on over 300 issue reports--- the classifier estimates a new issue in less than 15 seconds with a mean magnitude of relative error between 0.16 and 0.61. In addition, issue type, summary, description, and related components prove to be project dependent features pivotal for story point estimation.

References

[1]
T. K. Abdel-Hamid. Investigating the cost/schedule trade-off in software development. Software, IEEE, 7(1):97--105, 1990.
[2]
W. AbdelMoez, M. Kholief, and F. M. Elsalmy. Improving bug fix-time prediction model by filtering out outliers. In Proceedings of the 2013 International Conference on Technological Advances in Electrical, Electronics and Computer Engineering, pages 359--364, May 2013.
[3]
P. Abrahamsson, I. Fronza, R. Moser, J. Vlasenko, and W. Pedrycz. Predicting development effort from user stories. In Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement, pages 400--403. IEEE, 2011.
[4]
J. Aranda and S. Easterbrook. Anchoring and adjustment in software estimation. In Proceedings of the 2005 European Software Engineering Conference Held Jointly with the 2005 International Symposium on Foundations of Software Engineering, pages 346--355. ACM, 2005.
[5]
E. Aronson, T. D. Wilson, and R. M. Akert. Social Psychology (3rd Edition). Pearson, 1999.
[6]
R. L. Atkinson, R. C. Atkinson, E. E. Smith, D. J. Bem, and S. Nolen-Hoeksema. Hilgard's Introduction to Psychology (12th Edition). Orlando: Harcourt Brace College Publishers, 1996.
[7]
N. C. Augen. An empirical study of using planning poker for user story estimation. In Agile Conference, 2006, pages 9--pp. IEEE, 2006.
[8]
K. Beck, M. Beedle, A. Van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, et al. Manifesto for agile software development, 2001.
[9]
L. C. Briand. On the many ways software engineering can benefit from knowledge engineering. In Proceedings of the 2002 international conference on Software engineering and knowledge engineering, pages 3--6. ACM, 2002.
[10]
R. Brown. Group Processes: Dynamics Within and Between Groups. Wiley-Blackwell, 2000.
[11]
M. Cohn. It's effort, not complexity. https://goo.gl/nNYlL1, 2010. Accessed: 2016-06-02.
[12]
T. DeMarco. Controlling software projects: Management, measurement, and estimates. Prentice Hall PTR, 1986.
[13]
Why do the cards deviate slightly from the fibonacci sequence? http://goo.gl/xgs7PF. Accessed: 2016-07-13.
[14]
J. Grenning. Planning poker or how to avoid analysis paralysis while release planning. Hawthorn Woods: Renaissance Software Consulting, 3, 2002.
[15]
R. T. Hughes. Expert judgement as an estimating method. Information and Software Technology, 38(2):67--75, 1996.
[16]
Github issues tutorial. https://goo.gl/OR1gwr. Accessed: 2016-07-13.
[17]
What is an issue - atlassian documentation. https://goo.gl/JduWpL. Accessed: 2016-07-13.
[18]
N. Japkowicz and M. Shah. Evaluating learning algorithms: a classification perspective. Cambridge University Press, 2011.
[19]
Jira - pros and cons of cloud vs. server. https://goo.gl/5KF2rI. Accessed: 2016-07-13.
[20]
T. Joachims. Text categorization with support vector machines: Learning with many relevant features. Springer, 1998.
[21]
L. Marks, Y. Zou, and A. E. Hassan. Studying the fix-time for bugs in large open source projects. In Proceedings of the 2011 International Conference on Predictive Models in Software Engineering, page 11. ACM, 2011.
[22]
A. Murgia, G. Concas, R. Tonelli, M. Ortu, S. Demeyer, and M. Marchesi. On the influence of maintenance activity types on the issue resolution time. In Proceedings of the 2014 International Conference on Predictive Models in Software Engineering, pages 12--21. ACM, 2014.
[23]
A. Murgia, D. Janssens, S. Demeyer, and B. Vasilescu. Among the machines: Human-bot interaction on social Q&A websites. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA '16, pages 1272--1279, New York, NY, USA, 2016. ACM.
[24]
A. Murgia, P. Tourani, B. Adams, and M. Ortu. Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proceedings of the 11th Working Conference on Mining Software Repositories, MSR 2014, pages 262--271, New York, NY, USA, 2014. ACM.
[25]
J. W. Paulson, G. Succi, and A. Eberlein. An empirical study of open-source and closed-source software products. IEEE Transactions on Software Engineering, 30(4):246--256, 2004.
[26]
Planning poker 3.0. https://goo.gl/Tl5mws. Accessed: 2016-06-16.
[27]
I. Rish. An empirical study of the naive bayes classifier. Technical report, 2001.
[28]
Q. Song, M. Shepperd, M. Cartwright, and C. Mair. Software defect association mining and defect correction effort prediction. IEEE Transactions on Software Engineering, 32(2):69--82, 2006.
[29]
A. Trendowicz, J. Münch, and R. Jeffery. State of the practice in software effort estimation: a survey and literature review. In Software Engineering Techniques, pages 232--245. Springer, 2008.
[30]
M. Usman, E. Mendes, F. Weidt, and R. Britto. Effort estimation in agile software development: A systematic literature review. In Proceedings of the 2014 International Conference on Predictive Models in Software Engineering, pages 82--91. ACM, 2014.
[31]
C. Weiss, R. Premraj, T. Zimmermann, and A. Zeller. How long will it take to fix this bug? In Proceedings of the 2007 International Workshop on Mining Software Repositories, page 1. IEEE Computer Society, 2007.
[32]
What is a story point? https://goo.gl/Vfb9v1, 2007. Accessed: 2016-06-02.
[33]
H. Zeng and D. Rine. Estimation of software defects fix effort using neural networks. In Proceedings of the 2004 Annual International Conference on Computer Software and Applications, volume 2, pages 20--21. IEEE, 2004.

Cited By

View all
  • (2024)Enhancing Software Effort Estimation with Pre-Trained Word Embeddings: A Small-Dataset Solution for Accurate Story Point PredictionElectronics10.3390/electronics1323484313:23(4843)Online publication date: 8-Dec-2024
  • (2024)A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in AgileACM Computing Surveys10.1145/366336556:11(1-37)Online publication date: 28-Jun-2024
  • (2024)Case Study of a Model that evaluates the Learner Experience with DICTsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637138(1-9)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
PROMISE 2016: Proceedings of the The 12th International Conference on Predictive Models and Data Analytics in Software Engineering
September 2016
84 pages
ISBN:9781450347723
DOI:10.1145/2972958
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Machine learning
  2. agile
  3. issue report
  4. story points

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Institute for the Promotion of Innovation through Science and Technology in Flanders
  • Sardinia Regional Government

Conference

PROMISE 2016

Acceptance Rates

PROMISE 2016 Paper Acceptance Rate 10 of 23 submissions, 43%;
Overall Acceptance Rate 98 of 213 submissions, 46%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)88
  • Downloads (Last 6 weeks)6
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Enhancing Software Effort Estimation with Pre-Trained Word Embeddings: A Small-Dataset Solution for Accurate Story Point PredictionElectronics10.3390/electronics1323484313:23(4843)Online publication date: 8-Dec-2024
  • (2024)A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in AgileACM Computing Surveys10.1145/366336556:11(1-37)Online publication date: 28-Jun-2024
  • (2024)Case Study of a Model that evaluates the Learner Experience with DICTsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637138(1-9)Online publication date: 11-May-2024
  • (2024)Fine-SE: Integrating Semantic Features and Expert Features for Software Effort EstimationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623349(1-12)Online publication date: 20-May-2024
  • (2024)A Review on Improving the Accuracy of Effort Estimation in Software Development with Agile Method2024 11th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE)10.1109/ICITACEE62763.2024.10761956(31-36)Online publication date: 29-Aug-2024
  • (2024)Large Language Model Employment for Story Point Estimation Problems in AGILE Development2024 International Conference on Electrical Engineering and Computer Science (ICECOS)10.1109/ICECOS63900.2024.10791206(391-398)Online publication date: 25-Sep-2024
  • (2024)Estimating Story Points in Scrum: Balancing Accuracy and Interpretability with Explainable AI2024 6th International Conference on Advancements in Computing (ICAC)10.1109/ICAC64487.2024.10850952(139-144)Online publication date: 12-Dec-2024
  • (2024)Software Effort Estimation Using Deep Learning: A Gentle ReviewArtificial Intelligence and Sustainable Computing10.1007/978-981-97-0327-2_26(351-364)Online publication date: 24-Apr-2024
  • (2024)Analyzing the Influence of Processor Speed and Clock Speed on Remaining Useful Life Estimation of Software SystemsIntelligent Computing10.1007/978-3-031-62281-6_34(490-507)Online publication date: 14-Jun-2024
  • (2024)Predicting the Duration of User Stories in Agile Project ManagementSmart Technologies for a Sustainable Future10.1007/978-3-031-61905-2_31(316-328)Online publication date: 13-Jun-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media