Abstract
At an early stage in the development process, a development team must obtain insight into the software being developed to establish a reliable plan. Thus, the team members should investigate, in depth, any information relating to the development. A major challenge for developers is software development effort estimation (SDEE), which refers to gauging the amount of effort needed to develop the software. In agile methodologies, a project is delivered in iterations, each of which delivers a set of requirements known as user stories. Therefore, SDEE in agile focuses on estimating a single user story’s effort, not the project as a whole, as in traditional development. Among the various techniques, data-driven methods have proved effective in effort estimation, as they are unaffected by external pressure from managers. Moreover, no experts have to be available at the point when estimation is undertaken. By conducting a systematic literature review, this study presents a comprehensive overview of data-driven techniques for user story effort estimation. The results show that there has been limited work on this topic. Studies were analysed to address questions covering five main points: technique; performance evaluation method; accuracy, independent factors (effort drivers); and the characteristics of the datasets. The main performance evaluation methods are performance measures, baseline benchmarks, statistical tests, distribution of estimates, comparison against similar existing techniques and human estimation. Four types of independent factors were identified: personnel; product; process; and estimation. Furthermore, the story point was found to be the most frequently used effort metric in agile user stories.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availibility
Not applicable.
Code availability
Not applicable.
References
Abrahamsson P, Fronza I, Moser R, Vlasenko J, Pedrycz W (2011) Predicting development effort from user stories. In: 2011 international symposium on empirical software engineering and measurement, https://doi.org/10.1109/ESEM.2011.58
Adnan M, Afzal M (2017) Ontology based multiagent effort estimation system for scrum agile method. IEEE Access. https://doi.org/10.1109/ACCESS.2017.2771257
Alloghani M, Al-Jumeily Obe D, Mustafina J, Hussain A, Aljaaf A (2020) A systematic review on supervised and unsupervised machine learning algorithms for data science. Superv Unsuperv Learn Data Sci. https://doi.org/10.1007/978-3-030-22475-2_1
Alostad JM, Abdullah LRA, Aali LS (2017) A fuzzy based model for effort estimation in scrum projects. Int J Adv Comput Sci Appl (IJACSA)https://doi.org/10.14569/IJACSA.2017.080939
Altaleb A, Altherwi M, Gravell A (2020) An industrial investigation into effort estimation predictors for mobile app development in agile processes. In: 2020 9th international conference on industrial technology and management (ICITM), https://doi.org/10.1109/ICITM48982.2020.9080362
Arora M, Verma S, Kavita, Chopra S (2020) A systematic literature review of machine learning estimation approaches in scrum projects. https://doi.org/10.1007/978-981-15-1451-7_59
Aslam W, Ijaz F, Lali MIU, Mehmood W (2017) Risk aware and quality enriched effort estimation for mobile applications in distributed agile software development. J Inf Sci Eng 33:1481–1500
Azhar D, Mendes E, Riddle P (2012) A systematic review of web resource estimation. In: PROMISE ’12: proceedings of the 8th international conference on predictive models in software engineering, https://doi.org/10.1145/2365324.2365332
Basri S, Kama N, Haneem F, Ismail SA (2016a) Predicting effort for requirement changes during software development. In: proceedings of the seventh symposium on information and communication technology, https://doi.org/10.1145/3011077.3011096
Basri S, Kama N, Sarkan HM, Adli S, Haneem F (2016b) An algorithmic-based change effort estimation model for software development. In: 2016 23rd Asia-Pacific software engineering conference (APSEC), https://doi.org/10.1109/APSEC.2016.034
Bilgaiyan S, Aditya K, Mishra S, Das M (2018) Chaos-based modified morphological genetic algorithm for software development cost estimation. In: Progress in Computing, Analytics and Networking, https://doi.org/10.1007/978-981-10-7871-2_4
Bilgaiyan S, Mishra S, Das M (2019) Effort estimation in agile software development using experimental validation of neural network models. Int J Inf Technol. https://doi.org/10.1007/s41870-018-0131-2
Brezočnik L, Fister I, Podgorelec V (2019) Solving agile software development problems with swarm intelligence algorithms. In: international conference “new technologies, development and applications”, https://doi.org/10.1007/978-3-030-18072-0_35
Čelar S, Turić M, Vicković L (2014) Method for personal capability assessment in agile teams using personal points. In: 2014 22nd telecommunications forum telfor (TELFOR), https://doi.org/10.1109/TELFOR.2014.7034607
Choetkiertikul M, Dam HK, Tran T, Pham T, Ghose A, Menzies T (2019) A deep learning model for estimating story points. IEEE Trans Softw Eng. https://doi.org/10.1109/TSE.2018.2792473
Chongpakdee P, Vatanawood W (2017) Estimating user story points using document fingerprints. In: 8th IEEE international conference on software engineering and service science (ICSESS), https://doi.org/10.1109/ICSESS.2017.8342885
Coelho E, Basu A (2012) Effort estimation in agile software development using story points. Int J Appl Inf Syst (IJAIS). https://doi.org/10.5120/ijais12-450574
Cohn M (2005) Agile estimating and planning. Pearson Education
Conoscenti M, Besner V, Vetrò A, Fernández DM (2019) Combining data analytics and developers feedback for identifying reasons of inaccurate estimations in agile software development. J Syst Softw. https://doi.org/10.1016/j.jss.2019.06.075
Dam H (2019) Empowering Software Engineering with. Artif Intell. https://doi.org/10.1007/978-3-030-32242-7_3
Dam HK (2018) Empowering software engineering with artificial intelligence. In: Service Research and Innovation, https://doi.org/10.1007/978-3-030-32242-7_3
Dave VS, Dutta K (2014) Neural network based models for software effort estimation: a review. Artif Intell Rev. https://doi.org/10.1007/s10462-012-9339-x
Dewi R, Sarno R (2020). Software effort estimation using early cosmic to substitute use case weight. https://doi.org/10.1109/iSemantic50169.2020.9234227
de Guevara FGL, Fernández-Diego M, Lokan C (2016) The usage of isbsg data fields in software effort estimation: a systematic mapping study. J Syst Softw. https://doi.org/10.1016/j.jss.2015.11.040
Dhir S, Kumar D, Singh V (2017) Feedforward and feedbackward approach-based estimation model for agile software development. In: Advances in Computer and Computational Sciences, https://doi.org/10.1007/978-981-10-3770-2_7
Dragicevic S, Celar S, Turic M (2017) Bayesian network model for task effort estimation in agile software development. J Syst Softw. https://doi.org/10.1016/j.jss.2017.01.027
Durán M, Juárez-Ramírez R, Jiménez S, Tona C (2020) User story estimation based on the complexity decomposition using bayesian networks. Program Comput Softw. https://doi.org/10.1134/S0361768820080095
Fernández-Diego M, Méndez ER, González-Ladrón-De-Guevara F, Abrahão S, Insfran E (2020) An update on effort estimation in agile software development: a systematic literature review. IEEE Access. https://doi.org/10.1109/ACCESS.2020.3021664
Ferrucci F, Mendes E, Sarro F (2012). Web effort estimation: The value of cross-company data set compared to single-company data set. https://doi.org/10.1145/2365324.2365330
Grabis J, Minkēviča V, Haidabrus B, Popovs R (2020) Is team always right: Producing risk aware effort estimates in agile development. In: international conference on business informatics research, https://doi.org/10.1007/978-3-030-61140-8_7
Grimstad S, Jorgensen M (2009) Preliminary study of sequence effects in judgment-based software development work-effort estimation. IET Softw. https://doi.org/10.1049/iet-sen.2008.0110
Haugen N (2006) An empirical study of using planning poker for user story estimation. In: AGILE 2006 (AGILE’06), https://doi.org/10.1109/AGILE.2006.16
Hussain I, Kosseim L, Ormandjieva O (2010) Towards approximating cosmic functional size from user requirements in agile development processes using text mining. In: Natural Language Processing and Information Systems
Hussain I, Kosseim L, Ormandjieva O (2013) Approximation of cosmic functional size to support early effort estimation in agile. Data Knowl Eng. https://doi.org/10.1016/j.datak.2012.06.005
Idria A, Hosnia M, Abranb A (2016) Systematic literature review of ensemble effort estimation. J Syst Softw. https://doi.org/10.1016/j.jss.2016.05.016
Jorgensen M, Grimstad S (2012) Software development estimation biases: the role of interdependence. IEEE Trans Softw Eng. https://doi.org/10.1109/TSE.2011.40
Jørgensen M, Halkjelsvik T (2020) Sequence effects in the estimation of software development effort. J Syst Softw. https://doi.org/10.1016/j.jss.2019.110448
Jørgensen M, Boehm B, Rifkin S (2009) Software development effort estimation: formal models or expert judgment? IEEE Softw. https://doi.org/10.1109/MS.2009.47
Kaushik A, Tayal DK, Yadav K (2020) A comparative analysis on effort estimation for agile and non-agile software projects using dbn-alo. Arabian J Sci Eng. https://doi.org/10.1007/s13369-019-04250-6
Kaushik A, Tayal DK, Yadav K (2020b) A fuzzy approach for cost and time optimization in agile software development. In: Advanced Computing and Intelligent Engineering, https://doi.org/10.1007/978-981-15-1081-6_53
Kaushik A, Tayal DK, Yadav K (2022) The role of neural networks and metaheuristics in agile software development effort estimation. In: Research Anthology on Artificial Neural Network Applications, https://doi.org/10.4018/978-1-6684-2408-7.ch014
Khuat T, Thi My Hanh L (2016) An effort estimation approach for agile software development using fireworks algorithm optimized neural network. Int J Comput Sci Inf Secur. https://doi.org/10.1016/j.infsof.2011.09.002
Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering
Kocaguneli E, Menzies T, Keung J (2012) On the value of ensemble effort estimation. IEEE Trans Softw Eng. https://doi.org/10.1109/TSE.2011.111
Kompella L (2013). Advancement of decision-making in agile projects by applying logistic regression on estimates. https://doi.org/10.1109/ICGSEW.2013.9
Lenarduzzi V, Lunesu I, Matta M, Taibi D (2015) Functional size measures and effort estimation in agile development: a replicated study. In: international conference on agile software development, https://doi.org/10.1007/978-3-319-18612-2_9
López-Martínez J, Juárez-Ramírez R, Ramírez-Noriega A, Licea G, Navarro-Almanza R (2017) Estimating user stories’ complexity and importance in scrum with bayesian networks. In: world conference on information systems and technologies, https://doi.org/10.1007/978-3-319-56535-4_21
López-Martínez J, Ramírez-Noriega A, Juárez-Ramírez R, Licea G, Jiménez S (2018) User stories complexity estimation using bayesian networks for inexperienced developers. Cluster Comput. https://doi.org/10.1007/s10586-017-0996-z
Mahnič V, Hovelja T (2012) On using planning poker for estimating user stories. J Syst Softw. https://doi.org/10.1016/j.jss.2012.04.005
Malgonde O, Chari K (2019) An ensemble-based model for predicting agile software development effort. Empir Softw Eng. https://doi.org/10.1007/s10664-018-9647-0
Matharu GS, Mishra A, Singh H, Upadhyay P (2015) Empirical study of agile software development methodologies: a comparative analysis. SIGSOFT Softw Eng Notes 40:1–6
Minku LL, Yao X (2014) How to make best use of cross-company data in software effort estimation? In: proceedings of the 36th international conference on software engineering, https://doi.org/10.1145/2568225.2568228
Moharreri K, Sapre AV, Ramanathan J, Ramnath R (2016) Cost-effective supervised learning models for software effort estimation in agile environments. In: 2016 IEEE 40th annual computer software and applications conference (COMPSAC), https://doi.org/10.1109/COMPSAC.2016.85
Moløkken-Østvold K, Jørgensen M (2004) Group processes in software effort estimation. Empir Softw Eng. https://doi.org/10.1023/B:EMSE.0000039882.39206.5a
Nassif AB, Azzeh M, Idri A, Abran A (2019) Software development effort estimation using regression fuzzy models. Comput Intell Neurosci. https://doi.org/10.1155/2019/8367214
Panda A, Satapathy SM, Rath SK (2015) Empirical validation of neural network models for agile software effort estimation based on story points. Procedia Computer Science 57:772–781. https://doi.org/10.1016/j.procs.2015.07.474, 3rd international conference on recent trends in computing 2015 (ICRTC-2015)
Phannachitta P (2020) On an optimal analogy-based software effort estimation. Inf Softw Technol 125:106330. https://doi.org/10.1016/j.infsof.2020.106330
Popli R, Chauhan N (2014) Cost and effort estimation in agile software development. In: 2014 international conference on reliability optimization and information technology (ICROIT), https://doi.org/10.1109/ICROIT.2014.6798284
Pow-Sang JA, Imbert R (2012) Effort estimation in incremental software development projects using function points. In: Computer Applications for Software Engineering, Disaster Recovery, and Business Continuity, https://doi.org/10.1007/978-3-642-35267-6_61
Radu LD (2019) Effort prediction in agile software development with bayesian networks. In: ICSOFT
Rao CP, Kumar PS, Sree SR, Devi J (2018) An agile effort estimation based on story points using machine learning techniques. In: proceedings of the second international conference on computational intelligence and informatics, https://doi.org/10.1007/978-981-10-8228-3_20
Ratke C, Hoffmann HH, Gaspar T, Floriani PE (2019) Effort estimation using bayesian networks for agile development. In: 2019 2nd international conference on computer applications information security (ICCAIS), https://doi.org/10.1109/CAIS.2019.8769455
Rosa W, Madachy R, Clark B, Boehm B (2017) Early phase cost models for agile software processes in the us dod. In: 2017 ACM/IEEE international symposium on empirical software engineering and measurement (ESEM), https://doi.org/10.1109/ESEM.2017.10
Rosencrance L (2007) Survey: poor communication causes most it project failures
Rowe G, Wright G (1999) The delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15:353–375. https://doi.org/10.1016/S0169-2070(99)00018-7
Rubin K (2013) Essential scrum: a practical guide to the most popular agile process
Saeed A, Butt WH, Kazmi F, Arif M (2018) Survey of software development effort estimation techniques. In: proceedings of the 2018 7th international conference on software and computer applications, https://doi.org/10.1145/3185089.3185140
Saini A, Ahuja L, Khatri SK (2018) Effort estimation of agile development using fuzzy logic. In: 2018 7th international conference on reliability, infocom technologies and optimization (trends and future directions) (ICRITO), https://doi.org/10.1109/ICRITO.2018.8748381
Satapathy SM, Rath SK (2017) Empirical assessment of machine learning models for agile software development effort estimation using story points. Innov Syst Softw Eng. https://doi.org/10.1007/s11334-017-0288-z
Schwaber K, Sutherland J (2020) The scrum guide\({}^{{\rm TM}}\)
Schweighofer T, Kline A, Pavlic L, Hericko M (2016) How is effort estimated in agile software development projects? In: proceedings of 5th Workshop Software Qual., Anal., Monitor., Improvement, Appl. (SQAMIA)
Scott E, Pfahl D (2018). Using developers’ features to estimate story points. https://doi.org/10.1145/3202710.3203160
Serrador P, Pinto JK (2015) Does agile work?—A quantitative analysis of agile project success. Int J Project Manag. https://doi.org/10.1016/j.ijproman.2015.01.006
Sharma A, Chaudhary N (2020). Linear regression model for agile software development effort estimation. https://doi.org/10.1109/ICRAIE51050.2020.9358309
Sharma H, Tomar R, Dumka A, Aswal M (2015) Openecocomo: The algorithms and implementation of extended cost constructive model (e-cocomo). https://doi.org/10.1109/NGCT.2015.7375225
Soares RGF (2018) Effort estimation via text classification and autoencoders. In: 2018 international joint conference on neural networks (IJCNN), https://doi.org/10.1109/IJCNN.2018.8489030
Tanveer B, Guzmán L, Engel UM (2017) Effort estimation in agile software development: case study and improvement framework. J Softw Evol Process. https://doi.org/10.1002/smr.1862
Tanveer B, Vollmer AM, Braun S, Bin Ali N (2019) An evaluation of effort estimation supported by change impact analysis in agile software development. J Softw Evol Process. https://doi.org/10.1002/smr.2165
Trendowicz A, Jeffery R (2014) Software project effort estimation. Springer, Berlin
Ungan E, Çizmeli N, Demirörs O (2014) Comparison of functional size based estimation and story points, based on effort estimation effectiveness in scrum projects. In: 2014 40th EUROMICRO conference on software engineering and advanced applications, https://doi.org/10.1109/SEAA.2014.83
Usman M, Mendes E, Neiva F, Britto R (2014). Effort estimation in agile software development: a systematic literature review. https://doi.org/10.1145/2639490.2639503
Wen J, Li S, Lin Z, Hu Y, Huang C (2012) Systematic literature review of machine learning based software development effort estimation models. Inf Softw Technol. https://doi.org/10.1016/j.infsof.2011.09.002
Wińska E, Kot E, Dabrowski W (2021) Reducing the uncertainty of agile software development using a random forest classification algorithm. In: international conference on lean and agile software development, https://doi.org/10.1007/978-3-030-67084-9_9
Wohlin C (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. https://doi.org/10.1145/2601248.2601268
Zang J (2008) Agile estimation with monte carlo simulation. In: international conference on agile processes and extreme programming in software engineering, https://doi.org/10.1007/978-3-540-68255-4_17
Funding
None.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Alsaadi, B., Saeedi, K. Data-driven effort estimation techniques of agile user stories: a systematic literature review. Artif Intell Rev 55, 5485–5516 (2022). https://doi.org/10.1007/s10462-021-10132-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-021-10132-x