Abstract
Issue addressing is a vital task in the evolution of software projects. However, in practice, not all issues can be addressed on time. To facilitate the issue addressing process, monetary incentives (e.g., bounties) are used to attract developers to address issues. There are two types of core roles who are involved in this process: bounty backers, who propose bounties for an issue report via bounty platforms (e.g., Bountysource), and bounty hunters, who address the bounty issues and win the bounties. We wish to study the process of bounty issue addressing from the angle of two important roles (i.e., backers and hunters) and their related behaviors. With a better understanding of how they address bounty issues, stakeholders (e.g., operators and developers) of open source projects may have a reasonable estimation of what they can expect from backers and hunters. In this study, we investigate 2,955 bounty backers and 882 bounty hunters, and their associated 3,579 GitHub issue reports with 5,589 bounties that were proposed on Bountysource. We find that: 1) Overall, the value of a bounty is small (median bounty value of $20). Both individual and corporate backers prefer to support implementing new features rather than fixing bugs. Corporate backers tend to propose larger bounties and propose bounties more frequently than individual backers. 2) 85.0% of the bounty hunters addressed less than 3 bounty issues. The income of 56.7% of the bounty hunters is no more than $100 and only 2.7% of the hunters have earned more than $2,000. In addition, most of the regular hunters and big hunters are developers that made at least one commit before addressing a bounty issue. 3) The value of a bounty issue is not a statistically significant factor that attracts developers that have never made any commit before to address an issue. Based on our findings, we provide several suggestions for stakeholders of open source projects and hunters.
Similar content being viewed by others
Notes
References
Akobeng AK (2007) Understanding diagnostic tests 3: receiver operating characteristic curves. Acta Paediat 96(5):644–647
Androutsellis-Theotokis S, Spinellis D, Kechagia M, Gousios G, et al. (2011) Open source software: A survey from 10,000 feet. Found Trends Technol Inf Oper Manag 4(3–4):187–347
Apple Inc (2020) Apple Security Bounty). https://developer.apple.com/security-bounty/,. (last visited: Dec 12, 2020)
Atiq A, Tripathi A (2016) Impact of financial benefits on open source software sustainability. In: International conference on information systems (ICIS), pp 1–10
Avelino G, Passos L, Hora A, Valente MT (2016) A novel approach for estimating truck factors. In: IEEE 24th international conference on program comprehension (ICPC), pp 1–10
Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(1):281–305
Bissyandé TF, Thung F, Lo D, Jiang L, Réveillère L (2013) Popularity, interoperability, and impact of programming languages in 100,000 open source projects. In: IEEE 37th annual computer software and applications conference. IEEE, pp 303–312
Canfora G, Di Penta M, Oliveto R, Panichella S (2012) Who is going to mentor newcomers in open source projects?. In: Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering (FSE), pp 1–11
Coelho J, Valente MT, Silva LL, Hora A (2018) Why we engage in floss: Answers from core developers. In: Proceedings of the 11th international workshop on cooperative and human aspects of software engineering, pp 114–121
Comino S, Manenti FM, Parisi ML (2007) From planning to mature: on the success of open source projects. Res Policy 36(10):1575–1586
Dagenais B, Ossher H, Bellamy RKE, Robillard MP, de Vries JP (2010) Moving into a new software project landscape. In: Proceedings of the 32nd ACM/IEEE international conference on software engineering - Volume 1, ICSE ’10, pp 275–284
Dinnie M (2019) How to prioritize feature requests for software development. https://zenkit.com/en/blog/how-to-prioritize-feature-requests-for-software-development,. (last visited: November 8, 2019)
Duebendorfer T, Frei S (2009) Why silent updates boost security. TIK, ETH Zurich, Tech Rep 302
Eghbal N (2016) Roads and bridges: The unseen labor behind our digital infrastructure. Ford Foundation
Eghbal N (2019) A handy guide to financial support for open source
Finifter M, Akhawe D, Wagner D (2013) An empirical study of vulnerability rewards programs. In: USENIX Security Symp., pp 273–288
Frei S, Duebendorfer T, Plattner B (2008) Firefox (in) security update dynamics exposed. ACM SIGCOMM Comput Commun Rev 39(1):16–22
Frey BS, Goette L (1999) Does pay motivate volunteers? Working paper/Inst Empir Res Econ 7
HackerOne (2018) 118 fascinating facts from hackerone’s hacker-powered security report 2018. https://www.hackerone.com/blog/118-Fascinating-Facts-HackerOnes-Hacker-Powered-Security-Report-2018. (last visited: August 27, 2018)
Harhoff D, Henkel J, Von Hippel E (2003) Profiting from voluntary information spillovers: how users benefit by freely revealing their innovations. Res Pol 32(10):1753–1769
Hata H, Guo M, Babar MA (2017) Understanding the heterogeneity of contributors in bug bounty programs. In: Proc. of the ACM/IEEE int’l symp. on empirical software engineering and measurement, pp 223–228
Izquierdo JLC, Cabot J (2018) The role of foundations in open source projects. In: Proceedings of the 40th international conference on software engineering: software engineering in society, pp 3–12
Kanda T, Guo M, Hata H, Matsumoto K (2017) Towards understanding an open-source bounty: Analysis of Bountysource. In: Int’l conf. on software analysis, evolution and reengineering. IEEE, pp 577–578
Kochhar PS, Thung F, Lo D (2014) Automatic fine-grained issue report reclassification. In: 2014 19th international conference on engineering of complex computer systems. IEEE, pp 126–135
Krishnamurthy S, Tripathi AK (2006) Bounty programs in free/libre/open source software. In: The economics of open source software development. Elsevier, pp 165–183
Krishnamurthy S, Ou S, Tripathi AK (2014) Acceptance of monetary rewards in open source software development. Res Policy 43(4):632–644
Kuhn M, et al. (2008) Building predictive models in r using the caret package. J Stat Softw 28(5):1–26
Lakhani KR, Wolf RG (2003) Why hackers do what they do: Understanding motivation and effort in free/open source software projects
Lee A, Carver JC, Bosu A (2017) Understanding the impressions, motivations, and barriers of one time code contributors to floss projects: a survey. In: IEEE/ACM 39th international conference on software engineering (ICSE), pp 187–197
Maillart T, Zhao M, Grossklags J, Chuang J (2017) Given enough eyeballs, all bugs are shallow? Revisiting Eric Raymond with bug bounty programs. J Cybersec 3(2):81–90
Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5(9):1315–1316
Matt A (2020) Bug bounties won’t make you rich (but you should participate anyway). https://www.techrepublic.com/article/bug-bounties-wont-make-you-rich-but-you-should-participate-anyway/,. (last visited: January 21, 2020)
Mirko Z (2020) Full-time bug hunting:, Pros and cons of an emerging career. https://www.helpnetsecurity.com/2020/04/07/bug-hunting-career/. (April 7, 2020)
Mockus A, Fielding RT, Herbsleb JD (2002) Two case studies of open source software development: Apache and mozilla. ACM Trans Softw Eng Methodol (TOSEM) 11(3):309–346
Moore DS, Kirkland S (2007) The basic practice of statistics, vol 2. WH Freeman New York
Nakasai K, Hata H, Matsumoto K (2018) Are donation badges appealing?: a case study of developer responses to eclipse bug reports. IEEE Softw 36 (3):22–27
Rajbahadur GK, Wang S, Kamei Y, Hassan AE (2019) Impact of discretization noise of the dependent variable on machine learning classifiers in software engineering. IEEE Trans Softw Eng
Robert L (2019) Bug bounties continue to rise, but market has its own 1% problem). https://www.darkreading.com/vulnerabilities---threats/vulnerability-management/bug-bounties-continue-to-rise-but-market-has-its-own-1--problem/d/d-id/1335689
Roberts JA, Hann I-H, Slaughter SA (2006) Understanding the motivations, participation, and performance of open source software developers: a longitudinal study of the apache projects. Manag Sci 52(7):984–999
Robles G, Gonzalez-Barahona JM, Herraiz I (2009) Evolution of the core team of developers in libre software projects. In: 2009 6th IEEE international working conference on mining software repositories. IEEE, pp 167–170
Romano J, Kromrey JD, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: Should we really be using t-test and cohen’s d for evaluating group differences on the nsse and other surveys. In: Annual meeting of the Florida association of institutional research, pp 1–33
Shah SK (2006) Motivation, governance, and the viability of hybrid forms in open source software development. Manag Sci 52(7):1000–1014
Steinmacher I, Silva MAG, Gerosa MA (2014) Barriers faced by newcomers to open source projects: a systematic review. In: IFIP international conference on open source systems. Springer, pp 153–163
Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2016) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Softw Eng 43(1):1–18
Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2018) The impact of automated parameter optimization on defect prediction models. IEEE Trans Softw Eng 45(7):683–711
Tom R (2020) Firefox’s bug bounty in 2019 and into the future. https://blog.mozilla.org/security/2020/04/23/bug-bounty-2019-and-future/,. (last visited: April 23, 2020)
Vasilescu B, Posnett D, Ray B, van den Brand MG, Serebrenik A, Devanbu P, Filkov V (2015) Gender and tenure diversity in github teams. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 3789–3798
Von Hippel E (2007) Horizontal innovation networks—by and for users. Indust Corp Change 16(2):293–315
Von Krogh G, Haefliger S, Spaeth S, Wallin MW (2012) Carrots and rainbows: Motivation and social practice in open source software development. MIS Quart:649–676
Wang S, Chen T-H, Hassan AE (2018) Understanding the factors for fast answers in technical Q&A websites. Empir Softw Eng 23(3):1552–1593
Weiss M (2011) Control and diversity in company-led open source projects. Open Sourc Bus Res, (April 2011)
Ye Y, Kishida K (2003) Toward an understanding of the motivation open source software developers. In: Proceedings of the 25th international conference on software engineering (ICSE), pp 419–429
Zhao M, Grossklags J, Chen K (2014) An exploratory study of white hat behaviors in a web vulnerability disclosure program. In: Proc. of the workshop on security information workers. ACM, pp 51–58
Zhao M, Laszka A, Grossklags J (2017) Devising effective policies for bug-bounty platforms and security vulnerability discovery. J Inf Pol 7:372–418
Zhou J, Wang S, Bezemer C-P, Hassan AE (2020a) Bounties on technical Q&A sites: a case study of stack overflow bounties. Empir Softw Eng 25 (1):139–177
Zhou J, Wang S, Bezemer C-P, Zou Y, Hassan AE (2020b) Studying the association between bountysource bounties and the issue-addressing likelihood of github issue reports. IEEE Trans Softw Eng
Zhou M, Mockus A, Ma X, Zhang L, Mei H (2016) Inflow and retention in oss communities with commercial involvement: a case study of three hybrid projects. ACM Trans Softw Eng Methodol (TOSEM) 25(2): 1–29
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Kelly Blincoe
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work is not related to Jiayuan Zhou’s and Haoxiang Zhang’s roles at Huawei.
Rights and permissions
About this article
Cite this article
Zhou, J., Wang, S., Zhang, H. et al. Studying backers and hunters in bounty issue addressing process of open source projects. Empir Software Eng 26, 81 (2021). https://doi.org/10.1007/s10664-021-09979-z
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-021-09979-z