Ashley Williams
ABSTRACT
Background: Software engineering (SE) research continues to study the degree to which practitioners perceive research as relevant to practice. Such studies typically comprise surveys of practitioner opinions. In a preliminary, and relatively small scale, study of online articles we previously found few explicit citations to software testing research. Our previous study provided an in situ complement to the typical survey study, however the findings of the previous study were limited by the size of our sample.
Objective: To further investigate whether and how practitioners cite software testing research in the grey literature, by using a larger and more diverse dataset.
Method: We analyse four distinct datasets totalling over 400,000 online articles with approx. 2M external citations. Two datasets were generated by crawling predefined domains and two were generated by applying heuristics, developed in prior research, in Google searches. Citations are classified and then analysed.
Results: We find a (very) low percentage of citations to research.
Conclusion: Our replication corroborates our preliminary study and findings from others. In relative terms, topic--specific searches appear to return results that contain articles with more citations. Our results and method provide a basis for benchmarking.
- Kilim Choi. {n. d.}. Software Engineering Blogs. ({n. d.}). https://github.com/kilimchoi/engineering-blogs#-individualsGoogle Scholar
- Premkumar Devanbu, Thomas Zimmermann, and Christian Bird. 2016. Belief & evidence in empirical software engineering. In Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 108--119. Google ScholarDigital Library
- Vahid Garousi, Michael Felderer, and Mika V Mäntylä. 2016. The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering. ACM, 26. Google ScholarDigital Library
- Vahid Garousi and Mika V Mäntylä. 2016. When and what to automate in software testing? A multi-vocal literature review. Information and Software Technology 76 (2016), 92--117. Google ScholarDigital Library
- Vahid Garousi and Junji Zhi. 2013. A survey of software testing practices in Canada. Journal of Systems and Software 86, 5 (2013), 1354--1376. Google ScholarDigital Library
- David Lo, Nachiappan Nagappan, and Thomas Zimmermann. 2015. How practitioners perceive the relevance of software engineering research. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 415--425. Google ScholarDigital Library
- Austen Rainer and Ashley Williams. 2019. Heuristics for improving the rigour and relevance of grey literature searches for software engineering research. Information and Software Technology 106 (2019), 231--233.Google ScholarCross Ref
- Ashley Williams. 2018. Do software engineering practitioners cite research on software testing in their online articles?: A preliminary survey.. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018. ACM, 151--156. Google ScholarDigital Library
- Ashley Williams. 2018. Using reasoning markers to select the more rigorous software practitioners' online content when searching for grey literature. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018. ACM, 46--56. Google ScholarDigital Library
- Ashley Williams and Austen Rainer. 2016. Identifying practitioners' arguments and evidence in blogs: insights from a pilot study. In Software Engineering Conference (APSEC), 2016 23rd Asia-Pacific. IEEE, 345--348.Google ScholarCross Ref
Index Terms
- Do software engineering practitioners cite software testing research in their online articles?: A larger scale replication
Recommendations
Do software engineering practitioners cite research on software testing in their online articles?: A preliminary survey.
EASE '18: Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018Background: Software engineering (SE) research continues to study the degree to which practitioners perceive that SE research has impact on practice. Such studies typically comprise surveys of practitioner opinions. These surveys could be complemented ...
Examining the Transfer of Academic Knowledge to Business Practitioners: Doctoral Program Graduates as Intermediaries
This study explores whether practitioners who hold a Ph.D. in business act as intermediaries in the transfer of academic knowledge from academia to practice. Twenty Ph.D. graduates were interviewed, and the data were subjected to deductive content ...
How video articles are cited, the case of JoVE: Journal of Visualized Experiments
Journal of Visualized Experiments (JoVE) is a peer-reviewed journal that publishes video articles. In order to find out about the impact of video articles and how they are used in other journal articles, a random sample of 500 articles that cited at ...
Comments