Abstract
Over the past three years, our groups at the University of Leuven and the University of Trento have been conducting a number of experimental studies. In particular, two common themes can be easily identified within our work. First, we have investigated the value of several threat modeling and risk assessment techniques. The second theme relates to the problem of preserving security over time, i.e., security evolution. Although the empirical results obtained in our studies are interesting on their own, the main goal of this chapter is to share our experience. The objective is to provide useful, hands-on insight on this type of research work so that the work of other researchers in the community would be facilitated. The contribution of this chapter is the discussion of the challenges we faced during our experimental work. Contextually, we also outline those solutions that worked out in our studies and could be reused in the field by other studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nuseibeh, B.: Weaving together requirements and architectures. IEEE Computer 34, 115–119 (2001)
Heyman, T., Yskout, K., Scandariato, R., Schmidt, H., Yu, Y.: The security twin peaks. In: Erlingsson, Ú., Wieringa, R., Zannone, N. (eds.) ESSoS 2011. LNCS, vol. 6542, pp. 167–180. Springer, Heidelberg (2011)
Howard, M., Lipner, S.: The Security Development Lifecycle. Microsoft Press (2006)
McGraw, G.: Software Security: Building Security. Addison-Wesley (2006)
Lund, M.S., Solhaug, B., Stølen, K.: Model-Driven Risk Analysis: The CORAS Approach. Springer (2011)
Mellado, D., Fernández-Medina, E., Piattini, M.: Applying a security requirements engineering process. In: Gollmann, D., Meier, J., Sabelfeld, A. (eds.) ESORICS 2006. LNCS, vol. 4189, pp. 192–206. Springer, Heidelberg (2006)
Scandariato, R., Wuyts, K., Joosen, W.: A descriptive study of Microsoft’s threat modeling technique. Requirements Engineering (2014)
Labunets, K., Massacci, F., Paci, F., Tran, L.M.: An experimental comparison of two risk-based security methods. In: Proceedings of the 7th International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 163–172 (2013)
Massacci, F., Paci, F., Tran, L.M.S., Tedeschi, A.: Assessing a requirements evolution approach: Empirical studies in the air traffic management domain. Journal of Systems and Software (2013)
Yskout, K., Scandariato, R., Joosen, W.: Change patterns: Co-evolving requirements and architecture. Software and Systems Modeling (2012)
Massacci, F., Paci, F.: How to select a security requirements method? a comparative study with students and practitioners. In: Jøsang, A., Carlsson, B. (eds.) NordSec 2012. LNCS, vol. 7617, pp. 89–104. Springer, Heidelberg (2012)
Opdahl, A.L., Sindre, G.: Experimental comparison of attack trees and misuse cases for security threat identification. Information and Software Technology 51, 916–932 (2009)
Diallo, M.H., Romero-Mariona, J., Sim, S.E., Alspaugh, T., Richardson, D.J.: A comparative evaluation of three approaches to specifying security requirements. In: Proceeding of the 12th International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ (2006)
Hogganvik, I., Stølen, K.: On the comprehension of security risk scenarios. In: Proceedings of the 13th International Workshop on Program Comprehension (IWPC), pp. 115–124. IEEE (2005)
Hogganvik, I., Stølen, K.: A graphical approach to risk identification motivated by empirical investigations. In: Wang, J., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 574–588. Springer, Heidelberg (2006)
Hogganvik, I., Lund, M., Stølen, K.: Reducing the effort to comprehend risk models: Textlabels are often preferred over graphical means. Risk Analysis 51, 916–932 (2009)
Dhillon, D.: Developer-driven threat modeling: Lessons learned in the trenches. IEEE Security & Privacy 9, 41–47 (2011)
Villela, K., Dörr, J., John, I.: Evaluation of a method for proactively managing the evolving scope of a software product line. In: Wieringa, R., Persson, A. (eds.) REFSQ 2010. LNCS, vol. 6182, pp. 113–127. Springer, Heidelberg (2010)
Villela, K., Dörr, J., Gross, A.: Proactively managing the evolution of embedded system requirements. In: Proceeding of the 16th IEEE International Requirements Engineering Conference (RE), pp. 13–22. IEEE Computer Society (2008)
Basili, V., Rombach, H.: The TAME project: Towards improvement-oriented software environments. IEEE Transactions on Software Engineering 14, 758–773 (1988)
McGee, S., Greer, D.: Software requirements change taxonomy: Evaluation by case study. In: Proceeding of the 19th IEEE International Requirements Engineering Conference (RE), pp. 25–34 (2011)
Runeson, P., Host, M.: Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14, 131–164 (2009)
Herrmann, A., Wallnöfer, A., Paech, B.: Specifying changes only — a case study on delta requirements. In: Glinz, M., Heymans, P. (eds.) REFSQ 2009 Amsterdam. LNCS, vol. 5512, pp. 45–58. Springer, Heidelberg (2009)
Ncube, C., Lockerbie, J., Maiden, N.: Automatically generating requirements from i* models: Experiences with a complex airport operations system. In: Sawyer, P., Paech, B., Heymans, P. (eds.) REFSQ 2007. LNCS, vol. 4542, pp. 33–47. Springer, Heidelberg (2007)
Maiden, N., Robertson, S.: Integrating creativity into requirements processes: Experiences with an air traffic management system. In: Proceeding of the 13th IEEE International Requirements Engineering Conference (RE), pp. 105–116 (2005)
Maiden, N.A.M., Jones, S.V., Manning, S., Greenwood, J., Renou, L.: Model-driven requirements engineering: Synchronising models in an air traffic management case study. In: Persson, A., Stirna, J. (eds.) CAiSE 2004. LNCS, vol. 3084, pp. 368–383. Springer, Heidelberg (2004)
Grimes, D., Schulz, K.: Descriptive studies: what they can and cannot do. The Lancet 359, 145–149 (2002)
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 319–340 (1989)
Conover, W.J.: On methods of handling ties in the wilcoxon signed-rank test. Journal of the American Statistical Association 68, 985–988 (1973)
Tran, L.M.S., Massacci, F.: Dealing with known unknowns: Towards a game-theoretic foundation for software requirement evolution. In: Mouratidis, H., Rolland, C. (eds.) CAiSE 2011. LNCS, vol. 6741, pp. 62–76. Springer, Heidelberg (2011)
Tran, L.M.S.: Managing the Uncertainty of the Evolution of Requirements Model. PhD thesis, University of Trento (2014)
Moody, D.L.: The method evaluation model: A theoretical model for validating information systems design methods. In: Proceeding of the European Conference on Information Systems (ECIS), pp. 1327–1336 (2003)
Mens, T., Wermelinger, M., Ducasse, S., Demeyer, S., Hirschfeld, R., Jazayeri, M.: Challenges in software evolution. In: Proceeding of the 8th International Workshop on Principles of Software Evolution, pp. 13–22 (2005)
Si* Tool website: http://sesa.dit.unitn.it/sistar_tool
Topcased UML editor: http://www.topcased.org/
Tichy, W.: Hints for reviewing empirical work in software engineering. Empirical Software Engineering 5, 309–312 (2000)
Carver, J., Jaccheri, L., Morasca, S.: A checklist for integrating student empirical studies with research and teaching goals. Empirical Software Engineering 15, 35–59 (2010)
Runeson, P.: Using students as experiment subjects - an analysis on graduate and freshmen student data. In: Proceeding of the International Conference on Empirical Assessment in Software Engineering (EASE), pp. 95–102 (2003)
van den Berghe, A., Scandariato, R., Joosen, W.: Towards a systematic literature review on secure software design. In: Doctoral Symposium of the International Symposium on Engineering Secure Software and Systems, ESSoS-DS (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Scandariato, R. et al. (2014). Empirical Assessment of Security Requirements and Architecture: Lessons Learned. In: Heisel, M., Joosen, W., Lopez, J., Martinelli, F. (eds) Engineering Secure Future Internet Services and Systems. Lecture Notes in Computer Science, vol 8431. Springer, Cham. https://doi.org/10.1007/978-3-319-07452-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-07452-8_2
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07451-1
Online ISBN: 978-3-319-07452-8
eBook Packages: Computer ScienceComputer Science (R0)