Abstract
We present a theory of security testing based on the basic distinction between system specifications and security requirements. Specifications describe a system’s desired behavior over its interface. Security requirements, in contrast, specify desired properties of the world the system lives in. We propose the notion of a security rationale, which supports reductive security arguments for deriving a system specification and assumptions on the system’s environment sufficient for fulfilling stated security requirements. These reductions give rise to two types of tests: those that test the system with respect to its specification and those that test the validity of the assumptions about the adversarial environment. It is the second type of tests that distinguishes security testing from functional testing and defies systematization and automation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We will abstract away from further temporal aspects in this example. For instance, once the door has been closed, it remains closed until the next signal arrives, and only one person can pass through the door while it is open.
References
Abrial, J.-R.: Modeling in Event-B: System and Software Engineering, 1st edn. Cambridge University Press, New York (2010)
Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, New York (2008)
Basin, D.A., Capkun, S.: The research value of publishing attacks. Commun. ACM 55(11), 22–24 (2012)
Beizer, B.: Software Testing Techniques, 2nd edn. Van Nostrand Reinhold, New York (1990)
Bjorner, D.: Software Engineering 3: Domains, Requirements, and Software Design. Texts in Theoretical Computer Science. An EATCS Series. Springer, New York (2006)
BSI. A penetration testing model, The German Federal Office for Information Security(2003)
BSI. IT Grundschutz Kataloge, (Version: 14). The German Federal Office for Information Security (2014)
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Comput. 11(4), 34–41 (1978)
Dijkstra, E.W.: Notes on structured programming. Technical report T.H. Report 70-WSK-03, Technological University Eindhoven, April 1970
Glinz, M.: On non-functional requirements. In: 15th IEEE International Requirements Engineering Conference, RE, pp. 21–26. IEEE Computer Society (2007)
Godefroid, P., Levin, M.Y., Molnar, D.A.: SAGE: whitebox fuzzing for security testing. ACM Queue 10(1), 20 (2012)
Jackson, D.: A direct path to dependable software. Commun. ACM 52(4), 78–88 (2009)
Jackson, M.: The world and the machine. In: Proceedings of the 17th International Conference on Software Engineering, ICSE 1995, pp. 283–292. ACM, New York, NY, USA (1995)
Jackson, M.: Problem Frames. Addison-Wesley, Reading (2001)
Johnson, A.: Hitting the Brakes: Engineering Design and the Production of Knowledge. Duke University Press, London (2009)
Kim, Y., Daly, R., Kim, J., Fallin, C., Lee, J.-H., Lee, D., Wilkerson, C., Lai, K., Mutlu, O.: Flipping bits in memory without accessing them: an experimental study of DRAM disturbance errors. In: ACM/IEEE 41st International Symposium on Computer Architecture, ISCA, pp. 361–372. IEEE Computer Society (2014)
Levin, R., Cohen, E., Corwin, W., Pollack, F., Wulf, W.: Policy/mechanism separation in Hydra. SIGOPS Oper. Syst. Rev. 9(5), 132–140 (1975)
McGraw, G.: Software Security: Building Security In. Addison-Wesley Professional, Boston (2006)
Michael, C.C., van Wyk, K., Radosevich, W.: Risk-based and functional security testing, Accessed 05 July 2013. https://buildsecurityin.us-cert.gov/
Moloney, E.: A Secret History of IRA. Penguin, Canada (2003)
Myers, G., Sandler, C., Badgett, T.: The Art of Software Testing, 3rd edn. Wiley, New York (2011)
Nelson, R.: What is a secret - and - what does that have to do with computer security? In: Proceedings of the Workshop on New Security Paradigms, pp. 74–79. IEEE (1994)
Derived test requirements for FIPS PUB 140–2, security requirements for cryptographic modules, NIST, CSEC and CMVP Laboratories Draft (2011)
OWASP. Testing guide v. 4, Accessed on 9 March 2014. https://www.owasp.org
Reiter, R.: On closed world data bases. In: Gallaire, H., Minke, J. (eds.) Logic and Data Bases, pp. 55–76. Plenum Press, New York (1978)
Herbert, A.: Simon.: The architecture of complexity. Proc. Am. Philos. Soc. 106(6), 467–482 (1962)
Takanen, A., DeMott, J., Miller, C.: Fuzzing for Software Security Testing and Quality Assurance, 1st edn. Artech House Inc., Norwood (2008)
Tsankov, P., Dashti, M.T., Basin, D.A.: Semi-valid input coverage for fuzz testing. In: International Symposium on Software Testing and Analysis, ISSTA, pp. 56–66. ACM (2013)
Voas, J., McGraw, G.: Software Fault Injection. Wiley, New York (1998)
Wang, R., Zhou, Y., Chen, S., Qadeer, S., Evans, D., Gurevich, Y.: Explicating SDKs: Uncovering assumptions underlying secure authentication and authorization. In: Proceedings of the 22nd USENIX Conference on Security, pp. 399–414 (2013)
Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)
Acknowledgment
We thank Peter Müller and Petar Tsankov for their comments on this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Dashti , M.T., Basin, D. (2016). Security Testing Beyond Functional Tests. In: Caballero, J., Bodden, E., Athanasopoulos, E. (eds) Engineering Secure Software and Systems. ESSoS 2016. Lecture Notes in Computer Science(), vol 9639. Springer, Cham. https://doi.org/10.1007/978-3-319-30806-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-30806-7_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30805-0
Online ISBN: 978-3-319-30806-7
eBook Packages: Computer ScienceComputer Science (R0)