Abstract
Formal methods have been largely thought of in the context of safety-critical systems, where they have achieved major acceptance. Tens of millions of people trust their lives every day to such systems, based on formal proofs rather than “we haven’t found a bug” (yet!); but why is “we haven’t found a bug” an acceptable basis for systems trusted with hundreds of millions of people’s personal data?
This paper looks at some of these issues in cybersecurity, and the extent to which formal methods, ranging from “fully verified” to better tool support, could help. More importantly, recent policy reports and curricula initiatives appear to recommended formal methods in the limited context of “safety critical applications”; we suggest this is too limited in scope and ambition. Not only are formal methods needed in cybersecurity, the repeated and very public weaknesses of the cybersecurity industry provide a powerful motivation for formal methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The precise definition of cybersecurity is debatable: we can take is as failures of security, generally defined as “preserving the CIA—Confidentiality, Integrity and Availability” of digital information, where computer system played a critical part in the failure.
- 2.
Generally called “Marriott”, but in fact due to the Starwood chain before Marriott took it over.
- 3.
In military parlance, Equifax is being found not to have “defence in depth”. Defence in depth is certainly valuable: [8] described how Google was saved from the consequences of an ‘awesome’ attack on gmail by defence in depth. But the front line is still the first defence: in this case correct code.
- 4.
- 5.
Actually, Embedded Systems are a comparatively neglected, but important, cybersecurity area. See, for example, [32] for a description of a pervasive design fault in the “home security” market.
- 6.
A point made in the context of XP and Agile in 2004 [33].
References
Jacquel, M., Berkani, K., Delahaye, D., Dubois, C.: Verifying B proof rules using deep embedding and automated theorem proving. In: Barthe, G., Pardo, A., Schneider, G. (eds.) SEFM 2011. LNCS, vol. 7041, pp. 253–268. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24690-6_18
Bloomberg: Equifax Hack Lasted for 76 Days, Compromised 148 Million People, Government Report Says (2018). http://fortune.com/2018/12/10/equifax-hack-lasted-for-76-days-compromised-148-million-people-government-report-says/
Irwin, L.: Marriott downgrades severity of 2018 data breach: 383 million customers affected (2019). https://www.itgovernance.co.uk/blog/marriott-downgrades-severity-of-2018-data-breach-383-million-customers-affected
Ford, N.: Medical debt collection agency files for bankruptcy protection after data breach (2019). https://www.itgovernance.co.uk/blog/medical-debt-collection-agency-files-for-bankruptcy-protection-after-data-breach
The Guardian: BA faces & #x00A3;183m fine over passenger data breach (2019). https://www.theguardian.com/business/2019/jul/08/ba-fine-customer-data-breach-british-airways
Royal Society: Progress and research in cybersecurity: supporting a resilient and trustworthy system for the UK (2016). http://royalsociety.org/cybersecurity
United States Government Accountability Office: Actions Taken by Equifax and Federal Agencies in Response to the 2017 Breach (2018). https://www.gao.gov/assets/700/694158.pdf
Osborne, C.: Google patches ‘awesome’ XSS vulnerability in Gmail dynamic email feature (2019). https://www.zdnet.com/article/google-patches-awesome-xss-vulnerability-in-gmail/
Lenart, L.: Security Bulletin S2-045 (2017). https://cwiki.apache.org/confluence/display/WW/S2-045
Open Web Application Security Project (OWASP): The Ten Most Critical Web Application Security Risks (2017). https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project#tab=Main
Livshits, V., Lam, M.: Finding security vulnerabilities in Java applications with static analysis. In: Proceedings USENIX Security Symposium, pp. 271–286 (2005)
McGraw, G.: Software Security—Building Security In. Addison-Wesley, Boston (2006)
Payment Card Industry Security Standards Council (PCI SSC): Requirements and Security Assessment Procedures Version 3.2.1 (2018). https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-2-1.pdf
Ponemon Institute: The State of Web Application Firewalls. Ponemon Institute (2019)
Krebs, B.: What We Can Learn from the Capital One Hack (2019). https://krebsonsecurity.com/tag/capital-one-breach/
Kolochenko, I.: Web Application Firewall: a must-have security control or an outdated technology? (2016). https://www.csoonline.com/article/3032743/web-application-firewall-a-must-have-security-control-or-an-outdated-technology.html
Barth, B.: No fly-by-night operation: Researchers suspect Magecart group behind British Airways breach (2018). https://www.scmagazine.com/home/security-news/no-fly-by-night-operation-researchers-suspect-magecart-group-behind-british-airways-breach/
Klein, G., et al.: seL4: formal verification of an OS kernel. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 207–220 (2009)
The Guardian: Hacking risk leads to recall of 500,000 pacemakers due to patient death fears (2017). https://www.theguardian.com/technology/2017/aug/31/hacking-risk-recall-pacemakers-patient-death-fears-fda-firmware-update
Newman, L.: Hackers Made an App That Kills to Prove a Point (2019). https://www.wired.com/story/medtronic-insulin-pump-hack-app
Evans, M., Loftus, P.: Rattled by Cyberattacks, Hospitals Push Device Makers to Improve Security (2019). https://www.wsj.com/articles/rattled-by-cyberattacks-hospitals-push-device-makers-to-improve-security-11557662400
Food and Drug Administration: FDA Informs Patients, Providers and Manufacturers About Potential Cybersecurity Vulnerabilities in Certain Medical Devices with Bluetooth Low Energy (2020). https://www.fda.gov/news-events/press-announcements/fda-informs-patients-providers-and-manufacturers-about-potential-cybersecurity-vulnerabilities-0
Heiser, G.: What’s new in the world of seL4 (2019). https://archive.fosdem.org/2019/schedule/event/world_of_sel4/
Rescorla, E.: The Transport Layer Security (TLS) Protocol Version 1.3. RFC-8446 (2018)
Beck, K., et al.: The Agile Manifesto (2001). http://agilemanifesto.org/
Blodget, H.: Mark Zuckerberg on Innovation (2009). https://www.businessinsider.com/mark-zuckerberg-innovation-2009-10
Lane, A.: Security + Agile = FAIL (2018). https://securosis.com/assets/library/presentations/Security/AgileFAIL_OWASP.ppt_.pdf
Bartsch, S.: Practitioners’ perspectives on security in agile development. In: International Conference on Availability Reliability and Security, pp. 479–484 (2011)
van der Heijden, A., Broasca, C., Serebrenik, A.: An empirical perspective on security challenges in large-scale agile software development. In: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018, pp. 45:1–45:4. ACM, New York (2018)
Tahaei, M., Vaniea, K.: A Survey on Developer-Centred Security (2019). https://groups.inf.ed.ac.uk/tulips/papers/A_Survey_on_Developer_Centred_Security.pdf
Chapman, R.: Industrial experience with Agile in high-integrity software development. In: Parsons, M., Anderson, T. (eds.) Developing Safe Systems: Proceedings of the Twenty-fourth Safety-critical Systems Symposium, Safety-Critical Systems Club, pp. 143–154 (2016)
O’Connor, T., Enck, W., Reaves, B.: Blinded and confused: uncovering systemic flaws in device telemetry for smart-home Internet of Things. In: Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, pp. 140–150 (2019)
Wäyrynen, J., Bodén, M., Boström, G.: Security engineering and eXtreme programming: an impossible marriage? In: Zannier, C., Erdogmus, H., Lindstrom, L. (eds.) XP/Agile Universe 2004. LNCS, vol. 3134, pp. 117–128. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-27777-4_12
Statt, N.: Zuckerberg: ‘Move fast and break things’ isn’t how Facebook operates anymore (2014). https://www.cnet.com/news/zuckerberg-move-fast-and-break-things-isnt-how-we-operate-anymore/
Salz, R.: Software engineering and OpenSSL is not an oxymoron (presentation at Real World Cryptography 2017) (2017). https://rwc.iacr.org/2017/Slides/rich.saltz.pdf
Seggelmann, R., Tuexen, M., Williams, M.: Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) Heartbeat Extension (2012). https://tools.ietf.org/html/rfc6520
Brain, M., Schanda, F.: A lightweight technique for distributed and incremental program verification. In: Joshi, R., Müller, P., Podelski, A. (eds.) VSTTE 2012. LNCS, vol. 7152, pp. 114–129. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27705-4_10
Chapman, R., Moy, Y.: AdaCore Technologies for Cyber Security (2018). https://www.adacore.com/books/adacore-tech-for-cyber-security
Distefano, D., Fähndrich, M., Logozzo, F., O’Hearn, P.: Scaling static analyses at Facebook. Commun. ACM 62, 62–70 (2019)
Vogels, W.: Proving security at scale with automated reasoning (2019). https://www.allthingsdistributed.com/2019/05/proving-security-at-scale-with-automated-reasoning.html
Sadowski, C., Aftandilian, E., Eagle, A., Miller-Cushion, L., Jaspan, C.: Lessons from building static analysis tools at Google. Commun. ACM 61(4), 58–66 (2018)
Open Web Application Security Project (OWASP): DefectDojo: OpenSource Application Security Management (2019). https://www.defectdojo.org
Chapman, R.: Development and Formal Verification of Secure Updates for Embedded Systems (slides from Verification 2018) (2018). http://www.testandverification.com/conferences/verification-futures/vf2018/
Google (Project Zero): 0day “In the Wild” (2019). https://googleprojectzero.blogspot.com/p/0day.html
Thomas, G.: A proactive approach to more secure code (2019). https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/
Centre for the Protection of National Infrastructure: Rail Code of Practice for Security-Informed Safety. CPNI (2019)
ISO/IEC: TS 17961:2013, Information technology—Programming languages, their environments & system software interfaces—C Secure Coding Rules (2013). https://www.iso.org/standard/61134.html
Chong, N., et al.: Code-level model checking in the software development workflow. In: ICSE-SEIP 2020 (2020, to appear)
Cavalcanti, A., Miyazawa, A., Wellings, A., Woodcock, J., Zhao, S.: Java in the safety-critical domain. In: Bowen, J.P., Liu, Z., Zhang, Z. (eds.) SETSS 2016. LNCS, vol. 10215, pp. 110–150. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56841-6_4
Meng, N., Nagy, S., Yao, D., Zhuang, W., Arango Argoty, G.: Secure coding practices in Java: challenges and vulnerabilities. In: 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pp. 372–383 (2018)
Google (Chris Povirk): Denial of Service vulnerability for servers that use Guava and deserialize attacker data (2018). https://groups.google.com/forum/#!topic/guava-announce/xqWALw4W1vs/discussion
Guarnieri, S., Livshits, B.: GATEKEEPER: mostly static enforcement of security and reliability policies for JavaScript code. In: USENIX Security Symposium, vol. 10, pp. 76–85 (2009)
Meyerovich, L., Livshits, B.: ConScript: specifying and enforcing fine-grained security policies for JavaScript in the browser. In: 2010 IEEE Symposium on Security and Privacy, pp. 481–496. IEEE (2010)
Maffeis, S., Mitchell, J.C., Taly, A.: Isolating JavaScript with filters, rewriting, and wrappers. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 505–522. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04444-1_31
Maffeis, S., Taly, A.: Language-based isolation of untrusted JavaScript. In: Proceedings 22nd IEEE Computer Security Foundations Symposium, pp. 77–91 (2009)
Kotowicz, K.: Trusted Types help prevent Cross-Site Scripting (2019). https://developers.google.com/web/updates/2019/02/trusted-types
Cable, J.: Every Computer Science Degree Should Require a Course in Cybersecurity (2019). https://hbr.org/2019/08/every-computer-science-degree-should-require-a-course-in-cybersecurity
Brown, N.C.C., Sentance, S., Crick, T., Humphreys, S.: Restart: the resurgence of computer science in UK schools. ACM Trans. Comp. Sci. Edu. 14(2), 1–22 (2014). https://doi.org/10.1145/2602484
Davenport, J.H., Crick, T., Hourizi, R.: The institute of coding: a university-industry collaboration to address the UK’s digital skills crisis. In: Proceedings of IEEE Global Engineering Education Conference, pp. 1400–1408. IEEE Press (2020). https://doi.org/10.1109/EDUCON45650.2020.9125272
Davenport, J.H., Hayes, A., Hourizi, R., Crick, T.: Innovative pedagogical practices in the craft of computing. In: Proceedings of 4th International Conference on Learning and Teaching in Computing and Engineering (2016). https://doi.org/10.1109/LaTiCE.2016.38
Murphy, E., Crick, T., Davenport, J.H.: An analysis of introductory programming courses at UK universities. Art Sci. Eng. Prog. 1(2), 18 (2017). https://doi.org/10.22152/programming-journal.org/2017/1/18
Crick, T., Davenport, J.H., Hanna, P., Irons, A., Prickett, T.: Overcoming the challenges of teaching cybersecurity in UK computer science degree programmes. In: Proceedings of 50th Annual Frontiers in Education Conference, IEEE Press (2020). https://doi.org/10.1109/FIE44824.2020.9274033
Crick, T., Davenport, J., Irons, A., Prickett, T.: A UK case study on cybersecurity education and accreditation. In: Proceedings of FIE 2019 (2019)
Crick, T., Davenport, J.H., Hanna, P., Irons, A., Pearce, S., Prickett, T.: Repositioning BCS degree accreditation. ITNOW 62(1), 50–51 (2020). https://doi.org/10.1093/itnow/bwaa023
Naiakshina, A., Danilova, A., Tiefenau, C., Smith, M.: Deception task design in developer password studies: exploring a student sample. In: Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), pp. 297–313. USENIX Association (2018)
Naiakshina, A., Danilova, A., Gerlitz, E., von Zezschwitz, E., Smith, M.: “If you want, I can store the encrypted password”: a password-storage field study with freelance developers. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 140:1–140:12. ACM (2019)
Cimpanu, C.: 7-Eleven Japanese customers lose \$500,000 due to mobile app flaw (2019). https://www.zdnet.com/article/7-eleven-japanese-customers-lose-500000-due-to-mobile-app-flaw/
Biscoe, C.: MyFitnessPal data breach: 150 million app users affected (2018). https://www.itgovernance.co.uk/blog/myfitnesspal-data-breach-150-million-app-users-affected/
Blackmon, A.: Macy’s hit by data breach (2018). https://eu.freep.com/story/money/business/2018/07/06/macys-data-breach-online/763074002/
Inbenta (CEO): Inbenta and the Ticketmaster Data Breach (2018). http://web.archive.org/web/20181121184620/
Taylor, C., Sakharkar, S.: ’);DROP TABLE textbooks;–: an argument for SQL injection coverage in database textbooks. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE 2019), pp. 191–197. ACM (2019)
Fischer, F., et al.: Stack overflow considered harmful? The impact of copy&paste on Android application security. In: 38th IEEE Symposium on Security and Privacy (SP), pp. 121–136 (2017)
Open Web Application Security Project (OWASP): The Ten Most Critical Web Application Security Risks (2013). https://www.owasp.org/images/f/f8/OWASP_Top_10_-_2013.pdf
Fischer, F., et al.: Stack overflow considered helpful! Deep learning security nudges towards stronger cryptography. In: 28th USENIX Security Symposium (USENIX Security 2019), pp. 339–356 (2019)
Chen, M., Fischer, F., Meng, N., Wang, X., Grossklags, J.: How reliable is the crowdsourced knowledge of security implementation? https://arxiv.org/abs/1901.01327 (2019)
Zhang, M., Meng, W., Lee, S., Lee, B., Xing, X.: All Your Clicks Belong to Me: Investigating Click Interception on the Web (2019). https://www.microsoft.com/en-us/research/uploads/prod/2019/03/zhang-observer.pdf
Austin, A., Williams, L.: One technique is not enough: a comparison of vulnerability discovery techniques. In: Proceedings 2011 International Symposium on Empirical Software Engineering and Measurement, pp. 97–106 (2011)
Acknowledgements
A predecessor of this paper was given at the 2019 Working Formal Methods Symposium (FROM2019) in Timișoara, Romania. The authors are grateful to the referees and audiences of FROM2019 and FMFun2019 for useful comments. The first author is grateful to the Fulbright Programme for a Cybersecurity Scholarship at New York University in 2017, and to many correspondents and discussions, notably Alastair Irons, Tom Prickett and Tim French. This paper was partially supported by the Institute of Coding, which received £20m of funding from the Office for Students (OfS), as well as support from the Higher Education Funding Council for Wales (HEFCW).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Davenport, J.H., Crick, T. (2021). Cybersecurity Education and Formal Methods. In: Cerone, A., Roggenbach, M. (eds) Formal Methods – Fun for Everybody. FMFun 2019. Communications in Computer and Information Science, vol 1301. Springer, Cham. https://doi.org/10.1007/978-3-030-71374-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-71374-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71373-7
Online ISBN: 978-3-030-71374-4
eBook Packages: Computer ScienceComputer Science (R0)